More than 2,000 scientists back petition to raise awareness of the deveastating effects of a rule that specifies that if a grant proposal is not funded on the first submission, only one revision can be filed with the same specific aims.
According to National Institutes of Health policy, “Beginning with original new applications (i.e., never submitted) and competing renewal applications submitted for the January 25, 2009 due dates and beyond, the NIH will accept only a single amendment to the original application. Failure to receive funding after two submissions (i.e., the original and the single amendment) will mean that the applicant should substantially re-design the project rather than simply change the application in response to previous reviews. It is expected that this policy will lead to funding high quality applications earlier, with fewer resubmissions.”
At the present time, several NIH institutes are unable to fund applications unless they obtain scores of 7th or 8th percentile or better. Thus, even outstanding applications that obtain a 10th percentile ranking on second resubmission may not be funded. The following petition was circulated by Robert Benezra of Memorial Sloan-Kettering Cancer Center to raise awareness of this problem. Over 2,000 signatures were included on the petition below, which was sent to NIH director Francis Collins on Feb. 21, 2011.
Dear Drs. Collins and Scarpa,
I am writing on behalf of 2,356 U.S. scientists (listed below) committed to changing a new NIH policy that we believe will have an overwhelming negative impact on biomedical research in this country. The rule in question specifies that if a grant proposal is not funded on the first submission, only one revision can be filed with the same specific aims. If that revision is not funded, the proposal must be “substantially” changed. As we understand it, the rule was adopted to discourage “serial resubmitters” and was based on the observation that success rates of grants poorly scored in the first round did not benefit substantially from a second resubmission. In addition, it was designed to discourage the implicit “queuing” system, whereby poorer second revision applications (A2s) displaced A0s and A1s that were just “waiting in line” for funding. While such a policy could make sense in an era of reasonable pay lines, with the projected budgets rumored to be funding at the 7th percentile in some institutes, this could have a drastic and we would argue devastating effect on the biomedical research efforts in this country.
Consider the following:
The premise of our argument is based on the fact that all of us who have sat on study sections know that we cannot distinguish a 20th percentile grant (13 points from the hypothetical pay line) from a 5th percentile grant (which now is just in the fundable range). It is simply beyond the limit of resolution of the process. We are not after all just evaluating the impact or validity of a scientific finding or theory (as difficult as that can be), but the projected trajectory of some early findings, a process which is fraught with extraordinary uncertainties in fields as complex as ours.
Where then is the evidence that the majority of A1 applications that just missed the 7th percentile pay line (indistinguishable in quality from other A1s in that cycle that were funded) but were eventually funded as A2s, are not of great value and should be eliminated? So thoroughly flawed in fact that it is better to eliminate them entirely rather than displace (in their A2 submission round) some A0s and A1s into the next cycle? This argument is particularly worrisome if in fact the economic crisis ever abates even incrementally during a period when many potential meritorious A2s are being discarded from the pool.
Also, we have been told that a measure of success of the new policy is a noticeable increase in the fraction of successsful applications that are A0s. But this only makes sense if the majority of those funded A0s are derived from A2s that were forced to write “substantially new” and better applications. It is equally likely this results from the fact that there are fewer outstanding A2s in the pool that were eliminated essentially by chance. Trivially put, if no resubmissions were allowed then of course all funded applications would be A0s. Is that the goal of the new policy?