More than 2,000 scientists back petition to raise awareness of the deveastating effects of a rule that specifies that if a grant proposal is not funded on the first submission, only one revision can be filed with the same specific aims.
According to National Institutes of Health policy, “Beginning with original new applications (i.e., never submitted) and competing renewal applications submitted for the January 25, 2009 due dates and beyond, the NIH will accept only a single amendment to the original application. Failure to receive funding after two submissions (i.e., the original and the single amendment) will mean that the applicant should substantially re-design the project rather than simply change the application in response to previous reviews. It is expected that this policy will lead to funding high quality applications earlier, with fewer resubmissions.”
At the present time, several NIH institutes are unable to fund applications unless they obtain scores of 7th or 8th percentile or better. Thus, even outstanding applications that obtain a 10th percentile ranking on second resubmission may not be funded. The following petition was circulated by Robert Benezra of Memorial Sloan-Kettering Cancer Center to raise awareness of this problem. Over 2,000 signatures were included on the petition below, which was sent to NIH director Francis Collins on Feb. 21, 2011.
Dear Drs. Collins and Scarpa,
I am writing on behalf of 2,356 U.S. scientists (listed below) committed to changing a new NIH policy that we believe will have an overwhelming negative impact on biomedical research in this country. The rule in question specifies that if a grant proposal is not funded on the first submission, only one revision can be filed with the same specific aims. If that revision is not funded, the proposal must be “substantially” changed. As we understand it, the rule was adopted to discourage “serial resubmitters” and was based on the observation that success rates of grants poorly scored in the first round did not benefit substantially from a second resubmission. In addition, it was designed to discourage the implicit “queuing” system, whereby poorer second revision applications (A2s) displaced A0s and A1s that were just “waiting in line” for funding. While such a policy could make sense in an era of reasonable pay lines, with the projected budgets rumored to be funding at the 7th percentile in some institutes, this could have a drastic and we would argue devastating effect on the biomedical research efforts in this country.
Consider the following:
The premise of our argument is based on the fact that all of us who have sat on study sections know that we cannot distinguish a 20th percentile grant (13 points from the hypothetical pay line) from a 5th percentile grant (which now is just in the fundable range). It is simply beyond the limit of resolution of the process. We are not after all just evaluating the impact or validity of a scientific finding or theory (as difficult as that can be), but the projected trajectory of some early findings, a process which is fraught with extraordinary uncertainties in fields as complex as ours.
Where then is the evidence that the majority of A1 applications that just missed the 7th percentile pay line (indistinguishable in quality from other A1s in that cycle that were funded) but were eventually funded as A2s, are not of great value and should be eliminated? So thoroughly flawed in fact that it is better to eliminate them entirely rather than displace (in their A2 submission round) some A0s and A1s into the next cycle? This argument is particularly worrisome if in fact the economic crisis ever abates even incrementally during a period when many potential meritorious A2s are being discarded from the pool.
Also, we have been told that a measure of success of the new policy is a noticeable increase in the fraction of successsful applications that are A0s. But this only makes sense if the majority of those funded A0s are derived from A2s that were forced to write “substantially new” and better applications. It is equally likely this results from the fact that there are fewer outstanding A2s in the pool that were eliminated essentially by chance. Trivially put, if no resubmissions were allowed then of course all funded applications would be A0s. Is that the goal of the new policy?
The fraction of A0 (new, unamended), A1, and A2 applications within the pool of funded R01 grants for the Fiscal Years 1997-2007. Note the dramatic decrease in the pool of A0 applications from approximately 60% to less than 30%. From NIGMS Feedback Loop.
Therefore, we believe this new rule will have the consequence in the current funding climate of redirecting the efforts of many of our very best scientists on the basis of what will essentially be an arbitrary criterion.
The rule will have a disproportionately negative impact on young investigators with early stage and therefore less diverse programs (particularly those at the stage of their first renewal), or more senior investigators who also have more narrowly focused programs. How can a young investigator, for example, who is just starting to build their program “substantially” change their aims when they have to focus their efforts on a very limited number of projects undertaken with limited funds and staff? These investigators are often hired by senior faculty on the strength of their first proposals in intensely competitive job searches. To be told they must change their focus on the basis of applications that fail despite being ranked better than 90 percent of grants submitted, seems at odds with all of our objectives. And worse, it is likely to be profoundly discouraging and destructive.
As a result of these considerations, we are urging you to return to the two-revision system at least for the subset of applications that cross a certain threshold in scoring as A1s. Certainly a metric can be found which would identify the threshold that would be the most beneficial using currently available statistics. Many of us would be willing to participate in that discussion.
We understand that even if these changes are implemented, many outstanding proposals will still not get funding solely because of budgetary constraints. And we as a group are contemplating ideas to help address that issue as well. We nevertheless believe the change in revision policy advocated here would allow for a much fairer assessment of the research proposals being generated by the best and the brightest investigators in our country.
The Office of Extramural Research response to this petition can be found here.
In what we sincerely hope will be a transition period back to some semblance of the two revision system for grant submissions, investigators submitting “new” proposals now need more guidance on what constitutes a substantially revised application. We have read the CSR’s “Evaluation of Unallowable Resubmission and Overlapping Applications” but find that Program Officers themselves are not sure what rule to follow in certain circumstances. If for example an unfunded A1 has two aims that are considered to be outstanding with a weaker third aim, are we to understand that one of the outstanding aims cannot be pursued unless substantially changed even if conceptually intertwined with the other? We have heard things from “51 percent different”, “Change the tissue or cell type you are working on,” “Any aim included in either the first application or revision cannot be included,” and “If you are working on potassium channels, switch to calcium.” It would be helpful to have clear, unequivocal and sensible guidance on this point very soon as “new” proposals are being prepared by a large number of investigators at this time whose careers depend on these applications.
We urge you to give this petition serious consideration and look forward to your response.
Please note that any e-mail responses to this petition received from your offices will be forwarded to all of the signers below. Validation of each individual’s willingness to sign will be provided upon request. Signatures were collected from 2/11/2011 to 2/17/2011 in response to a mailing of an earlier, incomplete draft to a list of 39 original recipients.