Blotter

ASBMB comments on fairness in grant applications and peer review

In letter to NIH Center for Scientific Review, society recommends changes to reduce bias in study sections and programs for young scientists
Mallory Smith
April 14, 2022

The American Society for Biochemistry and Molecular Biology on Monday sent a letter to the National Institutes of Health Center for Scientific Review in which it advocated for changes aimed at reducing bias during the evaluation of research proposals and changes that would level the playing field for training grant applicants.

The letter was submitted in response to two matters discussed March 28 at a meeting of the CSR Advisory Council: recent bias training initiatives and ways to improve the review of fellowship applications.

In short, the ASBMB told the CSR that (A) it strongly supports the implementation of bias training at every level of peer review and that (B) it approves of the NRSA Working Group’s thoughtful recommendations— with a few tweaks. 

Bias training piloted

The CSR rolled out its “Bias Awareness in Review” training program in the fall.

Director Noni Byrnes said at the advisory council meeting that the program had a lot of successes as well as other “places to adjust” for the next round. Byrnes also emphasized that bias training “has to become mandatory” in the future.

In addition, Hope Cummings, a senior social science analyst at CSR, shared data collected over the past year about reviewers’ impressions of bias during review.

Strikingly, she reported that “the great majority of reviewers believe the NIH has a problem with bias in review” and “most reviewers (53%) never or rarely intervened when they thought bias was present.”

The advisory council recommended that CSR:

  1. Establish bias training as a requirement to participate in the review process.

  2. Create additional programming on implicit bias and other biases.

  3. Empower NIH scientific review officers and chairs of review panels to intervene when biased comments or assessments are made.

  4. Ensure that the framework for intervening in and reporting bias circumvents the power differentials present between review panel members. 

In its letter this week to CSR, the ASBMB emphasized that bias training should be mandatory for everyone involved in peer review — “all reviewers, chairs and scientific review officers” — and that the training should be “offered recurrently to facilitate sustained and enhanced awareness of bias.”

The society also recommended “a thoughtful review of feedback from the voluntary participants and incorporating it into new material that can enhance the curriculum, such as providing more examples of implicit and other types of bias and intervention tools for reviewers to use to counteract bias without fear of retaliation.”

The ASBMB has been vocal about instilling diversity, equity, and inclusion during scientific review, as exemplified by its recent recommendations to the CSR strategic plan.

Adjusting fellowship application criteria

At the advisory council meeting, Deputy Director Bruce Reed explained that CSR had heard “persistent concerns that fellowship review may disadvantage some highly qualified applicants,” leading CSR to convene a working group on fellowship review to “make recommendations to make the process as fair and effective as possible for all.”

According to CSR, review of NIH fellowship applications should “emphasize the candidate’s potential for a productive career, the candidate’s need for the proposed training, and the degree to which the research project and training plan, the sponsor(s), and the environment will satisfy those needs.”

Under this definition, the applicant’s scientific environment had become a significant factor in the evaluation criteria, leading to research-intensive universities having a leg up in securing fellowship applications under the current review criteria. Furthermore, research has shown that fellowship awardees do not always outperform their peers without fellowship awards.  

The working group recommended the following:

  1. Eliminate grades as indicators of qualifications.

  2. Eliminate the “Sponsor/Collaborator” and “Institutional Environment/Commitment to Training” criteria.

  3. Allow an optional statement of special circumstances (with an option to have the school submit a separate letter).

  4. Explicitly allow a wider range of career paths in fellowship training.

  5. Encourage a statement of qualifications that extends beyond grades and publications.

  6. Have review criteria that measure the total impact that the fellowship can bring to the individual and reduce the importance of external factors, such as mentor, institution, etc.

The working group also presented a “delta” concept that aims to provide a more holistic approach to review, reducing the emphasis on the sponsor and institution and focusing more on the applicant’s potential to succeed under the fellowship.

Elizabeth Villa, chair of the working group, explained that they were aiming to capture “the idea of casting a wider net to try to find the diamonds in the rough — incredible scientists with incredible potential that might have less strong conventional backgrounds.”

To accomplish this, she stated that they would “ask reviewers to try to not focus only on their accomplishments so far but also how likely it is that they will substantially benefit from the proposed fellowship — (to) pay a lot of attention to what they talk about their determination persistence creativity and otherwise potential to do incredibly well.”

The ASBMB already had made many of the above recommendations to improve fellowship review on Jan. 24. The society noted in its letter to CSR on Monday that it was pleased the working group had incorporated many of its suggestions, such as eliminating grades as an evaluation criterion since “grades are not an accurate measurement of success.”

However the society expressed concern about removing the sponsor/collaborator section altogether because “mentorship does play an important role in trainee success.” It recommended that the section be replaced with “a detailed mentorship plan, similar to the mentorship plan required by NSF fellowship applications.”

The ASBMB also had suggestions for how to implement an optional statement of special circumstances, which would give applicants the opportunity to explain setbacks from COVID-19, gaps in their training or transitions between labs. The society asked CSR to provide this option on future applications in a very intentional and clear format so that applicants “1) understand what qualifies as a special circumstance and 2) feel reassured that explaining certain special circumstances, such as harassment, will not penalize them in any way.”

Overall, the society was encouraged by the progress shared at the March meeting and asked that the CSR begin implementing the recommendations to the best of its ability.

Read the ASBMB’s full comments here.

Enjoy reading ASBMB Today?

Become a member to receive the print edition monthly and the digital edition weekly.

Learn more
Mallory Smith

Mallory Smith earned her Ph.D. in biochemistry and molecular biology from the University of Kansas Medical Center and held a postdoc at the National Institutes of Health before joining ASBMB as a science policy manager. She is passionate about improving the STEM workforce pipeline, supporting early-career researchers, and advocating for basic science at the institutional, local and national level. 

Get the latest from ASBMB Today

Enter your email address, and we’ll send you a weekly email with recent articles, interviews and more.

Latest in Policy

Policy highlights or most popular articles

ARPA-H threatens the biomedical innovation pipeline
Blotter

ARPA-H threatens the biomedical innovation pipeline

May 25, 2022

Congress must find a way to fund the new agency without crippling the NIH and the curiosity-driven research it supports.

FASEB and NIH partner on DataWorks! Prize
Contest

FASEB and NIH partner on DataWorks! Prize

May 21, 2022

Contest highlights power of data sharing and reuse and rewards innovators.

Meet the 2022 ASBMB Advocacy Training Program delegates
Announcement

Meet the 2022 ASBMB Advocacy Training Program delegates

May 19, 2022

The fourth cohort of the ASBMB ATP will learn how to advocate for science policy this summer.

Recap: 2022 ASBMB Capitol Hill Day
News from the Hill

Recap: 2022 ASBMB Capitol Hill Day

May 16, 2022

Twenty-six participants in 19 states connected with officials and staffers in 59 meetings to advocate for science funding and support.

I took the NIH implicit bias training
Essay

I took the NIH implicit bias training

April 29, 2022

"As important as it is to raise awareness of implicit biases, the truth is that NIH needs to deal with explicit biases."

NIH data-sharing requirements: a big step toward more open science
Essay

NIH data-sharing requirements: a big step toward more open science

April 23, 2022

The director of the Rochester Institute of Technology’s Open Programs Office weighs in.