ASBMB comments on fairness in grant applications and peer review

In letter to NIH Center for Scientific Review, society recommends changes to reduce bias in study sections and programs for young scientists
Mallory Smith
April 14, 2022

The American Society for Biochemistry and Molecular Biology on Monday sent a letter to the National Institutes of Health Center for Scientific Review in which it advocated for changes aimed at reducing bias during the evaluation of research proposals and changes that would level the playing field for training grant applicants.

The letter was submitted in response to two matters discussed March 28 at a meeting of the CSR Advisory Council: recent bias training initiatives and ways to improve the review of fellowship applications.

In short, the ASBMB told the CSR that (A) it strongly supports the implementation of bias training at every level of peer review and that (B) it approves of the NRSA Working Group’s thoughtful recommendations— with a few tweaks. 

Bias training piloted

The CSR rolled out its “Bias Awareness in Review” training program in the fall.

Director Noni Byrnes said at the advisory council meeting that the program had a lot of successes as well as other “places to adjust” for the next round. Byrnes also emphasized that bias training “has to become mandatory” in the future.

In addition, Hope Cummings, a senior social science analyst at CSR, shared data collected over the past year about reviewers’ impressions of bias during review.

Strikingly, she reported that “the great majority of reviewers believe the NIH has a problem with bias in review” and “most reviewers (53%) never or rarely intervened when they thought bias was present.”

The advisory council recommended that CSR:

  1. Establish bias training as a requirement to participate in the review process.

  2. Create additional programming on implicit bias and other biases.

  3. Empower NIH scientific review officers and chairs of review panels to intervene when biased comments or assessments are made.

  4. Ensure that the framework for intervening in and reporting bias circumvents the power differentials present between review panel members. 

In its letter this week to CSR, the ASBMB emphasized that bias training should be mandatory for everyone involved in peer review — “all reviewers, chairs and scientific review officers” — and that the training should be “offered recurrently to facilitate sustained and enhanced awareness of bias.”

The society also recommended “a thoughtful review of feedback from the voluntary participants and incorporating it into new material that can enhance the curriculum, such as providing more examples of implicit and other types of bias and intervention tools for reviewers to use to counteract bias without fear of retaliation.”

The ASBMB has been vocal about instilling diversity, equity, and inclusion during scientific review, as exemplified by its recent recommendations to the CSR strategic plan.

Adjusting fellowship application criteria

At the advisory council meeting, Deputy Director Bruce Reed explained that CSR had heard “persistent concerns that fellowship review may disadvantage some highly qualified applicants,” leading CSR to convene a working group on fellowship review to “make recommendations to make the process as fair and effective as possible for all.”

According to CSR, review of NIH fellowship applications should “emphasize the candidate’s potential for a productive career, the candidate’s need for the proposed training, and the degree to which the research project and training plan, the sponsor(s), and the environment will satisfy those needs.”

Under this definition, the applicant’s scientific environment had become a significant factor in the evaluation criteria, leading to research-intensive universities having a leg up in securing fellowship applications under the current review criteria. Furthermore, research has shown that fellowship awardees do not always outperform their peers without fellowship awards.  

The working group recommended the following:

  1. Eliminate grades as indicators of qualifications.

  2. Eliminate the “Sponsor/Collaborator” and “Institutional Environment/Commitment to Training” criteria.

  3. Allow an optional statement of special circumstances (with an option to have the school submit a separate letter).

  4. Explicitly allow a wider range of career paths in fellowship training.

  5. Encourage a statement of qualifications that extends beyond grades and publications.

  6. Have review criteria that measure the total impact that the fellowship can bring to the individual and reduce the importance of external factors, such as mentor, institution, etc.

The working group also presented a “delta” concept that aims to provide a more holistic approach to review, reducing the emphasis on the sponsor and institution and focusing more on the applicant’s potential to succeed under the fellowship.

Elizabeth Villa, chair of the working group, explained that they were aiming to capture “the idea of casting a wider net to try to find the diamonds in the rough — incredible scientists with incredible potential that might have less strong conventional backgrounds.”

To accomplish this, she stated that they would “ask reviewers to try to not focus only on their accomplishments so far but also how likely it is that they will substantially benefit from the proposed fellowship — (to) pay a lot of attention to what they talk about their determination persistence creativity and otherwise potential to do incredibly well.”

The ASBMB already had made many of the above recommendations to improve fellowship review on Jan. 24. The society noted in its letter to CSR on Monday that it was pleased the working group had incorporated many of its suggestions, such as eliminating grades as an evaluation criterion since “grades are not an accurate measurement of success.”

However the society expressed concern about removing the sponsor/collaborator section altogether because “mentorship does play an important role in trainee success.” It recommended that the section be replaced with “a detailed mentorship plan, similar to the mentorship plan required by NSF fellowship applications.”

The ASBMB also had suggestions for how to implement an optional statement of special circumstances, which would give applicants the opportunity to explain setbacks from COVID-19, gaps in their training or transitions between labs. The society asked CSR to provide this option on future applications in a very intentional and clear format so that applicants “1) understand what qualifies as a special circumstance and 2) feel reassured that explaining certain special circumstances, such as harassment, will not penalize them in any way.”

Overall, the society was encouraged by the progress shared at the March meeting and asked that the CSR begin implementing the recommendations to the best of its ability.

Read the ASBMB’s full comments here.

Enjoy reading ASBMB Today?

Become a member to receive the print edition monthly and the digital edition weekly.

Learn more
Mallory Smith

Mallory Smith earned her Ph.D. in biochemistry and molecular biology from the University of Kansas Medical Center and held a postdoc at the National Institutes of Health before joining ASBMB as a science policy manager. She is passionate about improving the STEM workforce pipeline, supporting early-career researchers, and advocating for basic science at the institutional, local and national level. Smith is chair of the National Postdoctoral Association Advocacy Committee.

Sign up for the ASBMB advocacy newsletter

Get the latest from ASBMB Today

Enter your email address, and we’ll send you a weekly email with recent articles, interviews and more.

Latest in Policy

Policy highlights or most popular articles

ATP delegates push for improved policies
Society News

ATP delegates push for improved policies

April 5, 2024

This ASBMB program helps advocates gain skills to address issues that affect science and scientists.

Advocacy workshops at Discover BMB 2024
Annual Meeting

Advocacy workshops at Discover BMB 2024

Feb. 7, 2024

Topics include running for office, becoming an advocate, and navigating the grant review process at the NIH.

NIH’s advisory committee releases report on re-envisioning postdoc training

NIH’s advisory committee releases report on re-envisioning postdoc training

Jan. 8, 2024

The working group developed six primary recommendations for the National Institutes of Health.

When authoritative sources hold  onto bad data

When authoritative sources hold onto bad data

Dec. 23, 2023

A legal scholar explains the need for government databases to retract information.

Can science publishing be both open and equitable?

Can science publishing be both open and equitable?

Dec. 14, 2023

An updated memo from the White House Office of Science and Technology Policy has researchers, funders and publishers looking ahead

ASBMB pushes federal agencies to help students struggling with loan debt

ASBMB pushes federal agencies to help students struggling with loan debt

Nov. 30, 2023

The society states that increasing student debt and financial strain are hurting the U.S. research enterprise and federal agencies must do more to ease this burden.