Rooting student assessment
in the literature

A proposal for a Web-based resource using current research papers

Figure 1. Bloom's Taxonomy

Imagine if biochemistry and molecular biology instructors could go to the cloud and download high-quality assessment questions based on current research published in journals, such as the Journal of Biological Chemistry, for undergraduate classes. Key to such a collection would be the quality of the assessment questions. They should probe students’ ability not just to remember and to understand but also to apply, analyze and synthesize information and concepts. These skills correspond to higher level cognitive learning objectives articulated by Bloom’s Taxonomy (modernized in figure 1). I would like to propose a model for such a Web collection to meet this need. By sharing this vision, I hope to generate interest and foster collaboration with the larger community of educators to develop such a resource.

Over the past 27 years of teaching, I have discovered that exams based on recent research publications readily target those higher level Bloom’s Taxonomy skills. It has become increasingly apparent to me that it is more important that students develop scientific reasoning and inquiry skills and an understanding of how our knowledge developed than that they memorize a canon of structures and pathways.

Existing resources

In line with these ideas, new initiatives by the American Society for Biochemistry and Molecular Biology and the Association of American Medical Colleges endorse and assess learning competencies and defined learning goals and objectives (instead of proscribing a list of courses) for students studying biochemistry and molecular biology and taking the new MCAT15 exam. The new MCAT, offered for the first time this spring, has an explicit section on biochemistry, as input from targeted cohorts showed it as important for academic success in medical school. The exam also puts greater emphasis on scientific reasoning and inquiry skills. Anyone familiar with the ASBMB core concepts and associated learning objectives can see immediate congruence with the MCAT goals (figure 2).

Over the past three years, the AAMC developed the Pre-health Collection of the iCollaborative. This Web repository of resources is for faculty members and students (especially those at under-resourced institutions) as they teach and prepare for all areas represented in the new MCAT. All material is searchable by, and tied explicitly to, biochemistry foundational concepts and associated content categories. As an editor for biochemistry submissions, I have seen the potential this site offers. The value of an even more expansive collection based on the research literature and covering a wider range of biochemistry and molecular biology topics becomes apparent.

In a parallel fashion, the ASBMB division dedicated to education has developed an institutional accreditation process for biochemistry and molecular biology degree programs and a way to certify graduates of those programs. For accreditation, institutions must have a curriculum focused on critical-reasoning skills, laboratory experience, and the development of students’ scientific communication skills.

For certification, students must pass an exam in their junior or senior years. Efforts also are continuing to develop summative assessment questions for that exam. Although the ASBMB efforts are not targeted toward formative assessment per se, the Education and Professional Development Committee also initiated a broad effort to build Web resources that will help move faculty members toward “ concept-driven teaching strategies.” Specially, the committee seeks to build a collection tools to assess key concepts and student skills.

Figure 2. How ASBMB's core concepts relate to MCAT core concepts

A proposal

An easier and more robust process to create formative assessments is still lacking and needed. I propose that the community of biochemistry educators build, contribute to, review and curate a Web-based collection of assessments based on research literature. This initiative very clearly would complement the AAMC and ASBMB initiatives. If a database of such open-ended, literature-based questions were available, it greatly would help faculty move toward a more progressive approach for assessment of student learning.

To offer an example of what the collection could look like, I developed a prototype collection containing four literature learning and assessment modules (called LLAMs for short) that target the four ASBMB core concepts and associated learning objectives and that broadly overlap MCAT2015 biochemistry foundational concepts and associated content categories. The chosen articles and questions are typical of what a broad group of educators might use as the basis of an assessment in a biochemistry class. The richness of experimental techniques used to develop assessment questions for one of the LLAMs should be obvious immediately from the collage of figures taken from one JBC paper used for that purpose (see figure 3).

I chose to use open-ended assessment questions in the LLAMs because they take less time and are easier to write (though not grade) than credible and valid multiple-choice questions. That’s also what I have used in my classroom.

In my prototype, faculty members can use a Web version, a downloadable version that they can adapt easily and a key. I also included a student version with the full assessment, a question-by-question version with links to an answer after each question, and a full key.

Each assessment begins with the relevant JBC reference, use guidelines, and tables that show the ASBMB and MCAT foundational concepts addressed by the questions. The main assessment consists of background from the paper followed by questions based on figures, data, graphs, images and legends. The format is obviously only one of many possible ones.

An established collection of resources theoretically could be available to both faculty (as a teaching resource) and students (as a learning resource, as is presently the case for the AAMC Pre-health Collection). If it were available to students, it could enhance their development of the scientific inquiry and reasoning skills needed for their majors, future education and careers. It also would help students prepare for summative exams including the ASBMB certification and MCAT exams. However, student access might dissuade faculty from contributing and using the resource. Until a sufficient number of resources is available, it would be best to limit access to faculty.

Figure 3. Sample JBC figures from http://www.jbc.org/content/289/11/7702

Participation needed

How can we move from this simple prototype to a structured, sustainable and utilized Web resource? Darrell Porcello and Sherry His at the University of California, Berkeley, recently described ideal characteristics of online educational resources for use in online courses. Of course, they must be high quality and user-friendly to promote use, sustainability and learning. They should contain metadata for content tagging; use both expert and community definitions of quality to provide a curated collection of resources; and request input from both submitters and users, whose comments can evolve the quality of the collection.

What’s required is a structure and process to organize the community. As faculty members would be both users and contributors, the database would require crowdsourcing for both contributions and improvements. Jay Pedersen and co-authors at the University of Nebraska, Omaha, developed a conceptual model for crowdsourcing (see figure 4).

Figure 4. Conceptual model for crowdsourcing by Jay Pedersonetal.

In crowdsourcing, a problem (in this case the development of a trusted and used Web collection) is addressed using a process (a step-by-step action plan), a governance structure sufficient to guide the contributors’ efforts, participants (owners, contributors and improvers with defined roles and obligations who trust the process), and technology that enables the enterprise.

Evidence shows that high-quality contributions arise from users who are engaged in the process and who have a personal interest in the outcome and “ that a positive user experience is a strong predictor of continued involvement for both problem owners and the crowd members”.

Both the educational wing of the ASBMB and the editorial staff of the journal Biochemistry and Molecular Biology Education would be logical stakeholders in the design, creation, review and curation of the suggested portal. The ASBMB is wrestling with similar issues as it develops assessment tools for certification. I would like to see the ASBMB either house the collection or facilitate submissions from individuals who house the files on their host institutions’ sites. Another possible host/portal site is the Pre-health Collection of the iCollaborative.

The JBC and other journals provide downloadable PowerPoints for educational use. To jump-start the proposed web collection, the JBC could encourage authors to include assessment questions targeted to undergraduates when they submit their manuscripts for review. This would provide a great experience for graduate students and postdoctoral researchers who are thinking about pursuing academic positions.

I believe that other biochemistry and molecular biology educators would find the benefit of such a collection compelling enough to motivate their participation as designers, contributors, users and reviewers. A critical mass is necessary to proceed. I would like this article to lead to further dialogue and potential development of this resource. I invite you to visit the following Web site if you are interested in contributing to the development and/or use of this proposed collection.

Henry Jakubowski Henry Jakubowski is a professor in the chemistry department of the College of St. Benedict & St. John’s University.