Most discipline-based education researchers (DBERs) were formally trained in the methods of scientific disciplines such as biology, chemistry, and physics, rather than social science disciplines such as psychology and education. As a result, DBERs may have never taken specific courses in the social science research methodology—either quantitative or qualitative—on which their scholarship often relies so heavily. One particular aspect of (quantitative) social science research that differs markedly from disciplines such as biology and chemistry is the instrumentation used to quantify phenomena. In response, this Research Methods essay offers a contemporary social science perspective on test validity and the validation process. The instructional piece explores the concepts of test validity, the validation process, validity evidence, and key threats to validity. The essay also includes an in-depth example of a validity argument and validation approach for a test of student argument analysis. In addition to DBERs, this essay should benefit practitioners (e.g., lab directors, faculty members) in the development, evaluation, and/or selection of instruments for their work assessing students or evaluating pedagogical innovations.

Research has suggested that teaching and learning in molecular and cellular biology (MCB) is difficult. We used a new lens to understand undergraduate reasoning about molecular mechanisms: the knowledge-integration approach to conceptual change. Knowledge integration is the dynamic process by which learners acquire new ideas, develop connections between ideas, and reorganize and restructure prior knowledge. Semistructured, clinical think-aloud interviews were conducted with introductory and upper-division MCB students. Interviews included a written conceptual assessment, a concept-mapping activity, and an opportunity to explain the biomechanisms of DNA replication, transcription, and translation. Student reasoning patterns were explored through mixed-method analyses. Results suggested that students must sort mechanistic entities into appropriate mental categories that reflect the nature of MCB mechanisms and that conflation between these categories is common. We also showed how connections between molecular mechanisms and their biological roles are part of building an integrated knowledge network as students develop expertise. We observed differences in the nature of connections between ideas related to different forms of reasoning. Finally, we provide a tentative model for MCB knowledge integration and suggest its implications for undergraduate learning.

In an introductory biology course, we implemented a learner-centered, model-based pedagogy that frequently engaged students in building conceptual models to explain how genes determine phenotypes. Model-building tasks were incorporated within case studies and aimed at eliciting students’ understanding of 1) the origin of variation in a population and 2) how genes/alleles determine phenotypes. Guided by theory on hierarchical development of systems-thinking skills, we scaffolded instruction and assessment so that students would first focus on articulating isolated relationships between pairs of molecular genetics structures and then integrate these relationships into an explanatory network. We analyzed models students generated on two exams to assess whether students’ learning of molecular genetics progressed along the theoretical hierarchical sequence of systems-thinking skills acquisition. With repeated practice, peer discussion, and instructor feedback over the course of the semester, students’ models became more accurate, better contextualized, and more meaningful. At the end of the semester, however, more than 25% of students still struggled to describe phenotype as an output of protein function. We therefore recommend that 1) practices like modeling, which require connecting genes to phenotypes; and 2) well-developed case studies highlighting proteins and their functions, take center stage in molecular genetics instruction.

We followed established best practices in concept inventory design and developed a 12-item inventory to assess student ability in statistical reasoning in biology (Statistical Reasoning in Biology Concept Inventory [SRBCI]). It is important to assess student thinking in this conceptual area, because it is a fundamental requirement of being statistically literate and associated skills are needed in almost all walks of life. Despite this, previous work shows that non–expert-like thinking in statistical reasoning is common, even after instruction. As science educators, our goal should be to move students along a novice-to-expert spectrum, which could be achieved with growing experience in statistical reasoning. We used item response theory analyses (the one-parameter Rasch model and associated analyses) to assess responses gathered from biology students in two populations at a large research university in Canada in order to test SRBCI’s robustness and sensitivity in capturing useful data relating to the students’ conceptual ability in statistical reasoning. Our analyses indicated that SRBCI is a unidimensional construct, with items that vary widely in difficulty and provide useful information about such student ability. SRBCI should be useful as a diagnostic tool in a variety of biology settings and as a means of measuring the success of teaching interventions designed to improve statistical reasoning skills.

Understanding sources of performance bias in science assessment provides important insights into whether science curricula and/or assessments are valid representations of student abilities. Research investigating assessment bias due to factors such as instrument structure, participant characteristics, and item types are well documented across a variety of disciplines. However, the relationships among these factors are unclear for tasks evaluating understanding through performance on scientific practices, such as explanation. Using item-response theory (Rasch analysis), we evaluated differences in performance by gender on a constructed-response (CR) assessment about natural selection (ACORNS). Three isomorphic item strands of the instrument were administered to a sample of undergraduate biology majors and nonmajors (Group 1: n = 662 [female = 51.6%]; G2: n = 184 [female = 55.9%]; G3: n = 642 [female = 55.1%]). Overall, our results identify relationships between item features and performance by gender; however, the effect is small in the majority of cases, suggesting that males and females tend to incorporate similar concepts into their CR explanations. These results highlight the importance of examining gender effects on performance in written assessment tasks in biology.

At the University of Maine, middle and high school teachers observed more than 250 university science, technology, engineering, and mathematics classes and collected information on the nature of instruction, including how clickers were being used. Comparisons of classes taught with (n = 80) and without (n = 184) clickers show that, while instructional behaviors differ, the use of clickers alone does not significantly impact the time instructors spend lecturing. One possible explanation stems from the observation of three distinct modes of clicker use: peer discussion, in which students had the opportunity to talk with one another during clicker questions; individual thinking, in which no peer discussion was observed; and alternative collaboration, in which students had time for discussion, but it was not paired with clicker questions. Investigation of these modes revealed differences in the range of behaviors, the amount of time instructors lecture, and how challenging the clicker questions were to answer. Because instructors can vary their instructional style from one clicker question to the next, we also explored differences in how individual instructors incorporated peer discussion during clicker questions. These findings provide new insights into the range of clicker implementation at a campus-wide level and how such findings can be used to inform targeted professional development for faculty.

CREATE (Consider, Read, Elucidate the hypotheses, Analyze and interpret the data, and Think of the next Experiment) is an innovative pedagogy for teaching science through the intensive analysis of scientific literature. Initiated at the City College of New York, a minority-serving institution, and regionally expanded in the New York/New Jersey/Pennsylvania area, this methodology has had multiple positive impacts on faculty and students in science, technology, engineering, and mathematics courses. To determine whether the CREATE strategy is effective at the community college (2-yr) level, we prepared 2-yr faculty to use CREATE methodologies and investigated CREATE implementation at community colleges in seven regions of the United States. We used outside evaluation combined with pre/postcourse assessments of students to test related hypotheses: 1) workshop-trained 2-yr faculty teach effectively with the CREATE strategy in their first attempt, and 2) 2-yr students in CREATE courses make cognitive and affective gains during their CREATE quarter or semester. Community college students demonstrated positive shifts in experimental design and critical-thinking ability concurrent with gains in attitudes/self-rated learning and maturation of epistemological beliefs about science.

Student-centered strategies are being incorporated into undergraduate classrooms in response to a call for reform. We tested whether teaching in an extensively student-centered manner (many active-learning pedagogies, consistent formative assessment, cooperative groups; the Extensive section) was more effective than teaching in a moderately student-centered manner (fewer active-learning pedagogies, less formative assessment, without groups; the Moderate section) in a large-enrollment course. One instructor taught both sections of Biology 101 during the same quarter, covering the same material. Students in the Extensive section had significantly higher mean scores on course exams. They also scored significantly higher on a content postassessment when accounting for preassessment score and student demographics. Item response theory analysis supported these results. Students in the Extensive section had greater changes in postinstruction abilities compared with students in the Moderate section. Finally, students in the Extensive section exhibited a statistically greater expert shift in their views about biology and learning biology. We suggest our results are explained by the greater number of active-learning pedagogies experienced by students in cooperative groups, the consistent use of formative assessment, and the frequent use of explicit metacognition in the Extensive section.

In science education, dissections of animals are an integral part of teaching, but they often evoke negative emotions. We aimed at reducing negative emotions (anxiety, negative affect [NA]) and increasing positive affect (PA) and self-efficacy by an experimental intervention using a predissection video to instruct students about fish dissection. We compared this treatment with another group that watched a life history video about the fish. The participants were 135 students studying to become biology teachers. Seventy received the treatment with the dissection video, and 65 viewed the life history video. We applied a pre/posttest treatment-comparison design and used the Positive and Negative Affect Schedule (PANAS), the State–Trait–Anxiety Inventory for State (STAI-S), and a self-efficacy measure three times: before the lesson (pretest), after the film treatment (posttest 1), and after the dissection (posttest 2). The dissection film group scored higher in PA, NA, and state anxiety (STAI-S) after the dissection video treatment and higher in self-efficacy after the dissection. The life history group showed no differences between the pretest and posttest 1. The dissection film has clear benefits—increasing PA and self-efficacy—that come at the cost of higher NA and higher STAI-S.