Recent Research in Science Teaching and Learning

This feature is designed to point CBE—Life Sciences Education readers to current articles of interest in life sciences education as well as more general and noteworthy publications in education research. URLs are provided for the abstracts or full text of articles. For articles listed as “Abstract available,” full text may be accessible at the indicated URL for readers whose institutions subscribe to the corresponding journal.

Online publication of this meta-analysis last spring no doubt launched a legion of local and national conversations about how science is best taught—as the authors state the essential issue, “Should we ask or should we tell?” To assess the relative effectiveness of active-learning (asking) versus lecture-based (telling) methods in college-level science, technology, engineering, and mathematics (STEM) classes, the authors scoured the published and unpublished literature for studies that performed a side-by-side comparison of the two general types of methods. Using five predetermined criteria for admission to the study (described fully in the materials and methods section), at least two independent coders examined each potentially eligible paper to winnow down the number of eligible studies from 642 to 225. The working definition of what constitutes active learning (used to determine potential eligibility) was obtained from distilling definitions written by 338 seminar attendees; what constitutes lecture was defined as “continuous exposition by the teacher” (quoted from Bligh, 2000). The eligible studies were situated in introductory and upper-division courses from a full range of enrollment sizes and multiple STEM disciplines and included majors and nonmajors as participants. The frequency of use and types of active-learning methodologies described in the 225 eligible studies varied widely.

Quantitative analysis of the eligible studies focused on comparison of two outcome variables: 1) scores on identical or formally equivalent examinations and 2) failure rates (receipt of a “D” or “F” grade or withdrawal from the course). Major findings were that student performance on exams and other assessments (such as concept inventories) was nearly half an SD higher in active-learning versus lecture courses, with an effect size (standardized mean weighted difference) of 0.47. Analyses also revealed that average failure rates were 55% higher for students in the lecture courses than in courses with active learning. Heterogeneity analyses indicated that 1) there were no statistically significant differences in outcomes with respect to disciplines; 2) effect sizes were lower when instructor-generated exams were used versus concept inventories with both types of courses (perhaps because concept inventories tend to require more higher-order thinking skills); 3) effect sizes were not significantly different in nonmajors versus majors courses or in lower versus upper-division courses; and 4) although active learning had the greatest positive effect in smaller-enrollment courses, effect sizes were higher with active learning at all enrollment sizes. Two types of analyses, calculation of fail-safe numbers and funnel plots, supported a lack of publication bias (tendency to not publish studies with low effect sizes). Finally, the authors demonstrated that there were no statistically significant differences in effect sizes despite variation in the quality of the controls on instructor and student equivalence, supporting the important conclusion that the differences in effectiveness between the two methods were not instructor dependent.

In one of the more compelling sections of this meta-analysis, the authors translated the relatively dry numbers resulting from statistical comparisons to potential impacts on the lives of the students taking STEM courses. For example, for the 29,300 students reported for the lecture treatments across all students, the average difference in failure rates (21.8% in active learning vs. 33.8% with lecture) suggests that 3516 fewer students would have failed if enrolled in an active-learning course. This and other implications for the more beneficial impact of active learning on STEM students led the authors to state, “If the experiments analyzed here had been conducted as randomized controlled trials of medical interventions, they may have been stopped for benefit.” That is, the control group condition would have been halted because of the clear, beneficial effects of the treatment. The authors conclude by suggesting additional important implications for future undergraduate STEM education research. It may no longer be justified to conduct more “first-generation” research comparing active-learning approaches with traditional lecture; rather, for greater impact on course design, second-generation researchers should focus on what types and intensities of exposure to active learning are most effective for different students, instructors, and topics.

This provocative commentary by Carl Weiman highlights the major findings reported in the Proceedings of the National Academy of Sciences by Freeman et al. (2014) and underscores the implications. The graphical representations displaying the key data on effect sizes and failure rates presented in the Freeman et al. meta-analysis are redrawn in the commentary in a way that is likely to be more familiar to the typical reader, making the differences in outcomes for active learning versus lecture appear more striking. Weiman concludes by elaborating on the important implications of the meta-analysis for college-level STEM educators and administrators, suggesting that it “makes a powerful case that any college or university that is teaching its STEM courses by traditional lectures is providing an inferior education to its students. One hopes that it will inspire administrators to start paying attention to the teaching methods used in their classrooms … establishing accountability for using active-learning methods.”

National societies, committee reports, and accrediting bodies recommend that engineering curricula be designed to prepare future engineers for the complex interdisciplinary nature of the field and for the multitude of skills and perspectives they will need to be successful practitioners. The authors posit that case-based instruction, with its emphasis on honing skills in solving authentic, interdisciplinary, and ill-defined problems, aligns well with these recommendations. However, the methodology is still relatively underutilized, and its effectiveness is underexamined. This article describes a study designed to advance these issues by comparing lecture- and case-based methods within the same offering of a 72-student, upper-level, required course in mechanical engineering.

The study used a within-subjects, posttest only, A-B-A-B research design across four key course topics. That is, two lecture-based modules (the A or baseline phases) alternated with case-based modules (the B or treatment phases). Following each module, students responded to open-response quiz questions and a survey about learning and engagement (adapted from the Student Assessment of Learning Gains instrument). The quiz questions assessed ability to apply knowledge to problem solving (so-called “traditional” questions) and ability to explain the concepts that were used (“conceptual” questions). This study design had the advantage that the same students experienced both the baseline and treatment conditions twice. The authors describe in detail the pedagogical approaches used in both sets of the A and B phases.

The quizzes were scored by independent raters (with high interrater reliability) on a 0–3 scale; scores were analyzed using appropriate statistical methods. Survey items were analyzed using a principal-components factor analysis; composite scores were generated for a learning confidence factor and an engagement–connections factor. Analyses revealed that the two pedagogical approaches had similar outcomes with respect to the traditional questions, but conceptual understanding scores (indicating better understanding of the concepts that were applied to problem solving) were significantly higher for the case-based modules. Students reported that they appreciated how cases were better than lecture in helping them make connections to real-world concerns and see the relevance of what they were learning, but there were no significant differences in students’ perceptions of their learning gains in the case-based versus the lecture modules. The authors note that many studies have likewise demonstrated that students’ perceptions of their learning gains in more learner-centered courses are often not accurate reflections of the actual learning outcomes.

The authors conclude that while these results are promising indications of the effectiveness of case-based instruction in engineering curricula, the studies need to be replicated across a number of semesters and in different engineering disciplines and extended to assess the long-term effect of case-based instruction on students’ ability to remember and apply their knowledge.

Although this study was limited to an engineering context, the case-based methodologies and research design seem well-suited for use in action research in other disciplines.

Well-documented challenges to conceptual change faced by students of evolution include the necessity of unseating existing naïve theories (such as natural selection having purposiveness), having the ability to view the complex and emergent nature of evolutionary processes through systems-type thinking, and being able to see the connections between evolutionary content learned in the classroom and everyday life events that can facilitate appreciation of its importance and motivate learning. To help students meet these challenges, the authors adapted a pedagogical model called Teaching for Transformative Experiences in Science (TTES) in the course of instruction on six major concepts in evolutionary biology. This article reports on a comparison of the effectiveness of TTES approaches in fostering conceptual change and positive affect with that of instruction enhanced with use of refutational texts (RT). Use of RTs to promote conceptual change, a strategy with documented effectiveness, entails first stating a misconception (the term used by the authors), then explicitly refuting it by elaborating on a scientific explanation. By contrast, the TTES model promotes teaching that fosters transformative learning experiences—teaching in which instructors 1) place the content in a context allows the students to see its utility or experiential value; 2) model their own transformative experiences in learning course concepts; and 3) scaffold a process that allows students to rethink or “resee” a concept from the perspective of their previous, related life experiences.

The authors designed the study to address three questions relevant to the comparison of the two approaches: would the TTES group (vs. the RT group) demonstrate or report 1) greater conceptual change, 2) higher levels of transformative experience, and 3) differences in topic emotions (more positive affect) related to learning about evolution? The study used three survey instruments, one that measured the types and depth of students’ transformative experiences (the Transformative Experience Survey, adapted from Pugh et al., 2010), another that assessed conceptual knowledge (Evolutionary Reasoning Scale; Shulman, 2006), and a third that evaluated the emotional reactions of students to the evolution content they were learning (Evolution Emotions Survey, derived from Broughton et al., 2011). In addition to Likert-scale items, the Transformative Experience Survey contained three open-ended response questions; the responses were scored by two independent raters using a coding scheme for degree of out-of-school engagement. The authors provide additional detail about the nuances of what these instruments were designed to measure and their scoring schemes and include the instruments in the appendices. The Evolutionary Reasoning Scale and the Evolution Emotions survey were administered as both pre- and posttests, and the Transformative Experience survey was administered only at the end of the intervention. The treatment (TTES, n = 28) and comparison (RT, n = 27) groups were not significantly different with respect to all measured demographic variables and the number of high school or college-level science courses taken.

Briefly, the evolutionary biology learning experience that participants were exposed to was 3 d in duration for both the treatment and comparison groups. On day 1, the instructor (the same person for both groups) gave a PowerPoint lecture on the same six evolutionary concepts, with illustrative examples. For the treatment group only, the instructor drew from his own transformative experiences in connection with the illustrative examples, describing how he used the concepts, what their value was to him, and how each had expanded his understanding and perception of evolution. On days 2 and 3 for the treatment group, the students and instructor engaged in whole-class discussions about their everyday experiences with evolution concepts (and related misconceptions) and their usefulness; the instructor scaffolded various “reseeing” experiences throughout the discussions. For the comparison group, misconceptions and refutations were addressed in the course of the day 1 lecture, and on days 2 and 3, the participants read refutational texts and then took part in discussions of the texts led by the instructor.

Survey results and accompanying statistical analyses indicated that both groups exhibited gains (with significant t statistics) in understanding of the evolution concepts as measured by the Evolutionary Reasoning Scale (Shulman, 2006). However, the gains were greater for the treatment (TTES) group: effect size, reported as a value for eta-squared, η², equaled 0.29. The authors point out by way of context for this outcome that use of RTs, along with follow-up discussions that contrast misconceptions with scientific explanations, has been previously shown to be effective in promoting conceptual change; thus, the comparison was with a well-regarded methodology. Additionally, the Transformation Experience survey findings indicated higher levels of transformative experience for the TTES group participants; they more extensively reported that the concepts had everyday value and meaning and expanded their perspectives. The TTES group alone showed pre- to posttest gains in enjoyment while learning about evolution, a positive emotion that may have classroom implications in terms of receptivity to learning about evolution and willingness to continue study in this and related fields.

The authors conclude that the TTES model can effectively engage students in transformative experiences in ways that can facilitate conceptual change in content areas in which that change is difficult to achieve. In discussing possible limitations of the study, they note in particular that the predominance of female study participants (71% of the total) argues for its replication with a more diverse sample.

I invite readers to suggest current themes or articles of interest in life sciences education, as well as influential papers published in the more distant past or in the broader field of education research, to be featured in Current Insights. Please send any suggestions to Deborah Allen (deallen@udel.edu).

REFERENCES