ASCB logo LSE Logo

Active Learning Outside the Classroom: Implementation and Outcomes of Peer-Led Team-Learning Workshops in Introductory Biology

    Published Online:https://doi.org/10.1187/cbe.16-01-0051

    Abstract

    Study group meetings (SGMs) are voluntary-attendance peer-led team-learning workshops that supplement introductory biology lectures at a selective liberal arts college. While supporting all students’ engagement with lecture material, specific aims are to improve the success of underrepresented minority (URM) students and those with weaker backgrounds in biology. Peer leaders with experience in biology courses and training in science pedagogy facilitate work on faculty-generated challenge problems. During the eight semesters assessed in this study, URM students and those with less preparation attended SGMs with equal or greater frequency than their counterparts. Most agreed that SGMs enhanced their comprehension of biology and ability to articulate solutions. The historical grade gap between URM and non-URM students narrowed slightly in Biology 2, but not in other biology and science, technology, engineering, and mathematics courses. Nonetheless, URM students taking introductory biology after program implementation have graduated with biology majors or minors at the same rates as non-URM students, and have enrolled in postcollege degree programs at equal or greater rates. These results suggest that improved performance as measured by science grade point average may not be necessary to improve the persistence of students from underrepresented groups as life sciences majors.

    INTRODUCTION

    While lectures are the traditional mode of college science instruction, critics have argued that lectures harden barriers to participation by students from many ethnic, cultural, and socioeconomic backgrounds, by presupposing familiarity with implicit premises and values that are culturally narrow (Brickhouse, 1994; Brickhouse et al., 2000; Lee et al. 1995; Aikenhead, 1996; Gilbert and Yerrick, 2001; Brown, 2004; Rahm, 2008). These students often feel inhibited from asking questions in large lectures, because they perceive that questions reveal a deficit in their presumed knowledge base and expose the questioner as an outsider. As a result, traditional lecture classes often marginalize students from diverse backgrounds (Johnson, 2007).

    By contrast, curricular structures that incorporate active learning are thought to improve learning and engagement, particularly when facilitated by peers (Woodward et al., 1993). Collaborative problem-solving reinforces argumentation skills as students advance and rebut provisional solutions. Good facilitators model conventions of language and behavior and provide scaffolds such as targeted explanations or Socratic questions (Collins et al., 1989; Dennen and Burner, 2007). Facilitators with whom students identify, such as peers, can also serve as role models for academic success and interest in science. Identification can ameliorate incompatibilities students perceive between science and their core beliefs and habits (Chambers, 1983; Brickhouse, 1994; Brickhouse et al., 2000). Thus, peers have remarkable potential as facilitators, because they embody “instructional congruence” (Lee and Fradd, 1998), bridging gaps between fellow students’ perceptions of science and of themselves or their desired future selves. Importantly, a sense of self-efficacy is essential for scientific identity formation (Trujillo and Tanner, 2014) and observing successful peers bolsters students’ self-efficacy beliefs (Bandura, 1977).

    Progressive science educators have pioneered curricular structures that exploit the potential of facilitated, active learning. A recent meta-analysis of 225 studies demonstrated that incorporation of active-learning interventions in a variety of science, technology, engineering, and mathematics (STEM) fields increases student grades and decreases failure rate (Freeman et al., 2014). These interventions range from incorporating think–pair–share activities into lecture-based courses to replacing lecture entirely with instructor-and-peer-facilitated group work in problem-based learning (PBL) courses (Eberlein et al., 2008). However, despite evidence supporting positive outcomes (Smith et al., 2009), it remains a challenge for many faculty to incorporate active-learning pedagogies into well-honed courses and sequences of lectures.

    As a more moderate alternative to supplanting lectures with PBL, the peer-led team-learning (PLTL) model represents a way to incorporate mentored active learning with less disruption to the classroom (Eberlein et al., 2008). Defining features of PLTL include peer leaders trained in tutorial methods; challenging questions suitable for collaborative problem solving; and designated meeting times outside lecture. Peer leaders are selected from among students who had been successful in the course in a previous semester. They are generally trained in pedagogical principles and practices and are paid for their time. Problems are designed by faculty lecturers to require critical integrative and analytical thinking (Sarquis et al., 2001). PLTL programs in different institutions vary by attendance requirements, ranging from walk-in (Drane et al., 2005; Amstutz et al., 2010) to voluntary-enrollment programs in which attendance is expected (Snyder et al., 2015) to mandatory workshops with graded assignments (Preszler, 2009). Programs also vary by institutional context, ranging from small, open-enrollment, 2-year colleges (Amstutz et al., 2010) to selective midsized universities (Drane et al., 2005).

    PLTL programs are often implemented with the intention of broadening participation in STEM. Previous studies describing outcomes of PLTL in introductory biology courses have assessed attendance, self-reported gains in comprehension and confidence, course performance gains (grades), and effects on attrition rates from the course or from a defined sequence of courses. Across contexts, students self-report gains in confidence and comprehension from PLTL attendance (Drane et al., 2005; Preszler, 2009; Amstutz et al., 2010; Stanger-Hall et al., 2010; Batz et al., 2015). Small performance gains of 0.2–0.3 grade points have been associated with several PLTL programs in different contexts, although these gains were often inconsistent when multiple semesters or quarters were studied (Drane et al., 2005; Preszler, 2009; Amstutz et al., 2010; Stanger-Hall et al., 2010; Batz et al., 2015). A recent study describes a full grade point associated with PLTL attendance for a subset of students who opted out of lab in one semester (Snyder et al., 2015). Another found a relationship between PLTL and performance for underrepresented minority (URM) students that approached significance (p = 0.07) in one of the three quarters in which impacts of the program were assessed (Drane et al., 2005). Others have reported no special benefit for URM students but greater gains for underprepared students (Batz et al., 2015).

    Importantly, PLTL programs have been reported to improve retention or persistence for URM or underprepared students, with persistence usually defined as reduced attrition from courses or sequences of courses (Drane et al., 2005; Preszler, 2009; Stanger-Hall et al., 2010; Batz et al., 2015). One study described a small (r = 0.1) but significant association between graduation rates and workshop attendance in a voluntary-attendance PLTL model, although this was not parsed by major, preparedness, or URM status (Amstutz et al., 2010). However, few or no studies to date have examined a putative association between PLTL program implementation and rates of graduation as STEM majors or minors or continuation to graduate schools. We contribute to the current body of work by describing the implementation and outcomes of our study-group meeting (SGM) program, a PLTL model in introductory biology at a selective, small liberal arts college. We address the following four research questions: To what extent do students from URM groups or those with weaker preparation in biology use the program relative to counterparts? What do students perceive as benefits of participating in SGMs? To what extent is participation associated with improved performance in the course? To what extent is program implementation associated with improved persistence of URM students in biology and other natural science majors and minors and enrollment in postgraduate programs?

    METHODS

    Institutional Context

    This study took place at Swarthmore College, a selective, undergraduate liberal arts college of roughly 1500 students set in suburban Philadelphia, Pennsylvania. All students are admitted as premajors and apply to departments to become majors or minors in the Spring semester of sophomore year. While biology is often reported as an intended major by incoming first-year students, many students are undecided about their intended major upon matriculation. The study focused on students who were admitted between 2000 and 2011 and who took introductory biology in their first year. The median Scholastic Aptitude Test (SAT) scores for incoming students were 730 for Verbal/Critical Reading and 720 for Math, and these remained unchanged from 2000 through 2011. URM students (defined as those who self-identified in admissions records as African American, Latino/a American, or Native American) averaged 16.9% of the student body between 2000 and 2005 and 20.0% between 2008 and 2013. College 6-year graduation rates for cohorts of students entering during this period ranged between 92 and 95% overall, and 82–96% for URM students.

    Course Context and Resources

    The SGM program supports the lecture component of two team-taught introductory biology courses, Biology 1: Cellular & Molecular Biology (Fall) and Biology 2: Organismal and Population Biology (Spring). Biology 1 and 2 have no prerequisites and can be taken by any student at the college. The content in each course is modular, so students may take one or both courses in any sequence. They are generally among the largest courses at the institution, with enrollments ranging from 85 to 140 students per semester. Each course attracts roughly 70–80% freshmen, with the remainder mostly sophomores with a few juniors and/or seniors.

    Students enroll in the courses for a variety of reasons. The courses are required for biology majors and minors and are therefore taken by those who anticipate majoring in biology or a related discipline. About half of Biology 1 and 2 students become biology majors, minors, or biology-related majors (e.g., neuroscience or biochemistry majors). Some register intending to pursue biology or a related discipline but do not persist. Other students use the courses to fulfill requirements for medical school or to satisfy college distribution requirements for graduation. Some students enroll in the courses to satisfy requirements but then decide to become biology majors. These courses also attract students who have earned a score of 5 on the Advanced Placement (AP) Biology exam even if they do not plan on continuing studies in biology, because students must take a biology course at Swarthmore to receive credit for their AP work.

    While grades are calculated for all students, all grades for first-semester freshmen appear only as credit or no-credit (CR/NC) on official student transcripts. Thus, most Biology 1 students in this study took the course CR/NC, whereas most Biology 2 students took the course for a letter grade. Biology 1 and Biology 2 have classroom and laboratory components that contribute approximately equally to the total grade for the course. In each team-taught course, four professors take turns lecturing in an auditorium using blackboards and projected images or videos for three 50-min periods per week. Grades for the classroom component stem from three to six quizzes and a comprehensive final exam. Question formats are primarily short answer with a few other question types. Each student attends one weekly 3-h laboratory section taught by one of the four lecturing professors alongside a professional BS-, MS-, or PhD-level instructor and an undergraduate laboratory teaching assistant (TA). Grades for the laboratory component stem from two or three laboratory reports and other short written assignments focused on data analysis. Professional academic support resources include office hours and open-door policies of the eight- to nine-member professional teaching staff. Peer academic support resources include evening data analysis help sessions with lab TAs and editorial meetings with student writing associates (WAs). In parallel, peer science associates (SAs) facilitate student work at evening SGMs.

    Challenge Questions

    While faculty generally do not attend SGMs, they provide “challenge questions” that are the focus of student work at SGMs. The expectation that study groups will discuss and solve difficult problems is based on the structure of the PLTL workshop program at Northwestern University (Born et al., 2002), which in turn was inspired by the workshop calculus model pioneered at the University of California–Berkeley (Fullilove and Treisman, 1990). Each faculty member assigns three to six challenge questions per lecture (see examples in the Supplemental Material). These are handed out to students in lecture and posted to the course website. These questions are meant to be difficult and address higher levels of understanding in Bloom’s taxonomy (Crowe et al., 2008). In lecture, students are told that the questions are hard and are encouraged to work in groups to solve them. The answers are not collected or graded. The challenge questions are not used as quiz or exam questions, but because they align with learning objectives for the lectures, solving the challenge questions is good preparation for summative assessments.

    Program Structure

    Some aspects of the SGM program were introduced in Biology 1 and 2 in 2004–2005. It was fully implemented 2005–2006 with the support of a grant from the Howard Hughes Medical Institute (HHMI). The structure of the program is aligned with the PLTL described previously.

    Peer facilitators, the SAs, host 2-h SGMs two evenings per week. An additional weekend SGM is offered before each quiz and an extended 4-h SGM is offered the night before each quiz for a mean of 30 SGMs each semester. Snacks are available to students in the SGM meeting room. Students may arrive or leave any time during an SGM. Participation in the program is strictly voluntary. Faculty lecturers and laboratory instructors encourage all students to attend by promoting the benefits of the group problem-solving experience. In addition, instructors typically offer direct encouragement to struggling students to attend SGMs.

    Eight SAs are selected each semester from a pool of applicants. URM students are encouraged to apply in order to generate a diverse pool. Biology faculty and instructors rate the suitability of each applicant based on experiences with these students in classes or other relevant contexts. SAs are selected based in part on these ratings and also with the aim of building a team that is diverse by class year, gender, and ethnicity. SAs have taken Biology 1 and 2 and are required to attend lecture during the semester(s) they are working as peer leaders. In addition, SAs meet with the faculty lecturer each week to review challenge questions and other lecture material. They are also required to attend weekly meetings of a course entitled Pedagogical Principles & Practices (syllabus in the Supplemental Material), taught by the program coordinator, for which they are paid but receive no academic credit. In this seminar-type course, SAs read articles or excerpts from the educational literature and discuss how to apply these ideas as peer facilitators at SGMs.

    The number of SAs present in a given SGM is a function of predicted attendance. Student attendance at SGMs is voluntary and patterns have been similar over many years, permitting targeted allocation of resources such as snack food and SAs. From two to six SAs are assigned a given shift depending on predicted attendance. Students sign an attendance sheet when they enter and sit in movable chairs to facilitate the formation of small groups. They typically form groups of between three and seven students, similar to group sizes reported for other PLTL programs in biology (Drane et al., 2005).

    SAs move about the room in the manner of restaurant wait staff. They make themselves available to answer questions or overhear conversations and make targeted comments. They are trained to not provide full answers to the challenge questions but to catalyze their peers’ critical thinking and discussion and inspire interest about challenge questions, lecture topics, and course themes. Their methods include Socratic questioning, restating student explanations with more appropriate terminology, and asking students to write or draw (concept maps, etc.) and “jigsaw” (a practice in which each member of a group is responsible for solving and sharing a given part of a problem or problem set). By working together (at the least in pairs), they model teamwork, community cooperation, and group problem solving if they are asked an especially difficult question. SAs rely on their training, intuition, and prior experiences to diagnose impediments to student comprehension and motivation and provide appropriate pedagogical interventions. For example, if a group is struggling to answer a challenge question, an SA would ask questions to probe the students’ understanding of the concepts the question is based on and guide them to the relevant lecture or text material.

    Student Populations

    The results are based on institutional records and surveys of students who enrolled in Biology 1 or 2 between Fall 2008 and Spring 2012. The control (before-program) cohort was composed of students who took the courses between Fall 2000 and Spring 2004, before SGM program implementation. Students who enrolled in both Biology 1 and 2 are counted with each course for SGM attendance, perceptions, and performance, which are course-based measures. Each student was counted once for analyses of student-based measures of persistence and outcomes.

    Because graduation rates at the college are already so high that they are unlikely to be influenced by these efforts, measures of persistence and outcomes focus on students’ continuation in biology or other STEM majors and minors. Specifically, analyses of persistence, subsequent course performance, and outcomes focus on those who took either Biology 1 or 2 as a first-year student and graduated within 4 years (N = 453 in before-program control group and N = 452 in after-program group). URM students (defined as those who self-identified in admissions records as African American, Latino/a American, or Native American) were 14.1% of the before-program population (N = 453) and 19.9% of the after-program population (N = 452). Those who did not self-identify were included in the non-URM group.

    The additional use of underpreparedness as a criterion to identify at-risk students has been justified by arguments that ethnicities are overaggregated (Alessandria, 2002; Lee, 1997). Underprepared students have been classified as such by parental education (Harackiewicz et al., 2013), socioeconomic/geographic identifiers (Gilbert and Yerrick, 2001; Rahm, 2008), or academic background such as AP status (Scott et al., 2010). In an effort to define two groups based on students’ academic preparation for introductory biology, those who reported scores on the AP Biology exam were considered to have more preparation than those who did not report AP Biology scores (non-AP). This criterion served as a proxy for positively identifying students who had taken more than one biology course in high school (AP group), while the non-AP group was more heterogeneous with regard to prior experience in biology. In the after-program population, 42% reported AP Biology scores, and 91% of these scores were 4 or 5. The mean math SAT score of the AP group was 726 compared with 699 for the non-AP group (t test, p < 0.0001). This grouping was implemented only for analyses of after-program data, since records of AP scores were not available for the before-program group.

    Statistics

    A variety of statistical methods were used, depending on the types of measures and hypotheses explored. Where possible, dependent variables were compared for students in the following groups: URM versus non-URM, non-AP versus AP, and before- versus after-program implementation. Frequencies of attending SGMs across the semester were compared for these groups using chi-square tests. Pearson correlations were used to explore relationships between SGM attendance and course grades.

    Confidential surveys were conducted by Swarthmore’s Office of Institutional Research at the end of each semester. Institutional research staff connected survey respondents to their institutional data by using the student ID requested as part of the survey. Because survey items reflecting perceptions of SGM benefits were ordinal, correlations calculated between SGM attendance and these perceptions used the nonparametric Spearman rank-order coefficient. Chi-square statistics were used to determine whether frequencies of positive responses (“somewhat helped” or “helped a lot”) on these items were related to group status.

    Using data from before the program implementation as a control raises the concern that any observed differences might reflect exogenous changes at the institution over time that could have influenced student performance. We assessed the influence of year on both biology and natural sciences and engineering (NSE) grade point averages (GPAs) by using Akaikes information criterion (AIC) model selection to contrast models of GPA expressed as a function of URM treatment, with year either excluded (linear models) or treated as a random effect (linear mixed models). As the top AIC model (delta AIC ≥ 2) in both cases was the linear model that excluded year, it was possible to conclude that effects attributable to year were minimal. Therefore, a simpler analysis of variance (ANOVA; pooling the years before and after program implementation) is justified and is presented here.

    A related concern is that student ability was not equivalent in the before- and after-program cohorts, particular for targeted groups. To test this possibility, we compared SAT Math scores of URM freshmen taking these introductory biology courses during the pooled before-program years with the pooled after-program years, using an independent samples t test. The scores were not significantly different (t = −1.66 with df = 561, p = 0.10).

    Pearson correlations were used to explore relationships between SGM attendance and course grades. Introductory course grades during each semester were compared for URM and non-URM, and for more prepared and less-prepared students, using independent sample t tests. Longitudinal performance data were also examined using a linear mixed model.

    Student persistence as biology or STEM majors and minors and subsequent enrollment in postgraduate programs are categorical data, and so the chi-square tests were used to determine relationships between persistence and groups, and postgraduate enrollment and groups.

    RESULTS

    Student Participation in SGMs

    The primary subjects of this study were students who enrolled in introductory biology during eight semesters (Fall 2008 through Spring 2012). A large majority of those in Biology 1 (82%) and Biology 2 (77%) attended at least one SGM. The median number of SGMs attended was three for the overall population and four for the URM and non-AP subsets of the population (Figure 1). The frequency of SGM attendance by students from URM groups exceeded that of non-URM students (Figure 1A; chi-square = 13.8, df = 5, p = 0.02). Rates of attendance by students in the non-AP and AP groups were not different.

    Figure 1.

    Figure 1. Percentages of students in subsets of the study population who attended different numbers of SGMs per semester over the 4-year study, for targeted (red bars) and nontargeted (blue bars) groups. For individuals who took both Biology 1 and Biology 2, attendance in each semester was counted separately. (A) Groups defined by URM status: non-URM (N = 727) vs. URM (N = 173); chi-square test, p = 0.02. (B) Groups defined by experience in AP Biology: AP (N = 282) vs. non-AP (N = 618); chi-square test, nonsignificant.

    Student Perceptions of SGMs

    Perceived benefits of attending SGMs were assessed using seven multiple-choice items on anonymous end-of-term surveys. Seven putative benefits were assessed: interest in biology, self-confidence in understanding biology, ability to think scientifically, enjoyment of the course, comfort level in taking biology courses, ability to articulate solutions to scientific problems, or interest in further learning in biology. Each item prompted students to fill in the statement “Going to SGMs ____” on a four-point scale: 1 = did not help, 2 = helped very little, 3 = somewhat helped, 4 = helped a lot.

    First, we compared percentages of students selecting each of the four responses to identify the modal response for each putative benefit. The mode was either somewhat helped (3) or helped a lot (4) for all seven benefits in Biology 1 and for six of the seven benefits in Biology 2. When results for the two introductory courses were pooled and ratings were parsed by URM status, we found that URM and non-URM students responded similarly in their ratings of each of the seven putative benefits (Figure 2A). By contrast, less-prepared (non-AP) students rated most of the putative benefits more highly than those who had reported AP Biology scores (Figure 2B). Specifically, significantly more non-AP students than AP students said that the SGMs either somewhat helped (3) or helped a lot (4) to enhance their interest in biology, their understanding of biology, their enjoyment of the course, and their comfort in taking biology courses (chi-square tests, p < 0.05).

    Figure 2.

    Figure 2. Percentages of students who indicated that attending SGMs enhanced each of seven putative benefits “somewhat” or “a lot” (3 or 4 on a four-point scale), comparing responses for targeted (red bars) and nontargeted (blue bars) groups over the 4-year study. (A) URM vs. non-URM groups; (B) groups defined by experience in AP Biology. * indicates p < 0.05; # indicates p < 0.1 (chi-square tests; N ranged from 550 to 561 freshman respondents to each survey item in Biology 1 and Biology 2).

    We also explored possible relationships between student ratings of the benefits of SGMs and their actual SGM attendance. As expected, student attendance was positively correlated with their ratings for all seven putative benefits in both courses (Spearman rho tests; r ranged from 0.22 to 0.40, p < 0.001).

    Finally, the surveys provided opportunities for students to expand on their ratings in free-response items. The majority of students in Biology 1 and 2 described benefits of defining features of the SGM program, including working with fellow students and SAs on faculty-designed challenge questions in SGMs. Typical examples include “The SAs and the way they were able to explain the material without feeding you the answers were extremely helpful” (Biology 1 student, Fall 2011), and “Learning from other people and teaching other people with the help of the SAs was good for my learning and I think helped me tremendously on the quizzes” (Biology 2 student, Spring 2012).

    Student Performance in Introductory Biology

    To examine the possibility that participation in SGMs enhanced academic performance, we calculated correlations between SGM attendance and course grades. We found a significant positive correlation between SGM attendance and grades in Biology 2 (Pearson correlation coefficient, r = 0.18, p < 0.001). A similar relationship was found when narrowing the population to less-prepared students (non-AP) in Biology 2 (r = 0.16, p < 0.01) but was not seen for URM students. There was no relationship between SGM attendance and grades for students in Biology 1.

    Longitudinal data comparing average grades from preprogram years through Spring 2012 (Figure 3) show that URM student grades were significantly lower than non-URM grades in both Biology 1 (t1165 = −4.386, p < 0.0001) and Biology 2 (t1000 = −7.23, p < 0.0001). There was no significant change in average grades after program implementation when compared with preprogram years (Figure 3). In Biology 2, however, a significant interaction between URM status and pre- versus postprogram years (linear mixed model applied to arcsine-transformed grades, p < 0.05) suggests that the historical grade gap between URM and non-URM students narrowed in this course.

    Figure 3.

    Figure 3. Course grades for URM and non-URM students over a 10-year period in Biology 1 (A) and Biology 2 (B). The SGM program was implemented in Biology 1 in 2005 and in Biology 2 in 2006, as indicated by shaded bars. Across all semesters, average grades in Biology 1 were 2.8 for non-URM vs. 2.2 for URM groups (t1165 = −4.386, p < 0.0001), and in Biology 2 were 3.1 for non-URM vs. 2.6 for URM groups (t1000 = −7.23, p < 0.0001). There is a significant interaction between URM status and preprogram vs. postprogram grades in Biology 2 (p < 0.05) but not in Biology 1.

    Student Persistence as STEM Majors and Minors

    To assess whether the persistence of URM students in pursuing biology or STEM degrees has improved since SGM program implementation, we focused on the graduation majors and minors of those who took Biology 1 or 2 as first-year students and graduated in 4 years. Because some features of the program were being developed and refined between 2004 and 2007, we defined the “Before” group here as those who enrolled in either Biology 1 or Biology 2 during the four academic years before the transition (Fall 2000 through Spring 2004). A chi-square test was used to examine the relationships between program (2, before vs. after implementation) × URM status (2, URM vs. non-URM) × persistence (2 levels, biology major or minor vs. not). The same approach was used to consider persistence as NSE majors or minors. For those who took Biology 1 or 2 in those 4 years before SGMs were introduced, 31% of URM students graduated with a major or minor in biology compared with 48% of non-URM students (chi-square = 5.9, p < 0.05; Figure 4A). A significant gap also was evident in the before-program cohort between the percentages of URM (38%) and non-URM (56%) students who majored or minored in any NSE department (chi-square = 7.6, p < 0.01; Figure 4B). In contrast, for those who took Biology 1 or 2 as first-year students after implementation of the SGM program (Fall 2008 through Spring 2012), the persistence of URM students increased to 50% biology majors or minors and 59% NSE majors or minors. As a result, there were no longer differences between URM and non-URM students in the rates of graduating with either biology or NSE degrees (Figure 4, A and B).

    Figure 4.

    Figure 4. Among those who took Biology 1 or 2 in their first year and graduated in 2004–2007 (before program, N = 453, 14.1% URM), smaller percentages of URM (red) students compared with non-URM students (blue) graduated with majors or minors in (A) biology (*, p < 0.05, chi-square test) or (B) any NSE department (**, p < 0.01, chi-square test). By contrast, among those who took the courses in their first year after program implementation and graduated in 2012–2015 (N = 452, 19.9% URM), percentages graduating with majors or minors in biology or other NSE disciplines were not different for URM compared with non-URM groups.

    In the after-program cohorts, 59% of the more-prepared group (who reported AP Biology scores) graduated with majors or minors in biology, compared with only 44% of the less-prepared (non-AP) group (chi-square = 10.1, p < 0.01). Rates of graduating in any NSE discipline also were higher for the AP group (69%) compared with the non-AP group (56%; chi-square = 8.4, p < 0.01). In the more-prepared (AP) population, these outcomes were not related to low or high rates of SGM attendance. For the subset of students in the less-prepared (non-AP) population, however, there was a small correlation between attendance and retention in biology (r = 0.112) that approached significance (p = 0.07). We were not able to quantify the relationships between preparedness and persistence for before-program cohorts, because relevant data (AP Biology status) were not available.

    Student Performance in Subsequent STEM Courses

    Given the evidence for improved persistence of URM students in studying biology and other STEM disciplines since implementation of the SGM program, we also looked for evidence of improved performance in subsequent biology and NSE courses, focusing on those who persisted in studying biology. We defined 4-year before-program and after-program cohorts in the same way as above (skipping 2004–2007, transitional years for the SGM program), and calculated GPAs in subsequent biology and subsequent NSE courses for those with a major or minor in biology-related disciplines (biology, biochemistry, neuroscience). Because both biology GPAs and NSE GPAs were modeled most effectively by linear models that excluded year (see Methods), longitudinal data were pooled in Before and After cohorts and subjected to ANOVAs. The mean overall (Before and After) biology GPA of URM students was 3.25 compared with 3.58 for non-URM students (ANOVA, p < 0.0001; Figure 5A). There was a small increase in biology GPAs after program implementation: the mean overall (URM and non-URM) biology GPA was 3.52 before and 3.54 after the program (ANOVA, p < 0.05; Figure 5A). The mean overall (Before and After) NSE GPA of URM students was 3.03 compared with 3.46 for non-URM students (ANOVA, p < 0.0001; Figure 5B). Overall NSE GPAs did not change after SGM program implementation (Figure 5B). These data provide no evidence for selective effects of the SGM experience on the biology or science GPAs of URM students who persisted in studying biology-related disciplines.

    Figure 5.

    Figure 5. Performance as measured by subsequent GPAs for non-URM (blue) and URM (red) students who persisted as majors or minors in biology-related disciplines (biology, biochemistry, neuroscience) after taking Biology 1 or 2 as first-year students either before (2000–2004) or after (2008–2012) SGM program implementation. (A) GPAs in subsequent biology courses for URM students were different from those for non-URM students (p < 0.0001, ANOVA). There is a small difference in biology GPAs between overall before- and after-program cohorts (*, p < 0.05, ANOVA) but no interaction between URM status and program implementation. (B) GPAs in subsequent NSE courses differed between URM and non-URM students (p < 0.0001, ANOVA). There was no difference between overall before- and after-program cohorts in NSE GPAs and no interaction between URM status and program implementation (before: non-URM N = 185, URM N = 20; after: non-URM N = 183, URM N = 45).

    Enrollments in Postcollege Educational Programs

    Finally, to assess possible impacts of the SGM program experience on career trajectories of biology majors and minors, we tracked postgraduation enrollments for these same cohorts of alumni through the National Student Clearinghouse database (National Student Clearinghouse, StudentTracker Services, data captured November 2015). Because the database does not reliably identify specific programs in which students enrolled, this is an imperfect measure of postgraduation persistence in STEM fields. Nonetheless, more than 50% of biology majors and minors from both URM and non-URM populations enrolled in subsequent educational programs within 2.5 years of their Swarthmore graduation (Figure 6), significantly different rates than the 40–45% of Swarthmore alumni overall (chi-square tests, before p < 0.0001, after p < 0.001, ). The results are not different for those who took Biology 1 or 2 before and after implementation of the SGM program. There were no significant differences associated with either URM status or SGM program experience in these postgraduation enrollment data.

    Figure 6.

    Figure 6. Percentages of graduates who enrolled in a postcollege degree program within 2.5 years of graduating from Swarthmore, comparing graduates from 2004–2007 (before program, 4 years) to graduates from 2012–2013 (after program, 2 years). Enrollment data were captured from the National Student Clearinghouse, StudentTracker Services in November 2015. Postgraduation enrollment rates are plotted for non-URM (blue) and URM (red) subsets of biology majors and minors compared with all graduates of the college (green). Enrollments for biology majors and minors overall are higher than the college average in both before-program (p < 0.0001, chi-square test) and after-program (p < 0.001, chi-square test) cohorts, while the results for non-URM and URM subsets of biology majors and minors are not different (before: bio non-URM N = 185, bio URM N = 20, college N = 1433; after: bio non-URM N = 91, bio URM N = 21, college N = 743).

    DISCUSSION

    There is growing evidence for the value of engaging aspiring science majors in active learning and inquiry-based practices from their earliest engagement with the curriculum in their disciplines. While many biology departments have responded by implementing significant transformations of their introductory curricula to more active and inquiry-based models and course-based undergraduate research experiences (Wei and Woodin, 2011; Knight et al., 2015), many others struggle with the challenges and expenses of revamping the content and style of well-honed sequences of lectures and laboratories. To support the active engagement of introductory biology students with lecture material outside the classroom, we implemented a PLTL program of evening SGMs, with peer SAs who facilitate collaborative problem solving as students work on challenge questions provided by faculty lecturers. The program was introduced in 2005–2006 with two major aims: 1) to improve student performance in introductory biology, and in particular to improve the success of students from underrepresented groups; and 2) to increase the number of students who succeed as biology and STEM majors, including those who enter the college without strong high school backgrounds in biology. Formal assessments were carried out from Fall 2008 through Spring 2012 in order to understand students’ views and reactions to the SGMs and the effects of participation on students’ grades. Subsequent grades and rates of graduating with degrees in biology and other STEM disciplines were tracked through institutional data. A notable outcome after eight semesters of the program was a dramatic improvement in the persistence of URM students in biology and other STEM majors and minors.

    To What Extent Do Students from URM Groups or Those with Weaker Preparation in Biology Use the Program Relative to Counterparts?

    Because attendance at SGM workshops was voluntary, student participation depended on the workshops being perceived as welcoming and useful. Although ∼20% of all students never participated, attendance for the majority of students ranged from once per semester to twice per week, with medians of three SGMs per semester in the overall population and four per semester among targeted groups. This is consistent with findings from other voluntary-attendance PLTL programs; for example, Amstutz et al. (2010) report an average attendance of four sessions per semester. The SGM workshops attracted URM students with greater frequency than their majority counterparts (Figure 1A). The recruitment of students to the SGMs likely stems from both the encouragement of the faculty and laboratory instructors about the program and the accessibility, enthusiasm, and helpfulness of the SAs. The SAs are selected based on their interpersonal skills and affect as well as their facility with course material. In addition, their pedagogical training includes an overview of sociocultural learning theories and cognate practices for teaching diverse learners (see the Supplemental Material).

    What Do Students Perceive as Benefits of Attending SGMs?

    Convergent evidence from end-of-semester surveys indicated that most students perceived SGMs as enhancing their understanding of biology and their abilities to think scientifically and to articulate solutions to problems, in accord with previous studies (Preszler, 2009; Batz et al., 2015). Ratings of benefits were positively correlated with SGM attendance, suggesting that students familiar with the program perceived value and that students who valued the program attended more frequently. Non-AP students perceived the program as facilitating interest in biology, understanding biology, enjoyment of the course, and comfort level in taking biology courses with greater frequencies than those with AP Biology experience (Figure 2B), suggesting that those with more experience in biology perceived less benefit from participating in SGMs.

    Is SGM Program Participation Associated with Improved Performance in Introductory Biology?

    Average grades across 10 years show significant gaps between URM and non-URM students in Biology 1 and Biology 2 (Figure 3). While grades overall did not differ between preprogram and postprogram years, a significant interaction (p < 0.05) between URM status and pre- versus postprogram years in Biology 2 suggests that the historical grade gap between URM and non-URM students narrowed in this course. Results from other PLTL programs similarly reflect small performance effects in some semesters but not others (Drane et al., 2005; Stanger-Hall et al., 2010). These intermittent PLTL-associated performance effects were typically in the range of 0.2−0.3 grade point increases on a 4.0 scale (Drane et al., 2005; Amstutz et al., 2010; Batz et al., 2015). One intriguing observation (Drane et al., 2005) of a marginally significant (p = 0.07) additional gain of 0.1−0.2 grade point for URM students is similar to our findings that suggest a slight narrowing of the grade difference between URM and non-URM students in one of the two SGM-supported courses.

    We also expected that participation in SGMs would improve students’ grades. The data revealed no correlation between attendance and grades in Biology 1 (Fall) and a weak but significant correlation in Biology 2 (Spring), including a small effect for the subset of less-prepared (non-AP) students. It is likely that positive effects of attendance on performance were masked by deliberate recruitment of low-performing students to SGMs in Biology 1 (Fall), in which average grades are significantly lower than in Biology 2 (Spring). Another complicating factor is that most students in Biology 1 were first-semester freshmen taking the course credit/no credit, which likely affects motivation and effort for at least a subset of students. In contrast, Biology 2 students were both more experienced and were usually taking the course for a letter grade.

    Is Program Implementation Associated with Improved Persistence in Biology and Other Natural Sciences?

    The persistence of URM students in studying biology and other STEM disciplines improved significantly after implementation of the SGM program. Mean SAT Math scores of the URM groups did not differ between the before- and after-program cohorts, yet for the after-program cohorts (those who enrolled in Biology 1 or 2 as first-year students in 2008–2012), the same percentages of URM and non-URM students graduated within 4 years with degrees in biology-related or other STEM disciplines (Figure 3). This finding is consistent with expectations based on theories that lecture courses can marginalize students from underrepresented groups and that this effect is potentially ameliorated by opportunities for guided participation with diverse peers (Lee and Fradd, 1998; Price, 2010). While other studies highlight the importance of first-year academic support programs by demonstrating that STEM attrition rates are highest among freshmen (Kramer, 2005), few if any previous studies have documented a relationship between PLTL and graduation with STEM degrees, an objective measure of long-term outcomes.

    In light of the equal rates of persistence in URM and non-URM populations of introductory biology students since SGM program implementation, it is interesting to note that mean GPAs in biology and other NSE courses continued to be significantly lower for URM biology majors and minors compared with majority counterparts (Figure 5). This seems to contradict implications of other studies that persistence is closely associated with performance (Haak et al., 2011). Our results suggest that significantly improved performance as measured by science GPA may not be necessary to improve the persistence of students from underrepresented groups. It is possible that SGM program participation can positively impact persistence by promoting changes in student attitudes about biology, their scientific identities, or their sense of connectedness with people in the department. Our survey results are consistent with this idea, as most students believed that SGMs improved their understanding and confidence in studying biology. Although we did not assess attitudinal factors like science identity and self-efficacy in this study, a consistent and essential feature of the program design is to promote inclusiveness by recruiting and training a diverse group of peer SAs who work to facilitate productive interactions among all students at the SGMs.

    Another goal of the SGM program is to improve the persistence of those who enter the college without strong high school backgrounds in biology. We used the criterion of “reported AP Biology score” as a proxy for prior experience in biology, based on evidence that students with AP experience are better prepared than students without AP experience for the first year of college biology (Scott et al., 2010). The larger non-AP group (who had not reported AP Biology scores) was a far more heterogeneous population in terms of preparation. They were more likely than the AP group to perceive positive benefits of SGMs, and their grades in Biology 2 were positively correlated with SGM attendance. Although they graduated with biology and other STEM degrees at significantly lower rates than their better-prepared (AP) counterparts, it may not be reasonable to expect equal rates of persistence in the AP and non-AP groups. There are likely to be many differences between them, including interest in and commitment to pursuing STEM degrees (Morgan et al., 2013).

    Limitations

    The SGM program was implemented in parallel with other institutional changes that are likely to have impacted student persistence in STEM disciplines. Some high-impact practices, such as writing-intensive courses, have been constant features of the curriculum over the time periods assessed. In contrast, first-year seminars were introduced to the curriculum in 2004. College-wide, more than 80% of students in the after-program cohort years enrolled in first-year seminars, while most students in the before-program group did not. Also, funding for on-campus summer research opportunities in the NSE division increased by 40% in the same interval. Among graduates with NSE majors between 2001 and 2007, 22% of URM and 38% of non-URM groups received summer research fellowships. More recently, among NSE majors graduating between 2008 and 2014, summer fellowships were awarded to 41% of URM and 44% of non-URM students. Importantly, the overall campus climate of inclusiveness has improved, as the student body became increasingly more diverse (increasing from 16% in the before-program cohort to 20% in the after-program cohort), more faculty of color were hired in the NSE division, and faculty generally became more aware of the importance of mentoring and supporting students from diverse backgrounds.

    By recruiting diverse students to be SAs, the SGM program has contributed to a more-inclusive climate in the biology department in a way that is especially salient for first-year students. Self-efficacy (Bandura, 1977) is integral to the formation of scientific identities (Trujillo and Tanner, 2014) and provides a speculative mechanism by which our program may have influenced persistence in biology and other STEM majors. Self-efficacy was not assessed in this study, however, and student attitudes were assessed only in limited ways in the after-program cohorts through faculty-authored items on student surveys.

    Implications for Future Research

    While the aforementioned institutional changes are likely to have positively influenced the persistence of URM students in STEM disciplines, recent analyses of persistence in other NSE departments would argue that the institutional changes are not sufficient to explain the outcomes in biology. While the persistence of URM students who take introductory biology in their first year has improved to the same level as that of majority students, that has not been uniformly true across other NSE departments in the college, including some that have implemented parts of biology’s PLTL program. It will be important for the college to assess and compare features of these programs in the future to identify aspects that are most effective in supporting diverse students. The continued development and analysis of SGMs and these other initiatives may allow further insights into the most important effectors of both persistence and success in biology and other STEM fields.

    The promising finding of a slight correlation between SGM attendance and persistence in biology for the less-prepared (non-AP) group might be related to different degrees of interest and commitment in this population. To implement and evaluate academic and social support structures that effectively improve the success for those with weaker backgrounds who aspire to major in biology, it will be important to develop better criteria for defining that subset of entering students and to include metrics like test scores and assessments of affect and self-efficacy.

    It also will be important to better characterize the possible relationship between self-efficacy and persistence. To enable comparisons across programs, departments, and institutions, future research should use established metrics for self-efficacy in STEM courses and career paths (Fencl and Scheel, 2005; Lent and Brown, 2006; Sawtelle et al., 2012).

    We have begun to use these metrics to analyze initial outcomes of recently implemented academic support programs. These include the new Swarthmore Summer Scholars Program for URM and first-generation college students with an early interest in STEM and our Fall and Spring semester Biology Scholars and Math Scholars Programs, half-credit supplemental instruction courses taken concurrently with introductory biology and math.

    Implications for Teaching

    Because PLTL programs are supplemental to lectures, it is essential that curriculum designers and instructors remain cognizant of the goal of broadening access to STEM content for students from underrepresented groups. This implies that lectures should not simply relay decontextualized scientific minutiae, a strategy described as marginalizing for URM students (Johnson, 2007). URM students begin to associate white race with scientist identity as early as elementary school, getting most of this image association from school and school-provided materials such as textbooks (Walls, 2012). Rather, bridging gaps students perceive between their nascent identities as scientists and their more established identities (e.g. racial, generational) requires 1) explicitly communicating all minutiae in socially relevant contexts, 2) showing images of diverse successful members of the scientific community, and 3) developing sensitivities for how scientific information is presented to avoid microaggression (Tanner and Allen, 2007).

    We speculate our SGM program helped facilitate the formation of students’ scientific identities and cognate self-efficacy beliefs by creating instructional congruence, especially through peer modeling and peer-facilitated active, collaborative work.

    Features of our SGM program we recommend include 1) selection of a diverse group of peer facilitators who have excellent interpersonal skills and characteristics, such as communication and compassion, and who model enthusiasm and self-efficacy in science; 2) training facilitators in pedagogical principles and practices, including sociocultural methods; 3) providing faculty-generated question sets suitable for group discussion and problem solving; and 4) providing space and time outside the classroom for regularly scheduled workshops.

    We are hopeful that PLTL programs and other progressive efforts at academic support, based on established educational principles and emerging evidence, will help make STEM education more inviting and inclusive for students of diverse backgrounds.

    ACKNOWLEDGMENTS

    Initial funding for the SGM program was provided by HHMI grants (52005202 and 52006315) to Swarthmore College. We thank E. Nichols and P. Borkowski-Valentin for crucial assistance with data analysis; M. Powell for help with figures; and A. Vollmer, B. Davidson, V. Formica, S. Hiebert Burch, N. Kaplinsky, and R. Merz for sharing their challenge questions. We are grateful to C. Hungerford and T. Stephenson and all faculty and laboratory instructors in the Swarthmore College Biology Department for their consistent support for the development, implementation, and assessment of the SGM program.

    REFERENCES

  • Aikenhead GS (1996). Science education: border crossing into the subculture of science. Stud Sci Educ 27, 1-52. Google Scholar
  • Alessandria KP (2002). Acknowledging white ethnic groups in multicultural counseling. Family J 10, 57-60. Google Scholar
  • Amstutz M, Wimbush K, Snyder D (2010). Effectiveness and student demographics of peer-led study groups in undergraduate animal science courses. North Am Coll Teach Agric J 54, 76-81. Google Scholar
  • Bandura A (1977). Self-efficacy: toward a unifying theory of behavioral change. Psychol Rev 84, 191-215. MedlineGoogle Scholar
  • Batz Z, Olsen BJ, Dumont J, Dastoor F, Smith MK (2015). Helping struggling students in introductory biology: a peer tutoring approach that improves performance, perception and retention. CBE Life Sci Educ 14, ar-16. Google Scholar
  • Born WK, Revelle W, Pinto LH (2002). Improving biology performance with workshop groups. J Sci Educ Technol 11, 347-365. Google Scholar
  • Brickhouse N (1994). Bringing in the outsiders: reshaping the sciences of the future. Curr Stud 26, 401-416. Google Scholar
  • Brickhouse NB, Lowry P, Schultz K (2000). What kind of a girl does science? The construction of school science identities. J Res Sci Teach 37, 441-458. Google Scholar
  • Brown BA (2004). Discursive identity: assimilation into the culture of science and its implications for minority students. J Res Sci Teach 41, 810-834. Google Scholar
  • Chambers DW (1983). Sterotypic images of the scientist: the Draw-A-Scientist test. Sci Educ 67, 255-265. Google Scholar
  • Collins A, Brown JS, Newman SE (1989, Ed. LB Resnick, Cognitive apprenticeship: teaching the craft of reading, writing, and mathematics In: Knowing, Learning, and Instruction: Essays in Honor of Robert Glaser, Hillsdale, NJ: Lawrence Erlbaum Associates, 453-494. Google Scholar
  • Crowe A, Dirks C, Wenderoth MP (2008). Biology in Bloom: implementing Bloom’s taxonomy to enhance student learning in biology. CBE Life Sci Educ 7, 368-381. LinkGoogle Scholar
  • Dennen VP, Burner KJ (2007, Ed. JM SpectorMD MerrillJ Van MerrienboerMP Driscoll, The cognitive apprenticeship model in educational practice In: Handbook of Research on Educational Communications and Technology, 3rd ed., Mahwah, NJ: Erlbaum, 425-439. Google Scholar
  • Drane D, Smith HD, Light G, Pinto L, Swarat S (2005). The Gateway Science Workshop program: enhancing student performance and retention in the sciences through peer-facilitated discussion. J Sci Educ Technol 14, 337-352. Google Scholar
  • Eberlein T, Kampmeier J, Minderhout V, Moog RS, Platt T, Varma-Nelson P, White HB (2008). Pedagogies of engagement in science: a comparison of PBL, POGIL, and PLTL. Biochem Mol Biol Educ 36, 262-273. MedlineGoogle Scholar
  • Fencl H, Scheel K (2005). Engaging students: an examination of the effects of teaching strategies on self-efficacy and course climate in a nonmajors physics course. J Coll Sci Teach 35, 20-24. Google Scholar
  • Freeman S, Eddy S, McDonough M, Smith MK, Okoroafor N, Jordt H, Wenderoth MP (2014). Active learning increases student performance in science, engineering and mathematics. Proc Natl Acad Sci USA 111, 8410-8415. MedlineGoogle Scholar
  • Fullilove RE, Treisman PU (1990). Mathematics achievement among African American undergraduates at the University of California, Berkeley: an evaluation of the Mathematics Workshop Program. J. Negro Educ 59, 463-478. Google Scholar
  • Gilbert A, Yerrick R (2001). Same school, separate worlds: a sociocultural study of identity, resistance, and negotiation in a rural, lower track science classroom. J Res Sci Teach 38, 574-98. Google Scholar
  • Haak D, HilleRisLambers J, Pitre E, Freeman S (2011). Increased structure and active learning reduce the achievement gap in introductory biology. Science 332, 1213-1216. MedlineGoogle Scholar
  • Harackiewicz JM, Canning EA, Tibbetts Y, Giffen CJ, Blair SS, Rouse DI, Hyde JS (2013). Closing the social class achievement gap for first-generation students in undergraduate biology. J Educ Psychol 106, 375-389. Google Scholar
  • Johnson A (2007). Unintended consequences: how science professors discourage women of color. Sci Educ 91, 805-821. Google Scholar
  • Knight KK, Wise SB, Rentsch J, Furtak E (2015). Cues matter: learning assistants influence introductory biology student interactions during clicker-question discussions. CBE Life Sci Educ 14, ar41. LinkGoogle Scholar
  • Kramer L (2005). Explaining faculty involvement in women’s retention. In: Proceedings of the 2005 American Society for Engineering Education Annual Conference & Exposition, 12–15 June 2005. Washington, DC: American Society for Engineering Education. Google Scholar
  • Lee O (1997). Diversity and equity for Asian American students in science education. Sci Educ 81, (), 107-122. Google Scholar
  • Lee O, Fradd SH (1998). Science for all, including students from non-English language backgrounds. Educ Res 27, (4), 12-21. Google Scholar
  • Lee O, Fradd SH, Sutman FX (1995). Science knowledge and cognitive strategy use among culturally and linguistically diverse students. J Res Sci Teach 32, 797-816. Google Scholar
  • Lent RW, Brown SD (2006). On conceptualizing and assessing social cognitive constructs in career research: a measurement guide. J Career Assess 14, 12. Google Scholar
  • Morgan SL, Gelbgiser D, Weeden KA (2013). Feeding the pipeline: gender, occupational plans, and college major selection. Soc Sci Res 42, 989-1005. MedlineGoogle Scholar
  • Preszler RW (2009). Replacing lecture with peer-led workshops improves student learning. CBE Life Sci Educ 8, 182-192. LinkGoogle Scholar
  • Price J (2010). The effect of instructor race and gender on student persistence in STEM fields. Econ Educ Rev 29, 901−910. Google Scholar
  • Rahm J (2008). Urban youths’ hybrid positioning in science practices at the margin: a look inside a school-museum-scientist partnership project and an afterschool science program. Cult Stud Sci Educ 3, 97-121. Google Scholar
  • Sarquis JL, Dixon LJ, Gosser DK, Kampmeier JA, Roth V, Strozak VS, Varma-Nelson P (2001, Ed. JE MillerJE GrocciaM Miller, The workshop project: peer-led team learning in chemistry In: Student-Assisted Teaching: A Guide to Faculty-Student Teamwork, Bolton MA: Anker, 150-155. Google Scholar
  • Sawtelle V, Brewe E, Kramer L (2012). Exploring the relationship between self-efficacy and retention in introductory physics. J Res Sci Teach 49, 1096-1121. Google Scholar
  • Scott TP, Tolson H, Lee Y (2010). Assessment of Advanced Placement participation and university academic success in the first semester: controlling for selected high school academic abilities. J Coll Admission 208, 27-30. Google Scholar
  • Smith MK, Wood WB, Adams WK, Wieman C, Knight JK, Guild N, Su TT (2009). Why peer discussion improves student performance on in-class concept questions. Science 323, 122-124. MedlineGoogle Scholar
  • Snyder JJ, Carter E, Wiles JR (2015). Implementation of the peer-led team-learning instructional model as a stopgap measure improves student achievement for students opting out of laboratory. CBE Life Sci Educ 14, ar-2. Google Scholar
  • Stanger-Hall KF, Lang S, Maas M (2010). Facilitating learning in large lecture classes: testing the “teaching team” approach to peer learning. CBE Life Sci Educ 9, 489-503. LinkGoogle Scholar
  • Tanner K, Allen D (2007). Cultural competence in the college biology classroom. CBE Life Sci Educ 6, 251-258. LinkGoogle Scholar
  • Trujillo G, Tanner K (2014). Considering the role of affect in learing: monitoring students’ self-efficacy, sense of belonging, and science identity. CBE Life Sci Educ 13, 6-15. LinkGoogle Scholar
  • Walls L (2012). Third grade African American students’ views of the nature of science. J Res Sci Teach 49, 1-37. Google Scholar
  • Wei CA, Woodin T (2011). Undergraduate research experiences in biology: alternatives to the apprenticeship model. CBE Life Sci Educ 10, 123-131. LinkGoogle Scholar
  • Woodward A, Gosser DK, Weiner M (1993). Problem-solving workshops in general chemistry. J Chem Educ 70, 651-652. Google Scholar