Improving Academic Performance, Belonging, and Retention through Increasing Structure of an Introductory Biology Course
Abstract
Integration of active-learning approaches into increased-structure postsecondary classrooms significantly improves student academic outcomes. We describe here two parallel sections of Introductory Biology that shared learning objectives and content but varied in course structure. The large-enrollment traditional course consisted of four 50-minute lectures coupled with minimal active-learning techniques, while an increased-structure intervention course integrated multiple active-learning approaches, had limited enrollment, and comprised three 50-minute lectures combined with a fourth peer-led team-learning discussion section. Additionally, the intervention course employed weekly review quizzes and multiple in-class formative assessments. The academic impact of these two course formats was evaluated by use of common exam questions, final grade, and student retention. We showed that academic achievement and retention of participants enrolled in the intervention course was significantly improved when compared with the traditional section. Further, we explored whether promoting in-class student–student/student–instructor interactions and peer-led discussion sections fostered a greater sense of belonging. At the end of the course, participants in the intervention course reported greater perceptions of classroom belonging. Therefore, this study begins to characterize the importance of combining pedagogical methods that promote both academic success and belonging to effectively improve retention in science, technology, engineering, and mathematics majors.
INTRODUCTION
Fewer than 40% of U.S. students entering into science, technology, engineering, and mathematics (STEM) majors complete their intended degree upon graduation (President’s Council of Advisors on Science and Technology [PCAST], 2012). This observation and the PCAST prognostication that the United States will require a 33% increase in STEM degree–holding individuals to support the growing knowledge economy have been followed by empirical assessment of the efficacy of various teaching practices employed at postsecondary institutions. Analysis of educational outcomes has revealed significant academic performance gaps between underrepresented minority (URM) or first-generation students and their continuing-generation peers in university STEM disciplines (Freeman et al., 2007a, 2011). These differences have resulted in American postsecondary institutions losing student diversity in science majors as individuals progress toward their degrees (National Science Foundation, 2006). The retention disparity results in a loss of diversity in the professional sciences at the national level (Anderson and Kim, 2006; Estrada et al., 2016), despite the increased demand for STEM graduates to fill increasing job vacancies. To meet this need, it is imperative that educators effectively instruct and retain all capable students. Therefore to address this issue, discipline-based educational research (DBER) of STEM pedagogical techniques has sought to construct and subsequently analyze the effectiveness of modified course structures that promote the success of the diverse student populations that enroll in science majors.
Courses with increased structure incorporate multiple formative assessments or interventions, including preclass videos/readings coupled with quizzes, in-class active-learning modules, weekly mock midterms, and small near-peer collaborative workshops (including process-oriented guided-inquiry learning) can effectively promote improved student academic performance (Ebert-May et al., 1997; Knight and Wood, 2005; Beichner et al., 2007; Freeman et al., 2007a, 2011, 2014; Walker et al., 2008; Armbruster et al., 2009; Moravec et al., 2010; Haak et al., 2011; Gross et al., 2015; Marbach-Ad et al., 2016; Cleveland et al., 2017). Interestingly, the frequency and diversity of structure or interventions enhances their effectiveness for academic improvement by participating students; courses that incorporate multiple interventions have been shown to promote student mastery to a higher level than less structured courses (Freeman et al., 2011; Adams et al., 2016; Connell et al., 2016; Styers et al., 2018). Beyond helping the general student population, increased-structure courses have been shown to promote academic equity among diverse students in multiple educational settings and contexts (Haak et al., 2011; Eddy and Hogan, 2014; Carmichael et al., 2016; Gavassa et al., 2019). Despite the wealth of evidence that increased-structure courses promote student academic success, implementation of active-learning or increased-structure course designs alone does not guarantee student improvement (Andrews et al., 2011). Collectively, this field of research suggests that instructor and institutional variables, as well as student experience and identity, must be accounted for in the design and implementation of increased-structure courses to effectively eliminate academic inequities.
Retention in STEM majors is not solely influenced by academic performance; there are multiple university, instructional, and student variables that can also impact retention rates. Classroom social belonging has been demonstrated to positively influence student learning and retention in STEM (Hausmann et al., 2007; Hurtado et al., 2007; Walton and Cohen, 2011; Ballen et al., 2017). Near-peer (student–student) connections are integral to fostering student perceptions of belonging and increased student retention, including first-generation and URM student groups (Strayhorn, 2008; Meeuwisse et al., 2010). The positive impact of innovative teaching on student retention in STEM majors has been well documented (Strenta et al., 1994; Dougherty et al., 1995; Moore and Miller, 1996; Watkins and Mazur, 2013). One recent study showed that active-learning techniques led to significant improvements in student self-efficacy among all participants, but improved social belonging only among non-URM students (Ballen et al., 2017). However, use of undergraduate learning assistants (based on the Colorado Learning Assistants; Otero et al., 2010) to foster these student interactions in guided peer-led team-based learning (PLTL; Wamser, 2006) of course content has been demonstrated to improve social belonging and scientific self-efficacy among all student demographics (Tien et al., 2002; Lewis and Lewis, 2005; Wamser, 2006; Hockings et al., 2008; Otero et al., 2010; Batz et al., 2015; Knight et al., 2015; Kudish et al., 2016; Snyder et al., 2016; Sellami et al., 2017; Stanich et al., 2018). Therefore, incorporating multiple active-learning techniques, coupled with PLTL discussions, has the potential to eliminate academic inequity while promoting self-efficacy and belongingness across a diversity of students.
Historically, the second-year Introductory Biology I course at the public R1 University of California, Santa Barbara (UCSB), was offered in one section and maintained a high enrollment of 1100 students. This population was composed of majors as well as nonmajors and included large populations of URM (33%), educational opportunities program–eligible (EOP eligibility is conferred by parental socioeconomic status; 30.4%), or low-income students (38.7%). Of the incoming 1100 first-year students who declare biology as a major, ∼50% leave the major by the end of their second year of study; further, disproportionately more URM and EOP students leave the major compared with their continuing-generation peers, leading to a significant decrease in student diversity present in the biology majors in a graduating class. To confront the loss of students at this critical second-year juncture, we sought to implement an entirely parallel increased-structure course (hereafter referred to as the “intervention course”) that combined multiple active-learning strategies while requiring participation in a learning assistant–mentored, small-group discussion section that conducted PLTL.
We describe here our analysis of the two parallel course offerings of Introductory Biology I over 3 academic years, 2015–2017. Given the demonstrated effectiveness of active-learning strategies on student academics and the importance of belonging in student retention, we sought to analyze whether the intervention course promoted overall student academic success while decreasing the aforementioned observed academic inequities of EOP and URM students at UCSB. To address this, we sought to answer three questions. First, does the increased structure of the intervention course lead to improved within-class performance as measured by common exam questions and earned final grade? Second, did the increased peer–peer and peer–mentor interactions of the intervention course promote a greater sense of student belonging in Introductory Biology I? And third, does the intervention course improve short-term student retention into the subsequent Introductory Biology II course?
METHODS
Course Design
In recent years, enrollment in Introductory Biology I was approaching 1100 second-year students. Of this population, ∼600 enrollees were declared biology majors. The traditional section of Introductory Biology I is broken into thirds (biochemistry, cell biology, and genetics) and is taught by three faculty with disparate teaching styles. Beginning in 2015, a second intervention course that implemented multiple active-learning strategies, taught by two co-instructors, was added to the university schedule. Table 1 summarizes the structure and enrollment of each section; in what follows, we provide details on key differences between these courses.
| Traditional | Intervention | |
|---|---|---|
| Sample description | ||
| Years included in analysis | 2015–2017 | 2015–2017 |
| Total enrollment in course (n) | n2015 = 882n2016 = 683n2017 = 721 | n2015 = 127n2016 = 263n2017 = 282 |
| Number of students in analysis (only includes declared biology majors with second-year standing, course repeaters excluded) | n = 1029 | n = 583 |
| Number of cohorts in analysis | n = 3 | n = 3 |
| Course design | ||
| Student population | Mixed majors (only biology majors are included in analyses) | Predominantly biology majors (only biology majors included in analyses) |
| Number of 50-minute lectures per week | n = 4 | n = 3 |
| Number of 50-minute discussion sections per week | n = 0 | n = 1 |
| Discussion section format | N/A | Enrollment = 30–35 students/section,Facilitated by graduate teaching assistant or instructor and two to three upper-division peer-learning assistants |
| Discussion section activities | N/A | Graded collaborative group problem sets |
| In-lecture activities | Lecture only,iClicker questions in last 3 weeks (one-third) of course | iClicker questions, think–pair–share, mock exam questions, whole-class discussions, muddiest point |
| Outside-lecture activities | Weekly, learning management system (LMS)-delivered review quizzes in last 3 weeks (one-third) of course | Weekly, LMS-delivered review quizzes (entire course), discussion sections (described earlier), targeted-reading assignments, preclass instructional videos |
| Components of final grade | 25% = midterm exam25% = midterm exam50% = cumulative final examBonus 2% for iClicker and LMS review quizzes | 10% = weekly LMS review quizzes, iClicker questions, weekly discussion sections20% = midterm exam20% = midterm exam50% = cumulative final exam |
| Qualitative description of course design | ||
| Description | Primarily lecture | Increased course structure, biology majors only, peer-led team learning in discussion section |
Instructors of the intervention (M.W. and E.G.) and traditional (R.C.) Introductory Biology I courses are involved in the analysis and authorship of this study. The five faculty of Introductory Biology I met throughout the quarter to ensure that both courses maintained the shared prescribed instructional objectives and course content; however, the instructors of the traditional section do not coteach with the instructors of the intervention course. The intervention course mirrors the content and student learning outcomes of the traditional section but is composed solely of declared biology majors and team-taught by two co-instructors who collaboratively design, participate, and evaluate the course. The intervention course uses discussion sections that are 50-minute, ∼30- to 35-student PLTL sessions led by an instructor each week. The first of the eight sections is led by the instructor to demonstrate for the graduate teaching assistant, who leads the subsequent seven sections. In each section, there are two undergraduate learning assistants who participate by circulating through the classroom while guiding student questioning and conversation. Over the 10-week quarter, students worked collaboratively with their peers in groups of four to six people to complete a group assignment that focused on topics or concepts that students had struggled with historically, based on analyzing previous exam results. Students were prompted to treat these groups as learning communities beyond the classroom setting. This encouraged students to work collaboratively to solve online weekly review quizzes, complete targeted reading assignments modeled after Lieu et al. (2017), or consistently distribute their studies for course examinations outside scheduled class times. Instructors, graduate teaching assistants, and undergraduate learning assistants were trained to promote student–student, near-peer mentoring, rather than act as tutors in the discussion section meetings.
Within the lecture portion of the intervention course, the instructors used multiple active-learning practices that engaged students with one another, including think–pair–share, mock exam questions, and class-wide discussions. Review of the 50-minute lecture recordings enabled us to identify the average number of active-learning activities per course. On average, there were approximately five active-learning activities in the intervention course and approximately none to two in the traditional section depending on the instructor.
Course Demographics
The pooled demographic composition of the traditional and intervention courses of Introductory Biology I from 2015 to 2017 are shown in Table 2. The UCSB designates students who are Black, Hispanic, Native American/Alaskan, or Native Hawaiian/Pacific Islander as URMs. Approximately 30% of the total student population at UCSB self-identify as Hispanic, Latin@, or Chican@, making UCSB a Hispanic-serving institution. Students from traditionally disadvantaged economic and/or educational backgrounds (e.g., first-generation college students) are eligible to participate in the EOP at UCSB. It is important to note that the EOP and URM student populations enrolled in Introductory Biology I overlap (∼40% are both EOP/URM).
| Total | Traditional | Intervention | |
|---|---|---|---|
| Biology major students (n) | 1612 | 1029 | 583 |
| Female | 998 (62%) | 627 (61%) | 371 (63.5%) |
| URM | 395 (24.5%) | 257 (25%) | 138 (24%) |
| EOP | 480 (30%) | 308 (30%) | 172 (29.5%) |
Summary of Analyses Performed
To analyze whether the intervention course promoted improved academic performance, perceptions of belonging, and retention, we analyzed within-course exam performance and final course grade, assessed student sense of belonging, and tracked student retention into the subsequent biology course, respectively. Given the disproportionate loss of student diversity and the inequity faced by certain demographic groups in UCSB biology, part of our analysis characterizes whether the intervention course had a disproportionate impact on URM and EOP student populations. We summarize our approach to data analysis in Table 3. We describe specific methods of analysis in detail in the following sections.
| Result | Method | Years analyzed |
|---|---|---|
| Exam performance | Wald’s t test statistical analysis of correctly answered common exam questions between traditional and intervention section | 2015–2017, each year analyzed independently |
| Final course grade | Multilevel linear regression analysis | 2015–2017 combined with cohort year as random intercept variable |
| Sense of belonging | Multiple linear regression analysis | 2017, end of quarter |
| Student retention in subsequent Introductory Biology II course | Multiple logistic regression analysis and mediation analysis | 2015–2017 combined |
Common Exam Assessments
Given that both sections of the course were offered simultaneously and shared primary learning objectives, common exam questions were employed to quantify the impact of course design on student academic performance while controlling for yearly changes in student demographics. Importantly, certain topics were delivered to students of each section strictly by lecture; common questions on these topics act as negative controls enabling the measurement of baseline academic performance between sections. To assess whether there were differences between the traditional and intervention courses, we calculated the proportion of correctly answered shared questions for each student in each course section. Given that individual student demographic data could not be connected to an individual’s exam performance by UCSB Institutional Research, we had to employ Wald’s t test statistical analysis rather than conduct regression analysis to assess statistical significance.
Regression Analyses of Introductory Biology I Final Grade, Sense of Belonging, and Short-Term Retention
For the remaining study outcomes, Institutional Research provided demographic variables that were connected to individual students enrolled in each section of Introductory Biology I; therefore, for the subsequent analyses, we built regression models to compare differences between the two course structures while allowing for changes in variables previously linked to success in STEM courses (Theobald and Freeman, 2014). These included academic measures like total Scholastic Aptitude Test (SAT) score (Freeman et al., 2007a) and demographic variables like URM status (Haak et al., 2011) or EOP enrollment. Additional variables included in the models were course structure (intervention or traditional), gender, ethnicity, UCSB cumulative science grade point average (GPA) at the end of first-year Spring quarter (significant variable in predicting fourth-year retention in major; Supplemental Table 1), and Introductory Biology I final grade. Given that our study has course sections nested within cohort year, our observations are not truly independent; therefore, we used 2015–2017 cohort year and section within cohort year as grouping variables used for random intercepts. Regression analyses presented and discussed are the models with the lowest Akaike information criterion (AIC; described further below in Introductory Biology I Final Grade Analysis and outlined by Theobald, 2018).
Introductory Biology I Final Grade Analysis
To compare final grade differences between the two course structures, we built a multilevel linear regression model using cohort year as a random intercept to allow for variation in the student population across years (Theobald, 2018). We fit all multilevel models using the lme4 package in R (Bates et al., 2015; Pinheiro et al., 2018; using RStudio: Integrated development environment for R (Version 1.0.143) ( www .rstudio.com/). Because the numerical variables of total SAT scores, cumulative science GPA, and our response variable of Introductory Biology I final grade differed by orders of magnitude, the data were scaled and centered before identification of the best-fitting regression model so that an average value (SAT score or grade) is equal to 0. We specified an initial model allowing for pairwise interactions between fixed effects of the intervention and demographic variables, along with the specified random intercept. Then we compared our full model with a model without the random intercept using AIC to determine whether the random intercept was important. We confirmed our selection by performing a nested-model analysis of variance (ANOVA) between the models and analyzing caterpillar plots of the random intercepts (Winter, 2013; Theobald, 2018; caterpillar plot produced using code from StackOverflow Caracal, 2013, https://stackoverflow.com/a/16511206).
Sense of Student Belonging Analysis
We were interested in assessing whether the intervention course design with near peer–led discussion sections might promote increased perceptions of classroom belonging, which in turn might influence student retention (Solanki et al., 2019). Therefore, students’ perceptions of class belonging were collected by a Qualtrics survey conducted at the end of the 2017 Fall quarter. We modified a previously published survey that has been validated with undergraduate students (Hoffman et al., 2002) and that has been recently used with a demographically similar population (Solanki et al., 2019). The question items targeted three noncognitive measures, including belongingness, motivation, and growth mindset (Dweck and Leggett, 1988; Hoffman et al., 2002). The 12-item survey instrument consisted of subscales of perceived peer support (five items), perceived faculty support (five items), and perceived classroom comfort (two items), as determined by previous factor analyses (Tovar and Simon, 2010). These items on motivation and belongingness were adapted to be course specific, asking explicitly about these constructs with respect to this Introductory Biology I class. Student completion of the surveys was tied to course credit in the laboratory course, Introductory Biology Lab I (MCDB 1AL), associated with the Introductory Biology I course (MCDB 1A), which yielded a response rate of ∼61%.
For the sense of belonging items, principal components factor analysis on 12 items pertaining to students’ sense of belonging was conducted, using promax rotation to allow for correlation between factors. Cronbach’s alphas (α) were then checked for each factor’s reliability. Consistent with the findings of previous factor analyses (Tovar and Simon, 2010), this produced a three-factor structure consisting of perceived peer support (α = 0.85), perceived faculty support (α = 0.85), and perceived classroom comfort (α = 0.76). A full list of items by construct can be found in the Supplemental Material.
To control for additional variables in these associations, multiple regressions predicting sense of belonging and each of its three subscales were conducted in STATA 15 (StataCorp, 2017). These regressions controlled for ethnicity, EOP status, gender, and prior academic achievement, including total SAT scores and the first-year cumulative science GPA. In addition, the course format’s association with sense of belonging was also tested for interactions with each control variable.
Short-Term Student Retention Analysis
Declared biology majors participating in either the intervention course or the traditional section of Introductory Biology I in the Fall quarter were tracked via unique identifiers to determine whether they progressed into the subsequent majors’ required course, Introductory Biology II, offered in the Winter quarter. To determine whether cohort year was a significant predictor of student retention over the three-year study period, we built a multilevel logistic regression model with the effect of cohort year modeled as a random intercept. Using the model selection criteria outlined earlier in the section on regression analyses, we identified the logistic regression model had the lower AIC and confirmed that the random effect variable was not significant by performing an ANOVA. Subsequently, using the same variables outlined earlier, we performed a mediation analysis via the methods of Pearl (2001) and Robins and Greenland (1992). We hypothesized that students in the intervention section would be more likely to be retained in the biology course sequence and that this effect would be mediated by the grade received in Introductory Biology I. Specifically, we were interested in the indirect effect that results from grade being a mediator, following guidelines provided by Hayes (2009). All mediation analyses were done using the mediation package in R (Tingley et al., 2014).
Human Subjects and Study Inclusion
This study was conducted under the guidelines of the UCSB Office of Research Human Subjects approved Institutional Review Board (IRB) protocol number 2-17-0610. Under this IRB, demographic data, course retention information, and final grade data were available for all enrolled students, but we were unable to link individual exam performance data to demographic variables. For assessment of student perceptions of belonging, students were invited to participate though an email distributed by UCSB Institutional Research. Bonus course credit for the associated Introductory Biology I lab course was offered to all participants. Those who completed the survey were given an option to opt out of their responses being used in this study. Those who selected to opt out of the belonging survey were removed from the data gathered but still received bonus course credit. Those participants who remained were anonymized through the removal of all identifiers by Institutional Research before analysis.
RESULTS
Here, we present evidence that the majority of student attrition and loss of diversity in the UCSB biology major occurs within the first 2 years of study. We subsequently describe the impacts of the second-year intervention course on student academic performance and perceptions of belonging. These analyses enabled us to 1) identify student variables that predict student academic performance; 2) compare the effectiveness of the intervention course on common exam questions and earned final grade; 3) explore whether increased student–student and student–instructor interactions influence student belonging; and 4) analyze whether the intervention course improved second-year student retention.
Demographics of Introductory Biology
The 1100 students who declare biology yearly at UCSB are representative of the overall campus demographics. However, within the first 2 years of study, the number of declared biology majors decreases by ∼50% (Figure 1A). This loss of students is partially explained by significantly lower cumulative science GPAs and leads to significantly more URM and EOP students transferring out of the major compared with their continuing-generation peers (Supplemental Tables 1 and 2). The majority of UCSB students who leave the biology major go on to complete their degrees in a non-STEM field (Figure 1B). A summary of student demographics of participants in our Introductory Biology I course, stratified by gender, EOP, and racial status can be found in Table 2. Multilevel logistic regression analysis of student demographics (gender, ethnicity, EOP status) and prior academic performance (first-year cumulative science GPA, total SAT scores) revealed that, although most identified variables did not differ between the two sections, total SAT scores did (Supplemental Tables 3 and 4). Given our focus on assessing whether the intervention section improved academic performance, sense of belonging, and short-term retention, these variables are included in all subsequent regression analyses.
FIGURE 1. EOP students are retained in the biology major at a lower rate than their non-EOP/continuing-generation peers. (A) Student enrollment in the biology major during a representative year. Presented are the raw numbers of EOP or non-EOP students at each level of study during one academic year. Data presented are representative of current student demographics (the 2011 entry cohort is presented). (B) Declared majors analysis of three separate cohorts of EOP and non-EOP students who entered and declared biology in the first year of study. Cohorts of students who entered in 2007 (cohort 1), 2006 (cohort 2), and 2005 (cohort 3) were tracked over 4 years. Data presented are the declared majors (biology, STEM, or non-STEM) of the cohorts upon graduation.
Increased Structure Improves Student Performance on Common Exam Questions
Exam-based assessments of student performance in each section of the Introductory Biology I course were conducted using two midterms and a cumulative final. Common question content on each exam ranged from 24% to 70% of the total question set each year. Those enrolled in the intervention course, in which content was delivered by active-learning practices, outperformed their peers in the traditional lecture section by ∼12% on common exam questions (Figure 2A; Fall 2017 n = 57, Welch’s two-sample t test, p value = 6.523e-16; data were similar for Fall 2015 and Fall 2016; Supplemental Figure 1). Topics delivered to students in both the traditional and intervention courses strictly by lecture showed no significant difference between the two courses, suggesting that increased student grades earned in the intervention class were likely not solely due to differences in student populations between the sections (Figure 2B; Fall 2017 n = 22, Welch’s two-sample t test, p value = 0.1294).
FIGURE 2. The increased-structure section outperforms the traditional section on common exam questions. Box plots represent the average number of correct responses on the shared exam questions (±SD). (A) Comparison of 54 shared exam questions of topics delivered by active-learning methods in the intervention course. The asterisk denotes statistically significant differences as quantified by a Welch’s two-sample t test (p = 6.523 e-16, traditional course mean = 69.157, intervention course mean = 82.765). (B) Comparison of 22 shared exam questions of topics delivered only by lecture in the intervention and traditional sections. Correct response rates were not significantly different between the two sections as quantified by a Welch’s two-sample t test (p = 0.1294, traditional course mean = 73.121, intervention course mean = 75.885). Data presented are for the Fall 2017 cohort; highly similar results were obtained for Fall 2015 and Fall 2016 (Supplemental Figure 1).
Intervention Participants Earn Higher Final Course Grades
To assess whether the design of Introductory Biology I significantly impacted final course grade, we fit a multiple linear regression using data from the past 3 years (2015–2017) that best explained observed variation in earned student grade. To determine whether year to year variation significantly impacted our model, we performed multilevel linear regression analysis incorporating cohort year as a random intercept. Interestingly, the random intercept for cohort year was statistically significant, as indicated in the ANOVA table and caterpillar dot plot of random effect size of the cohort years variable (Supplemental Table 5 and Supplemental Figure 2). Further, the multilevel linear regression had a lower AIC than the multiple linear regression, resulting in our selection of the former as the most appropriate model. In our multilevel model, enrollment in the intervention course is associated with a significantly higher Introductory Biology I final grade when compared with the traditional section (Table 4). However, interaction terms of URM or EOP variables and course section were insignificant, highlighting that URM or EOP students in the intervention course are not disproportionately increasing their final grades relative to their non-URM or non-EOP peers in the same section.
| Coefficients | Estimate (β) | SE | t value | Pr(>|t|) | pa |
|---|---|---|---|---|---|
| (Intercept) | −1.742e-01 | 6.610e-02 | −2.636 | 0.03921 | * |
| Intervention course | 3.168e-01 | 7.478e-02 | 4.236 | 2.41e-05 | *** |
| Gender male | 5.582e-03 | 4.579e-02 | 0.122 | 0.90300 | |
| Ethnicity Caucasian | 1.491e-01 | 5.375e-02 | 2.774 | 0.00560 | ** |
| Ethnicity URM | 2.206e-02 | 6.079e-02 | 0.363 | 0.71677 | |
| EOP | −2.882e-02 | 5.465e-02 | −0.527 | 0.59807 | |
| Total SAT score | 1.526e-01 | 2.226e-02 | 6.858 | 1.00e-11 | *** |
| Cumulative science GPA Spring first-year | 5.727e-01 | 2.001e-02 | 28.628 | <2e-16 | *** |
| Intervention course × gender male | 6.535e-02 | 7.668e-02 | 0.852 | 0.39423 | |
| Intervention course × ethnicity Caucasian | −3.806e-02 | 8.856e-02 | −0.430 | 0.66743 | |
| Intervention course × ethnicity URM | 1.316e-01 | 9.842e-02 | 1.337 | 0.18150 | |
| Intervention course × EOP | −8.176e-02 | 8.730e-02 | −0.937 | 0.34914 |
Increased-Structure Introductory Biology May Promote Student Belonging
Next, we sought to explore the extent to which the intervention course design influenced student perceptions of belonging to the course. The end-of-quarter survey administered in 2017 was completed by 61% (n = 162) of the traditional course and 61% (n = 171) of the intervention course biology majors. Participation in the intervention course was associated with a significantly higher perception of student belonging when compared with peers in the traditional section (Table 5; all survey data are included in the survey in the Supplemental Material). A regression analysis, controlling for ethnicity, EOP status, gender, and achievement (total SAT scores and cumulative science GPA), revealed that the increase in overall sense of belonging attributable to course structure is equal to 0.33 SDs (Table 5). Further, subscale analyses showed that this association with students’ overall sense of belonging was primarily driven by higher perceptions of faculty support (0.40 SD) and classroom comfort (0.27 SD), not higher perceptions of peer support (0.09 SD).
| Sense of belonging (all) | Perceived peer support | Perceived faculty support | Classroom comfort | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| m1 | m2 | m3 | m1 | m2 | m3 | m1 | m2 | m3 | m1 | m2 | m3 | |
| Intervention course | 0.33*** | 0.36*** | 0.30*** | 0.09 | 0.00 | 0.03 | 0.40*** | 0.46*** | 0.36*** | 0.27*** | 0.27* | 0.31*** |
| (0.06) | (0.13) | (0.08) | (0.12) | (0.24) | (0.14) | (0.10) | (0.20) | (0.12) | (0.13) | (0.25) | (0.15) | |
| Interactions | ||||||||||||
| Intervention course × race | ||||||||||||
| Intervention course | −0.02 | 0.02 | 0.02 | −0.05 | ||||||||
| × Asian (vs. URM) | (0.16) | (0.31) | (0.26) | (0.32) | ||||||||
| Intervention course | −0.04 | 0.15 | −0.13 | 0.04 | ||||||||
| × white (vs. URM) | (0.16) | (0.30) | (0.25) | (0.32) | ||||||||
| Intervention course × income | ||||||||||||
| Intervention course | −0.07 | −0.15 | −0.10 | 0.09 | ||||||||
| × EOP status | (0.13) | (0.25) | (0.21) | (0.26) | ||||||||
| Covariates | ||||||||||||
| Race | ||||||||||||
| Asian (vs. URM) | −0.03 | −0.02 | −0.03 | −0.02 | −0.04 | −0.04 | 0.01 | 0.00 | 0.00 | −0.13 | −0.10 | −0.12 |
| (0.09) | (0.12) | (0.09) | (0.16) | (0.22) | (0.16) | (0.13) | (0.19) | (0.14) | (0.17) | (0.24) | (0.17) | |
| White (vs. URM) | −0.03 | 0.00 | −0.03 | −0.06 | −0.16 | −0.08 | 0.02 | 0.10 | 0.01 | −0.06 | −0.08 | −0.05 |
| (0.09) | (0.12) | (0.09) | (0.16) | (0.23) | (0.16) | (0.14) | (0.19) | (0.14) | (0.17) | (0.24) | (0.17) | |
| Income | ||||||||||||
| EOP status | −0.05 | −0.05 | −0.01 | −0.08 | −0.09 | 0.01 | −0.03 | −0.03 | 0.03 | 0.02 | 0.02 | −0.04 |
| (0.08) | (0.08) | (0.10) | (0.15) | (0.15) | (0.19) | (0.12) | (0.12) | (0.16) | (0.15) | (0.15) | (0.20) | |
| Gender | ||||||||||||
| Female | −0.06 | −0.05 | −0.05 | 0.14* | 0.14* | 0.15* | −0.08 | −0.07 | −0.07 | −0.14* | −0.14** | −0.14** |
| (0.07) | (0.07) | (0.07) | (0.13) | (0.13) | (0.13) | (0.11) | (0.11) | (0.11) | (0.13) | (0.13) | (0.13) | |
| N | 317 | 317 | 317 | 317 | 317 | 317 | 317 | 317 | 317 | 317 | 317 | 317 |
Interestingly, interaction terms including URM or EOP variables and course section (Table 5) were insignificant, suggesting that for URM and EOP students the intervention course is not more effective at cultivating belonging when compared with their non-URM and non-EOP peers in that same section, respectively. The lone exception was the interaction of the intervention course with gender when predicting students’ perceptions of peer support (Supplemental Table 6). The main effects model showed that, although the intervention course did not significantly affect students’ perceptions of peer support, female students had significantly higher perceptions of support than males. The interaction model reveals that this is likely because the intervention course raised perceptions of peer support significantly more for females than it did for males (Supplemental Table 6B). Taken collectively, these results suggest that the intervention course impacts males’ sense of belonging by improving perceptions of faculty support and classroom comfort, but for females it also increases perceptions of peer support.
Students’ overall sense of belonging was positively predicted by cumulative science GPA at the end of Spring quarter (Supplemental Table 6A). Analyses on subscales of belonging suggest that this association is most strongly associated with perceptions of classroom comfort (e.g., comfort sharing ideas and asking questions in class), rather than higher perceptions of support from peers or faculty (Supplemental Table 6, B–D). Although cumulative science GPA was significantly associated with students’ sense of belonging, their prior academic ability, as measured by total SAT scores, was not significantly associated with belonging (Supplemental Table 6, A–D).
Short-Term Student Retention Is Mediated by Introductory Biology I Grade
Tracking of individuals enrolled in either section of Introductory Biology I revealed that those participating in the intervention course were more likely to proceed into Introductory Biology II in the subsequent quarter (intervention retention ∼95% vs. traditional retention ∼82%). Multilevel logistic regression analysis with cohort year as a random intercept did not produce a significantly different model with a lower AIC than logistic regression, suggesting cohort year was not a significant explanatory variable for student retention into Introductory Biology II (Supplemental Table 7 and Supplemental Figure 3). Using the same variables as our regressions, we performed mediation analysis to determine whether grade was a significant mediator of the relationship between course taken (intervention or traditional) and student retention into Introductory Biology II (see Supplemental Figure 4 for covariates, effect sizes, and R code). Both of the estimated average causal mediation effects (ACMEs) are statistically significant, as seen in Table 6. However, the estimated average direct effect and total effects were not. This indicates that enrollment in the intervention course may have increased student grade, which in turn increased the likelihood of a student being retained and enrolling into Introductory Biology II.
| Coefficientsb | Estimate | 95% CI lower | 95% CI upper | p |
|---|---|---|---|---|
| ACME (traditional course) | 0.01045 | 0.00630 | 0.02 | <2e-16 |
| ACME (intervention course) | 0.01238 | 0.00513 | 0.02 | <2e-16 |
| ADE (traditional course) | −0.00752 | −0.03055 | 0.01 | 0.47 |
| ADE (intervention course) | −0.00558 | −0.02065 | 0.01 | 0.41 |
| Total effect | 0.00486 | −0.01175 | 0.02 | 0.57 |
| Proportion mediated (traditional course) | 2.14884 | −15.01883 | 16.93 | 0.57 |
| Proportion mediated (intervention course) | 2.54686 | −22.29710 | 24.38 | 0.57 |
| ACME (average) | 0.01141 | 0.00690 | 0.02 | <2e-16 |
| ADE (average) | −0.00655 | −0.02557 | 0.01 | 0.45 |
| Proportion mediated (average) | 2.34785 | −18.95768 | 20.14 | 0.57 |
DISCUSSION
This study contributes further evidence that carefully designed intervention courses promote improved undergraduate academic performance in biology across multiple institutions and instructors. It contributes to a growing body of research by 1) illustrating that participants benefit academically from this pedagogical approach; 2) providing some evidence that students engaged in active-learning techniques coupled with PLTL may build greater perceptions of belonging at the classroom level; and 3) documenting that this approach, through improved grade performance, promotes greater short-term student retention.
Course Structure and PLTL Promote Student Academic Success
We sought to replicate previous DBER research that employed increased course structure (Haak et al., 2011; Eddy and Hogan, 2014; Gavassa et al., 2019) and to assess whether our course design would promote academic improvement among a diverse undergraduate population. Combining preclass assignments, in-class active learning, and small-group PLTL in place of one lecture per week significantly improved the academic performance of participating students (Figure 2 and Supplemental Figure 1). However, our regression analyses do not suggest that, in our context, historically disadvantaged students benefited disproportionately by our intervention design (Tables 4 and 5). Employing common exam questions, we confirmed that the students participating in the intervention course academically outperformed those in the traditional lecture section by a 12% increase in correct response rates (Figure 2 and Supplemental Figure 1). These results confirm that, in our context, an intervention course that employs active-learning and PLTL broadly improves academic performance among students, while confirming that these approaches work in a new educational setting (Freeman et al., 2007a, 2014; Haak et al., 2011; Eddy and Hogan, 2014).
Participation in the Intervention Course Correlates with Increased Student Belonging
When compared with their peers, URM and EOP students possess a lower sense of belonging to the university community that, in turn, is linked to decreased student retention (Zea et al., 1997; Just, 1999; Hofman and Van Den Berg, 2003; Swail et al., 2003). This loss of diversity is echoed at UCSB, where ∼30–40% of the incoming EOP biology students will complete their degrees in the major; however, we do not know whether a lower sense of belonging, grades, or some combination of academic and social experiences are the cause for students leaving the major (Figure 1; similar trend observed for URM students). Given that both formal and informal interactions among and between students and their instructors can influence their perceptions of belonging (Umbach and Wawrzynski, 2005; Meeuwisse et al., 2010), we designed our intervention course to promote these interactions extensively both inside and outside scheduled class times.
We hypothesized that these approaches may increase student perceptions of belonging to a course. To assess whether this hypothesis is worth pursuing in more depth, we surveyed biology students enrolled in both sections at the end of the course. Our results suggest that enrollees in the intervention course had a significantly higher sense of student belonging; however, we did not observe a disproportionate increase in sense of belonging for URM or EOP students relative to their continuing-generation non-URM or non-EOP peers (Table 5). Importantly, breaking down sense of belonging and separately analyzing perceptions of peer support, perceptions of faculty support, and classroom comfort gives us insight into what may have made the biggest difference. After controlling for demographic differences between the sections, we saw that the intervention course was most strongly related to students’ perceptions of faculty support and classroom comfort. Conversely, students in the traditional section perceived much weaker faculty support and classroom comfort. In these areas, where there clearly was room for improvement, the intervention course succeeded. Research on the antecedents of sense of belonging have determined that it can be promoted by encouraging participation (Freeman et al., 2007b) and by being more responsive and adaptive in the classroom (Anderman, 2003). When comparing the features of the two sections in this study, we hypothesized this may happen due to the lower teacher–student ratio and the greater provision for discussion during lecture periods. This finding suggests that the increase in student–instructor interactions arising during the collaborative problem-solving portion of the active-learning classroom, in which there are more casual interactions with students, may have fostered stronger instructor–student relationships and a more comfortable classroom climate (Ballen et al., 2017).
Peer support is a critical component of belonging. Students in the traditional section of Introductory Biology I reported that they had relatively strong peer relationships; however, unlike faculty support and classroom comfort, the distribution of peer support skewed toward the maximum (Table 5). Therefore, with little room for improvement in students’ perceptions of peer relationships in the traditional lecture section, we should not have expected that the intervention course would significantly increase this aspect of students’ sense of belonging. Subscale analyses of sense of belonging, including interaction terms of gender and section, revealed that females in the intervention course perceived stronger peer support than those enrolled in the traditional section (Supplemental Table 6). It is possible that the significant student–student interactions implemented in the intervention course, especially in the 50-minute PLTL discussion sections, led to this increase in peer support. This would echo recent evidence that establishment of near peer–mentored learning communities promotes greater social integration of female participants (Solanki et al., 2019). The promising initial results that the intervention course promotes a greater sense of belonging through the improvement of multiple subscales needs to be examined further via replication with both a precourse and postcourse analysis of belonging to ensure comparable initial student perceptions.
Short-Term Student Retention
Small-group discussion sections that promote informal interactions among students have been demonstrated to improve URM student retention to graduation (Kudish et al., 2016). Additional research highlights that cooperative active-learning environments help students integrate (Braxton et al., 2000; Prince, 2004) and improve student perceptions of belonging (Umbach and Wawrzynski, 2005). Yet incorporation of active-learning strategies alone is insufficient to promote URM perceptions of social belonging (Ballen et al., 2017). Our approach of combining small-group discussions and in-class and external learning activities significantly increased perceptions of social belonging at the classroom level while significantly increasing student academic performance. However, our analyses did not demonstrate a disproportionate benefit to EOP or URM participants in the intervention course when compared with the traditional section (Table 5).
We sought to assess whether the intervention course influenced student retention into the subsequent introductory biology course. The results of the mediation analysis were consistent with the hypothesis that enrollment in the intervention course helped to improve final grades, which in turn positively impacted student retention into the subsequent course (Tables 4 and 6). However, we note the limitation that traditional mediation analysis requires that subjects be randomly assigned to treatment and control groups. We were unable to randomly assign students to sections but have found the two sections to be demographically and academically comparable (except for total SAT; Supplemental Table 3). It will be important to assess the longer-term impacts of participation in these types of intervention courses, beyond promoting academic performance and short-term retention, to add to the growing body of literature in which student social belonging and integration into academic and social communities at universities are linked to increased student retention (Hurtado and Carter, 1997; Tinto, 1997; Zepke et al., 2006; Tinto, 2010).
Analyzing the impact of the two sections of Introductory Biology I on different groups of students, we found that EOP and URM students did not disproportionately benefit academically or in their perceptions of belonging in the intervention section (Tables 4 and 5). This result is perplexing, given the growing body of biology DBER literature demonstrating that active-learning practices and peer-led learning communities, two components of the intervention course, are associated with disproportionate increases in academic achievement and social–psychological metrics (Freeman et al., 2007a; Haak et al., 2011; Eddy and Hogan, 2014; Ballen et al., 2017; Gavassa et al., 2019; Solanki et al., 2019). A possible source of this observation could be that Introductory Biology I, a second-year course, requires the completion of a full year of prerequisites during a student’s first year. Therefore, students may complete Introductory Biology I by employing established academic routines and mindsets stemming from their first year of study that the active-learning methodologies in the intervention are not capable of circumventing. Another possible explanation could be the unique student composition of the biology major at UCSB, with large URM and first-generation college populations, differs significantly from the campuses of previously published studies. Therefore, it will be important to compare our presented results with those gathered using the same active-learning and peer-led pedagogical approaches at demographically similar, selective public institutions.
Limitations
The purpose of this research was to evaluate the impacts of course design on student academic performance and retention while exploring whether the intervention course promotes greater perceptions of student belonging. We underscore that, at this point, it remains unclear whether the observed academic improvements were the result of the intervention course design alone, the particular instructors alone, or some combination of these factors. Further, it is also important to acknowledge that we did not identify the extent to which various elements of the intervention course promoted student academic achievement. However, because we identified faculty support as a key driver for the significant increase in reported belonging by students in the intervention course, and research has previously shown that faculty–student relationships promote academic achievement (Kommaraju et al., 2010), we must limit our attribution of observed improved academic performance as a result of course structure, as this improvement could greatly rely on the instructor differences between courses. Further, the significant difference in course enrollment, ∼700 of various majors in the traditional section and ∼280 biology majors in the intervention course, could influence both student comfort in the classroom itself and establishment of relationships with course instructors.
In addition to differences in faculty and course structure, the distribution of course points assigned by the two sections could also be confounding our analysis. Because the intervention course enabled students to earn 10% of their final course grade from in-class participation, graded tutorial worksheets, and review quizzes, while only a bonus 2.5% of the traditional section final grade was assigned by clickers or quizzes, the noncognitive impact of course structure may also have influenced student performance. Therefore, final grade comparisons between sections must be viewed under the lens that the final grade calculations are not identical. Although the use of shared exam questions illustrates that students in the intervention section may be outperforming their traditional section peers on common material, due to our inability to link demographic data to exam performance data, we are limited in our ability to compare exam performance of student populations between the two course sections.
Given that the belonging survey assessment was not implemented in a pre/post manner, we cannot rule out that our results may be due to initial variation in student populations. Although the two sections were scheduled for the same class times over the 3 years of study, students were able to self-select into each section. Therefore, it is possible that students became aware of course differences and preferentially enrolled in one section versus another. This could lead to variation in other noncognitive metrics, like motivation or interpersonal interactions before entering this second-year course.
Conclusions and Future Perspectives
We conclude that intervention courses should be designed to incorporate multiple approaches of instruction that target student comprehension of core learning objectives and also foster establishment of a classroom community. Although this study underscores the effectiveness of this approach at a key pivotal moment experienced by biology majors at UCSB, there remain gaps in the greater DBER field of assessing the scalability and long-term impacts of these course experiences. First, can this approach be scaled effectively to class sizes greater than 300 and maintain student perceptions of belonging? Or, will this increase in class size diminish student–faculty comfort that we identified in this study? Conducting research characterizing student–student and instructor–student experiences and tracking retention in high-enrollment courses would address this. Second, does participation in an intervention course have long-lasting impacts on student academic success and retention? It will be important as a field to illustrate whether participation in these courses leads to improved student performance or retention as individuals progress through their majors.
ACKNOWLEDGMENTS
This program was supported in part by a grant to UCSB from the Howard Hughes Medical Institute through the Science Education Program (grant number 52008111). We thank Sarah Eddy for critically reading the article and the reviewers for their helpful suggestions.
