Adding Authenticity to Inquiry in a First-Year, Research-Based, Biology Laboratory Course

INTRODUCTION

The best way to learn science is to do science. Yet science traditionally has been taught through prescriptive, “cookbook” laboratory exercises that do not expose students to scientific discovery. To improve student learning and persistence in science, many universities and colleges have replaced traditional laboratory courses with course-based undergraduate research experiences (CUREs). CUREs are a student-centered educational approach in which students conduct scientific research in the classroom. Auchincloss et al. (2014) defined CUREs as having all of the following five elements: 1) scientific practices, such as asking questions, proposing hypotheses, designing studies, collecting and analyzing data, and communicating results; 2) discovery of new knowledge; 3) relevant or meaningful work; 4) collaboration; and 5) iteration. CUREs involve students in many aspects of authentic research, because the research may be unscripted, the outcomes may be unknown, and the research can be highly relevant to the scientific community. Every genuinely authentic research experience entails involvement in the process of science and the creation of new knowledge (Table 1; Spell et al., 2014; Rowland et al., 2016).

Undergraduate laboratory course experiences lie along a continuum from prescribed laboratory exercises with a low level of inquiry to authentic CURES that give students much freedom (Table 1). Along this continuum are different levels of inquiry-based research experiences that involve students in the research process with increasing levels of independence and responsibility (National Research Council [NRC], 2000; Brown et al., 2006; Table 1). In inquiry-based research, an instructor assigns the problem or question (Buck et al., 2008; Weaver et al., 2008) but gives students some independence to design and conduct their own investigations. In inquiry-based labs, results usually are not known to the students, but often are known to the instructor and scientific community (Dolan, 2016). As students are given increased freedom to ask their own research questions and devise their own investigations, the experience becomes increasingly authentic, with students engaging in the discovery process as scientists. The more freedom students have to make decisions about their projects, the more authentic the experience is. In CUREs, students conduct original research in that neither students nor instructors know the outcomes, which are relevant to current research within the scientific community. Furthermore, students may be provided the opportunity to use sophisticated lab equipment for their data collection (Weaver et al., 2008), resulting in potentially publishable data. CUREs often culminate in public presentations of their findings (Weaver et al., 2008; Table 1).

Because they are embedded within courses, CUREs can accommodate a greater number and more diverse group of students than traditional mentor–apprenticeship undergraduate research experiences (UREs), which often are limited by funding, space, and time (Desai et al., 2008; Wei and Wooden, 2011; President’s Council of Advisors on Science and Technology [PCAST], 2012; Bangera and Brownell, 2014). Typically, students selected for traditional UREs are high-achieving or upper-division science, technology, engineering, and mathematics (STEM) majors already motivated to succeed in their chosen field (Lopatto, 2007; Linn et al., 2015). In contrast, CUREs can engage first- and second-year undergraduates of different abilities and backgrounds who may not yet have decided on a major or career pathway and are therefore vulnerable to not pursuing a STEM major, even if they are interested in STEM (Seymour and Hewitt, 2000). Hence, the broad inclusivity of CUREs may increase the number and diversity of students pursuing STEM degrees (Nagda et al., 1998; Lopatto, 2004; Russell et al., 2007; Bascom-Slack et al., 2012; Hanauer et al., 2012; Eagan et al., 2013; Graham et al., 2013).

CUREs provide many of the same benefits as traditional mentor–apprenticeship UREs (Lopatto, 2007; Shaffer et al., 2010, 2014; Wei and Woodin, 2011; Auchincloss et al., 2014; Table 1). Both experiences provide students a hands-on opportunity to participate in the discovery of novel information through participation in all scientific practices, such as reading and evaluating the scientific literature, selecting appropriate methods and experimental designs, collecting data, analyzing results, and working collaboratively (Corwin et al., 2015). These activities have been shown to improve students’ understanding of science and to increase their ability and confidence to conduct research, motivating them to learn science and self-identify as a scientist (Hunter et al., 2007; Thiry and Laursen, 2009; Adedokun et al., 2013; Corwin et al., 2015). Thus, CUREs, like UREs, influence students’ decisions to remain in STEM majors and, ultimately, to pursue STEM careers (Lopatto, 2004, 2007; Hunter et al., 2007; Harrison et al., 2011; Adedokun et al., 2012; Graham et al., 2013; Auchincloss et al., 2014; Corwin et al., 2015; Linn et al., 2015).

CUREs have become a common way to provide authentic research experiences for students. For example, the multi-institution Science Education Alliance—Phage Hunters Advancing Genomics and Evolutionary Science (SEA-PHAGES) program engages undergraduates in authentic research (Hanauer et al., 2017) through discovery of novel bacteriophages from environmental samples (Jordan et al., 2014). The multi-institutional Genomics Education Partnership (GEP) is a CURE integrated into genetics courses, in which students do authentic research in the field of genome evolution by improving Drosophila DNA sequence data and annotating chromosomes (Lopatto et al., 2008; Shaffer et al., 2010). While GEP and SEA-PHAGES are one and two semesters, respectively, the Freshman Research Initiative (FRI) at the University of Texas, Austin, is a three-­semester research course series. FRI students start out by learning general research methods in the first semester. Following this, they move into a selected research stream. In the first research-stream course, they learn methods pertinent to their research topics and participate in guided inquiry-based projects. In the following course, students conduct independent authentic research and may have the freedom to ask their own research questions and design their own projects (Rodenbusch et al., 2016; National Academies of Sciences, Engineering, and Medicine, 2017).

Although it is widely agreed that CUREs positively influence student learning and persistence in the sciences, many reports of CUREs have been descriptive (National Academies of Sciences, Engineering, and Medicine, 2017) with a lack of outcome data. Most studies have assessed small numbers of students, and few have had matched control groups, limiting conclusions about outcomes (Brownell et al., 2012, 2013; Dolan, 2016; Rodenbusch et al., 2016; National Academies of Sciences, Engineering, and Medicine, 2017). Evaluations of CUREs should control for students’ backgrounds and academic preparation (Theobald and Freeman, 2014). Moreover, because student self-reporting does not always correlate well with actual learning (Falchikov and Boud, 1989; Shortlidge and Brownell, 2016), evaluation of CURE learning gains should be supported with objective measures of academic achievement (Linn et al., 2015; National Academies of Sciences, Engineering, and Medicine, 2015). An exemplary study of FRI program participants conducted by Rodenbusch and colleagues (2016) has met these more rigorous requirements.

More than a decade ago, we developed and implemented a CURE in an introductory general biology lab course for first-year students at the University of Miami (UM). The UM Authentic Research Laboratories (the acronym “UMARL” reflects South Florida’s prevalent limestone) courses imported faculty and their research into the classroom, where students conducted investigations of their own design within the realm of the faculty members’ research topics. Students were given freedom to ask their own novel questions, to form hypotheses, and then to devise research that would generate data to test their hypotheses. Their results had the potential to contribute to the faculty instructors’ research programs. Student-generated research questions and project designs conferred an authentic research experience.

The goals of this report are threefold. First, we describe the design and pedagogy of our UMARL course. Second, we compare student self-reported learning gains with national data. Third, using demographic and academic data of students who took UMARL over a span of 10 years and a matched comparison group from UM’s traditional introductory biology labs, we evaluate outcomes, including participation in individual research, retention in STEM majors, time to graduation, and graduation with honors. We hypothesized that UMARL courses would have a positive influence on students’ learning gains, because the unscripted research of UMARL engaged students in many of the same activities as traditional UREs, interesting them in subsequent research experiences, retaining them in STEM, and improving academic achievement (Tai et al., 2006; Hansen and Birol, 2014; Olimpo et al., 2016). We hypothesized that students in UMARL would report greater learning gains in research skills compared with all students responding to a national CURE survey, which potentially included respondents experiencing different authenticity. We also specifically predicted that UMARL students would be more likely to participate in individual research, graduate with a STEM degree, graduate with honors, and graduate within 4 years than a matched group of UM’s traditional biology lab students.

METHODS

Course Design and Implementation

The goals of the authentic research UMARL course were to 1) lead students to the realization that the outcomes of research are not known in advance, hypotheses are not “proven,” and things do not always go as planned; 2) expose students to what scientists do and how they think and to research as a career; and 3) engage research faculty in teaching first-year undergraduates, potentially fostering subsequent mentoring of undergraduates’ individual research. The participation of research faculty also provided first-year undergraduates with an opportunity to interact individually with faculty, which is difficult in the large lecture setting of many introductory science courses and rarely happens in introductory labs, typically taught by graduate students.

In UMARL, four groups of five or six students devised and carried out their own investigations based on a faculty instructor’s general area of research. Two sections, each with 20–24 students, ran concurrently, meeting for 3 hours per week. Halfway through the semester, faculty and their teaching teams interchanged their entire sections of students, such that students switched research themes. In each new research theme, students developed new novel questions, hypotheses, and experiments. After this midsemester switch, students stayed in their same groups, although they were free to form new research groups within their new lab sections in the following Spring semester. Thus, during the two-course, 1-year lab sequence, students were exposed to four different faculty members and their research areas. These authentic research labs were taken in place of the traditional two-semester introductory biology lab course sequence (i.e., students in UMARL earned introductory biology lab course credits).

Teaching Teams.

Three-person teams consisting of a faculty member, a graduate teaching assistant, and an undergraduate peer facilitator (either paid a small stipend or awarded credit) led each section of four student groups. All three members of the teaching team were present throughout every lab session. The primary role of the faculty member was to bring his or her research program into the classroom and to provide guidance for a broad range of possible student investigations in a nonprescriptive manner. Faculty were funded investigators or otherwise engaged in research. The graduate teaching assistant helped student groups design, conduct, and report their research (through PowerPoint presentations and scientific poster preparation) and assisted with grading. The undergraduate peer facilitator usually was an alumnus of UMARL who preferably had taken the lab from the same faculty instructor. The graduate teaching assistant and undergraduate peer facilitator fostered interactions among student groups, modeled collaboration with faculty, and underscored that the labs are student-centered, all of which can enhance student learning and academic performance (Otero et al., 2006).

Research Themes.

UMARL students were exposed to a diversity of research themes. Over 10 years, we had more than 20 different research themes led by as many different faculty, mostly from the biology department but also including faculty from UM’s marine and medical schools (Table 2). Themes involved different numbers of independent and response variables, and those with a high product of independent and response variables offered students the most opportunity for creativity and control over the design of their investigations. For example, investigations of a glycoprotein called “glomalin,” which is produced by ubiquitous arbuscular mycorrhizal fungi, involved measuring just a single dependent variable, the amount of protein in extracts of soil. Nevertheless, because glomalin can be found in almost every soil, students could test many different hypotheses regarding factors affecting its abundance. Another research theme was the health of soft corals (maintained in aquaria) assessed by three response variables (numbers of photosynthetic endosymbionts, chlorophyll concentration, and protein concentration of small tissue samples) after exposure to an environmental stressor (warming, petrochemicals, fertilizer runoff, over-the-counter pharmaceuticals) limited primarily by students’ imagination. Additional examples of research themes and suggestions for their implementation are available in Supplemental Table S1.

Course Progression and Deliverables.

Each UMARL section followed the same general pattern and involved the same formative assessments (or deliverables). During the first class meeting, students were provided background information in a faculty lecture and were trained hands-on in analytical methods useful for investigating questions related to the research theme. The faculty instructor presented general examples of questions that student groups might pose but avoided being at all prescriptive. Each group of students then formulated their own research question, stated relevant hypotheses, and devised a protocol for testing them which the group presented orally at the start of the second lab. Additionally, students conducted a literature search for specific background information and potential methodologies, thus becoming acquainted with bibliographic search tools and interpreting scientific literature. Not only did student groups conduct investigations of their own devising, but they also had to report their work—an integral part of the scientific process. During the semester, students gave brief oral PowerPoint presentations to their lab sections summarizing their groups’ progress, challenges, accomplishments, and next steps. In the early weeks, the presentations focused on refinement of the hypothesis being tested, as well as continuing incorporation of background information drawn from pertinent journal articles. Intermediate weeks often dealt with data analysis and statistics. The final week’s oral presentations summarized results and conclusions. The ensuing discussions helped groups with formulation of the final research paper and scientific poster.

Every student was expected to participate in all reporting activities, which were consistent with what practicing scientists do professionally. Students were assessed on these deliverables for each half-semester project. Emphasis was placed on collaboration, reportage, and constructive criticism. Each student not only was required to give an oral presentation, but also was expected to participate in discussion of the presentations. Getting first-year students to criticize their peers is difficult, no matter how constructive the criticism. Consequently, the teaching team had to establish an ethos of “we’re all in this together, and everyone can benefit from receiving and giving constructive criticism.” In doing so, the undergraduate peer facilitator was especially helpful, because he or she was the least threatening and most relatable member of the teaching team. Each group of students was responsible for preparing a poster on their research project for display in a high-traffic area outside the laboratory classrooms at the end of the semester during a public scientific poster session held the week of final exams.

Students’ lab grades were not dependent on the results of their research. Both oral and poster presentation activities were graded as “pass/fail.” Avoiding letter grades minimized speakers’ anxiety and individual competition among group members. For final, course letter grades, each student had to keep a laboratory notebook, write a one-page background summary of a relevant publication found through a literature search, and report their research in a standard (i.e., Introduction, Methods, Results, Discussion) scientific paper limited to two pages of text plus any needed additional pages of literature citations, tables, and figures with appropriate legends. A draft of the final research paper was thoroughly critiqued by the faculty instructor before a final revision was submitted for grading. Before the first paper draft was due, it was especially helpful to distribute a “model” paper conforming to the required format, written by the instructor. Other graded assessments, such as closed-book quizzes, take-home calculations, and problem sets, were included at the discretion of instructors. An example syllabus is available in Supplemental Table S2.

Laboratory Classrooms.

The construction of two specially designed laboratory classrooms contributed to the success of UMARL. The rooms feature four custom-built, desk-height, octagonal worktables for student research groups, comfortable chairs on casters, full Internet connectivity, two laptop computers per table, a permanently mounted computer projector, and both standard and electronic whiteboards. The always-available computer projector facilitated the required student presentations, and at-hand laptops and the electronic whiteboard encouraged an emphasis on quantitative methods. The worktables helped to facilitate the collaborative group-work nature of the research projects, which would have been more difficult in traditional teaching lab classrooms with stools along the sides of stand-up-height, long, rectangular workbenches that impeded students facing and conversing with their group.

Course Enrollment

The majority of students who took UMARL were first-year undergraduates with majors in biology, chemistry, neuroscience, biochemistry, and other STEM disciplines. UMARL students’ non–STEM majors included classics, business, political science, and psychology. Participant selection for these labs changed since the first implementation of the course in 2004. Originally, incoming freshmen majoring in biology or marine biology were rank-ordered by Scholastic Aptitude Test (SAT) or ACT scores, and every second student (effectively a stratified random group) was sent a letter of invitation to participate in UMARL. Because students had to have enough proficiency in mathematics to be able to do the graphical and statistical analyses required, only students with an SAT quantitative score at or above 550 or an ACT score of 26 or greater were invited. For the third year of implementation, we expanded the invitation list to include students on a premedical or preveterinary track, but not majoring in biology or marine biology. Responses to invitations were positive. For example, for the Fall 2007 semester, of 189 incoming freshmen who were sent invitations, 60% responded, with 111 (98.2% of respondents) accepting and only two declining. Before registration for the second semester, we offered students enrolled in the first-semester lab the opportunity to continue, and then opened enrollment as space allowed. Typically, students chose in which section to enroll based on their course schedule, and their majors were not considered in their assignments to lab sections and, therefore, research themes. Although the instructors for each section were known during enrollment, the research themes to be covered in a particular section where not explicitly stated in each semester’s course listings. A substantial majority (137/213 = 64%) of first-semester invitees over the first 3 years (Fall 2004 through Spring 2007) of UMARL implementation chose to continue for the second semester. Although we did not collect data on attrition, anecdotal evidence suggests the other 36% did not continue because they had scheduling conflicts or no requirement for a second semester of introductory biology lab in their program (such as the biomedical engineering major) or because some students did not like the uncertainty of real, authentic research.

In Fall 2008, UMARL was incorporated into a new program, the Advanced Program for Integrated Science and Mathematics (PRISM). The program’s curriculum takes an integrative, interdisciplinary approach to teaching the basic sciences. PRISM students are block-scheduled and complete their introductory science coursework together during the first 2 years of undergraduate study. To qualify for PRISM, students are required to have a minimum SAT quantitative score of 720 (or ACT equivalent). Although most PRISM students are life science majors, that is not a requirement for admission to the program. PRISM students filled UMARL sections, with a few other students enrolled as space allowed.

Evaluation of UMARL

Student Learning Gains.

To assess student engagement, we used the Classroom Undergraduate Research Experiences (CURE) survey (Lopatto, 2010) as an indirect measure of course learning gains. This survey is a helpful tool to evaluate the impact of CUREs on students, even though the survey is not formally validated (Hanauer and Dolan, 2014). It enables a comparison to other lab courses via national averages of CURE survey responses. At the end of each semester from Fall 2008 to Fall 2014 (inadvertently omitting Spring 2009), we invited UMARL students to participate in the CURE postcourse national survey administered online by David Lopatto and Leslie Jaworski from Grinnell College (www.grinnell.edu/academics/areas/psychology/assessments/cure-survey). The survey measures student self-perceived learning gains in course elements related to research skills, learning benefits, and attitudes toward science. Students rated their learning gains in course elements on a Likert scale (1 = no gain, through 5 = very large gain). Lopatto and Jaworski tabulated the mean response scores of our UMARL students and students nationally (national means included our students’ responses), which they provided in a report each semester. Students in the comparison group participated in lab courses with different kinds of research experiences (i.e., structured inquiry, guided inquiry, open inquiry, or authentic research). The comparison group data were pooled from many institutions, the identities of which were not included in the reports provided by Lopatto and Jaworski.

Academic and Demographic Data for Summative Assessment.

To assess objective outcomes of UMARL, we obtained from UM’s Planning, Institutional Research, and Assessment Office the academic and demographic data for students who took traditional introductory biology labs (control group) and UMARL students (treatment group) from Fall 2004 through Spring 2014.

We requested the following data for each student: gender, race/ethnicity, SAT and/or ACT scores (including SAT-equivalent ACT scores), high school grade point average (GPA), term of matriculation, major at matriculation, introductory biology labs taken, terms when those labs were taken, participation in PRISM, research credits earned subsequent to introductory biology labs, term of graduation or last term enrolled, latest UM GPA, undergraduate degree earned, major at graduation or latest major, and graduation with honors.

After omitting students who might misrepresent either group (e.g., 1878 students who matriculated after Fall 2010 who did not have the opportunity to graduate in 4 years; Supplemental Table S3), our final sample included 3297 nontransfer, undergraduate degree-seeking students (2756 from traditional labs and 541 from UMARL), who matriculated between 2004 and 2010. Students who had taken at least one semester of UMARL were included with the treatment group, because they received the intervention (even if only for one semester).

Propensity Score Matching.

We conducted propensity score matching to identify an unbiased, control group of students from UM’s traditional introductory biology laboratory courses. These traditional labs were prescriptive, with students conducting different, stand-alone laboratory exercises each week. For each lab, students worked in groups of four to follow step-by-step instructions and answer questions given in the lab manual about their observations. The first-semester traditional biology lab covered topics such as the structure and division of cells (examination of slides under a light microscope), photosynthesis (extraction of chlorophyll from plants and analysis of light absorption properties), and developmental biology (observation of sea urchin embryo cleavage under a light microscope). The second semester of the traditional biology lab focused on biodiversity. In this lab, students examined slides under a light microscope, observed living and preserved specimens, and performed dissections to learn about various taxonomic groups.

Propensity scores correct for course selection bias and reflect the probability that a traditional lab student might have taken UMARL based upon a set of covariate descriptors of the actual UMARL participants (Rosenbaum and Rubin, 1983, 1984; West et al., 2008). Propensity score matching resulted in a traditional lab control group with a similar mean SAT score to the UMARL group, accounting for the high SAT scores of PRISM students who took UMARL courses. We matched students using SAT score and STEM major at matriculation (0 = non-STEM, 1 = STEM) as covariates. We initially intended to use high school GPA as a covariate, but did not because of uncertain equivalencies and missing data. Gender, race/ethnicity, and graduation status (0 = not yet graduated, 1 = graduated) were used in exact matching. Gender and race/ethnicity are closely associated with student choice to major in STEM (Ma, 2011; Wang, 2013). Graduation status was used to ensure similar numbers of students in both groups for assessing outcomes associated with graduation. Using the Propensity Score Matching program for SPSS v. 3.0.4 (Thoemmes, 2011), we matched groups with the nearest-neighbor logistic regression as the algorithm with a caliper of 0.2 SDs of the logit of the estimated propensity score. The propensity score matching procedure resulted in traditional lab and UMARL groups of similar sizes, demographic makeup, SAT scores, numbers of students matriculated as STEM majors, and graduation status (Supplemental Table S4). The difference between propensity scores of both groups was reduced from 0.039 before matching to 0.002 after matching, a 96% overall reduction (Supplemental Figure S1).

Statistical Analyses

Student Learning Gains.

We assessed student responses to all 21 learning gain items on the postcourse CURE survey taken by UMARL students and students nationally over 11 semesters (N = 280 and N = 38,160, respectively). We calculated Student’s t tests using weighted means and standard deviations to compare differences in responses for all survey items related to course benefits. Differences were considered significant at α = 0.05.

Outcomes.

We examined the effect of UMARL on outcome variables associated with student engagement in research, progress toward a STEM career, and academic achievement. We compared the propensity score matched groups using hierarchical logistic regressions, because our outcome data were binary (success or failure; students either enrolled in research or did not) and our observations of students fell into discrete categories, such as gender and ethnicity, in the traditional and UMARL groups. We then accounted for predictors that might influence results; for example, race is known to be associated with college GPA (Brown and Lee, 2005). Gender, race/ethnicity, and SAT score were known predictors included in the first step of the hierarchical models to account for effects correlated with outcomes. We checked for other potential predictors by examining correlations (Pearson’s r) with each outcome variable before running hierarchical logistic regressions, and if significant, those predictors were added to the first step of the model. After all predictors were accounted for in step 1 (Supplemental Table S5), the hierarchical logistic model then investigated the likelihood of students who participate in UMARL having “success” (coded as “1”) or not (coded as “0”) compared with students in the traditional labs. This likelihood was provided as the odds ratio, or the ratio of the “odds” (i.e., the success vs. failure) of the UMARL group compared with that of the traditional lab group. Model fit was assessed using the Hosmer and Lemeshow goodness-of-fit test (p > 0.05, because the null hypothesis is that the chosen model is deemed appropriate) at each step of the regression (Peng et al., 2002). The outcome variables we analyzed were whether students participated in subsequent individual research experiences (1) or not (0), graduated with a STEM major (1) or not (0), graduated in 4 years (1) or not (0), and graduated with honors (1) or not (0). We had planned to analyze undergraduate GPA at graduation (or latest GPA), but a generalized linear model was a poor fit for the data, so we did not proceed further.

Subsequent Individual Research Experiences.

We used enrollment in independent research courses as a proxy for participation in subsequent individual research experiences. Although students also can obtain research experiences through lab internships, employment, and volunteering, that information is difficult to obtain. Research courses at UM allow students to earn academic credit for conducting independent research projects in faculty members’ laboratories. We compared acquisition of research credits between the two groups of students by coding this variable (1 = with research credits, 0 = no research credits). In addition to gender, race/ethnicity, and SAT score, this logistic regression model included whether students matriculated as STEM majors in the first step of the regression (1 = STEM, 0 = non-STEM), because this variable was significantly correlated with whether a student acquired research credits or not (Pearson’s r = 0.143, p < 0.001).

Major at Graduation.

We looked at the likelihood that a student graduated with a STEM degree (1 = STEM, 0 = non-STEM) after participation in UMARL versus traditional biology labs. Whether a student was a STEM major at matriculation and the number of semesters (either one or two) a student participated in traditional introductory biology labs or UMARL were variables significantly correlated with STEM degree at graduation (STEM major at matriculation: Pearson’s r = 0.251, p ≤ 0.0001; number of semesters: Pearson’s r = 0.135, p ≤ 0.0001). Thus, we accounted for these predictors in the first step of the hierarchical logistic model. For this outcome, we considered only students who had graduated.

Time to Graduation.

We examined the likelihood that UMARL students graduated in 4 years or fewer versus traditional lab students (1 = 4 years or fewer, 0 = more than 4 years). The number of semesters of introductory biology lab taken (either one or two) and number of research credits acquired were significantly correlated with graduation in 4 years or fewer (number of semesters: Pearson’s r = 0.123, p ≤ 0.0001; number of research credits: Pearson’s r = 0.188, p ≤ 0.0001) and thus were included in the first step of the logistic regression. All propensity matched students, regardless of graduation status, were included in this analysis.

Graduation with Honors.

We investigated whether UMARL students were more likely than traditional lab students to receive honorable distinctions at graduation (1 = graduation with honors, 0 = regular graduation). Graduation with honors included students who graduated with general honors, departmental honors, and/or university honors (cum laude, summa cum laude, or magna cum laude). Students who graduated with general honors were required to complete honors courses and maintain a GPA of 3.5 or higher, and those with departmental honors, of which there were only three in our sample, also were required to have a minimum GPA of 3.5 and to conduct 6 credit hours or more of independent study, senior thesis, senior project, or advanced courses. University honors distinctions are based on GPA percentiles within each graduating class. In the regressions for this outcome, we used only students who had graduated. Not surprisingly, graduation with honors was significantly correlated with undergraduate GPA (Pearson’s r = 0.708, p ≤ 0.0001), so we consider this outcome an indicator of student achievement.

RESULTS

Student Learning Gains

Students who participated in UMARL reported significantly greater learning gains than students nationally for all 21 course benefit items (Figure 1 and Supplemental Table S6). The largest mean differences between UMARL students and students nationally were for elements related to communicating scientific research (effective oral presentation and science writing) followed by elements related to perseverance and self-efficacy (tolerance for obstacles, self-confidence, and readiness for research; Figure 1 and Supplemental Table S6). UMARL students reported their greatest learning gains overall, however, for items related to the scientific research process, such as learning lab techniques (mean rating_weighted = 4.21 ± 0.21 SD_weighted), ability to analyze data (4.19 ± 0.18), and skill in interpretation of results (4.07 ± 0.19), as well as items related to understanding the nature of science, such as understanding how scientists work on real problems (4.11 ± 0.2), understanding that scientific assertions require supporting evidence (4.08 ± 0.26), and understanding science (4.04 ± 0.27). Additionally, students indicated large learning gains in tolerance for obstacles (4.05 ± 0.28), skill in science writing (4.03 ± 0.24), and skill in how to give an effective oral presentation (4.01 ± 0.32). Although still significantly greater than students nationally, UMARL students reported moderate learning gains for items not directly addressed by the UMARL courses, including confidence in potential as a teacher (3.25 ± 0.29) and learning to work independently (3.43 ± 0.36).

Propensity Score Matching

Propensity score matching resulted in two similarly composed groups of students (N_traditional = 532, N_UMARL = 541). Overall, the two groups were 61.6% female and 38.4% male with an ethnicity profile of 15.2% Asian or Pacific Islander, 0.5% American Indian or Alaska Native, 8.3% Black, 16.25% Hispanic, 52% white, 1.65% two or more races, and 6.15% unknown or missing (Supplemental Table S4). The mean SAT score was 1316 for both groups, with 92.25% of students having declared a STEM major at matriculation and 82.75% having graduated (Supplemental Table S4). The UMARL group was 28.5% PRISM students, with no PRISM students in the traditional labs.

Subsequent Individual Research Experiences

Participation in UMARL had a significant association with the likelihood of having subsequent individual research experiences, as measured by enrollment in research course credits (β = 0.421, Wald χ² = 8.164; p = 0.004; Table 3 and Supplemental Table S5). After accounting for the significantly positive effects of STEM major at matriculation and SAT score, the odds of earning research course credit for UMARL students were 1.53 times greater than the odds for students from the traditional labs (Table 4). Overall, 38% of UMARL students and 29% of traditional lab students earned research credits (Figure 2A), and for both groups of students, the majority participated in one to three courses (Figure 2B).

Major at Graduation

UMARL was significantly associated with the likelihood that students graduated with a STEM major (β = 0.445, Wald χ² = 7.493; p = 0.006; Table 3 and Supplemental Table S5). The odds of UMARL students graduating with a STEM degree were 1.56 times greater than the odds for traditional lab students after accounting for ethnicity, SAT score, STEM major at matriculation, and the number of semesters of introductory biology lab taken (Table 4). Overall, 75.6% of UMARL and 69.2% of traditional lab students graduated with a STEM degree (Figure 3).

Of the variables included in step 1 of the logistic regression, STEM major at matriculation was the most highly correlated with graduation with a STEM degree. The odds of graduating with a STEM degree were 6.2 times greater for students who declared a STEM major at matriculation than the odds for students who declared a non-STEM major at matriculation (Table 4). Students who participated in two semesters of introductory biology lab had greater odds of graduating with a STEM degree than students who took only one semester (Table 4). SAT score also was significantly correlated with graduating with a STEM degree; high SAT scores were associated with STEM degree at graduation. Although both SAT score and STEM major at matriculation influenced graduating with a STEM major, they were not correlated with one another (Pearson’s r = 0.052, p = 0.088). Students of Asian or Pacific Islander ethnicity had greater odds of graduating with a STEM major compared with the odds for students of other ethnicities, regardless of lab type (traditional or UMARL; Table 4). This was the only effect of ethnicity we found in our analyses.

Time to Graduation

UMARL was significantly associated with time to graduation (β = 0.447, Wald χ² = 10.924; p = 0.001; Table 3 and Supplemental Table S4). The odds of graduating in 4 years or fewer for UMARL students were 1.56 times greater than the odds for traditional lab students after accounting for the effects of gender, SAT score, research credit courses, and number of semesters of introductory biology lab (Table 4). In the UMARL group, 81% of students graduated in 4 years or fewer, as did 70.3% of traditional lab students (Figure 3). Regardless of lab type, females had greater odds than males to graduate in 4 years or fewer (Table 4). SAT score was positively correlated with this outcome (Table 4). Students with research course credits had greater odds of graduating in 4 years or fewer than students without research course credits, and those who took two semesters of introductory biology lab also had greater odds of graduating in 4 years or fewer than those who only took one semester of lab (Table 4).

Graduation with Honors

UMARL was positively associated with graduating with honors, even after accounting for SAT score (β = 0.535, Wald χ² = 12.011; p = 0.001; Tables 3 and 4 and Supplemental Table S5). The odds for UMARL students to graduate with honors were 1.71 times greater than the odds for students who took the traditional labs (Table 4). Because graduation with honors was significantly associated with GPA, we inferred that UMARL students also likely had greater odds than traditional lab students of graduating with a high GPA. Overall, 10% more UMARL students graduated with honors than students who took traditional labs (Figure 3).

DISCUSSION

Our hypothesis that UMARL would have a positive influence on students’ learning gains compared with students nationally was supported by our results. Additionally, UMARL students had greater odds of participating in individual research, graduating with a STEM degree, graduating with honors, and graduating within 4 years than a matched group of UM’s traditional biology lab students, as we initially predicted. Our approach of exposing first-year students to authentic research in laboratory classrooms may have helped students to understand scientific research, provided an experience in which students can be scientists, and may have fostered students’ research and psychosocial skills.

Authenticity Is Important

Our CURE model embodies authentic research, which we propose is vital for the positive outcomes that we observed for UMARL students in comparison to UM’s traditional biology lab students. Authenticity was created through students’ freedom to ask their own, novel research questions, design their own experiments, and participate in discovery of knowledge new to science. These course attributes gave students responsibility for their own learning and a sense of personal investment that may have led to a high level of project ownership (Hanauer et al., 2012; Hanauer and Dolan, 2014). Thus, students developed self-efficacy, including confidence in their ability to conduct research and a sense of belonging to the scientific community, which may be key factors for student persistence in STEM (Estrada et al., 2011; Hernandez et al., 2013; Robnett et al., 2015).

UMARL required students to do what real scientists do, and so students’ self-reported learning gains reflected their attainments in scientific research skills and, especially, their understanding of how scientists work. Our course design involved collaboration and iteration (with each switch of research theme), which were highly effective in teaching students to communicate science orally and in writing. Our CURE survey results suggest that UMARL students learned research skills while conducting research, much as graduate students do (Sadler et al., 2010).

Improvements in student self-efficacy that we detected may be necessary for the positive benefits achieved by UREs (Sadler et al., 2010; Adedokun et al., 2013). UMARL students self-reported gains in excess of national means for both “tolerance for obstacles faced in the research process” and “self-confidence.” When their data were inconclusive or unexpected, UMARL students discovered their ability to persevere. Moreover, repetition of activities with each new research theme, a fundamental tenet of our approach, also likely contributed to increased self-efficacy, because, as the saying goes “Practice makes perfect.” Through four different research projects over the course of the academic year, students experienced the universal utility of the scientific research method and gained confidence in their ability to successfully conduct research.

Our analyses found significant benefits of UMARL, even with inclusion of students who took only one semester of the lab, despite the fact that some studies have found a correlation between the benefits of research experiences and time spent participating in research (Sadler et al., 2010; Rodenbusch et al., 2016). Regardless of their number of semesters of UMARL, approximately four UMARL students for every three traditional lab students had subsequent UREs, as has been determined qualitatively for other CUREs (Bascom-Slack et al., 2012; Hanauer et al., 2012; Harvey et al., 2014; Kowalski et al., 2016). Those studies reported that 75–92% of students had UREs, however, while only 38% of our UMARL students had them. Nevertheless, our analysis used research course credit as an objective measure of subsequent UREs, resulting in a highly conservative estimate.

Even though 92% of students in both of our propensity score matched groups declared STEM majors at matriculation, likely because of their precollege experiences (Tai et al., 2006; Crisp et al., 2009), UMARL still was positively associated with graduating with a STEM degree. Such an effect can be especially important in keeping minorities and females from leaving science (Nagda et al., 1998; Hurtado et al., 2009; Schultz et al., 2011; Bangera and Brownell, 2014; National Academies of Sciences, Engineering, and Medicine, 2015).

Our results accord with findings of Rodenbusch et al. (2016) that participation in all three FRI courses had a significant effect on the probability of graduating with a STEM major and graduating within 6 years. Students who participated in only one or two FRI courses, however, were not any more likely than non-FRI students to have those outcomes. In contrast, regardless of the number of semesters for which students participated in UMARL, we found that UMARL students had increased odds of graduating in 4 years or fewer versus the odds for traditional lab students. Rodenbusch et al. (2016) found no effect of FRI courses on GPA at graduation, but our analysis did indicate a positive effect of UMARL on academic achievement, as reflected by graduation with honors.

UMARL has the hallmarks of authentic research described by Brownell and colleagues (2012): development of student-generated research questions with unknown answers, a focus on one or two research questions during the duration of the course, implementation of experimental designs that are developed as part of the course, collaboration, and presentation of research results. Similarly, SEA-PHAGES, the GEP, and FRI bring authenticity to course-based research via these elements, but through a different course structure and implementation model. For example, SEA-PHAGES focuses on one research theme with instructors who have been trained in phage discovery and bioinformatics for comparative phage genome analyses, which may or may not be their areas of expertise. In contrast, FRI students conduct research under a single theme led by faculty upon whose research programs the themes are based, which is similar to UMARL. Nevertheless, UMARL students conduct research in four different research themes, whereas SEA-PHAGES, GEP, and FRI students typically conduct research under one research theme for an extended time. The differences among these CUREs illustrate that authenticity can be established through a variety of course models.

Implementation Considerations

Implementing any CURE is not without challenges. Lack of time for faculty to develop research projects is a major barrier to implementation of CUREs across different institution types (Kloser et al., 2011; Lopatto et al., 2014; Spell et al., 2014; Kowalski et al., 2016), but we circumvented this by basing course research on faculty instructors’ own research programs and by giving students freedom to develop their own projects. Other CUREs have been developed around faculty research, but most focus on one research area (e.g., Bascom-Slack et al., 2012; Kloser et al., 2013; Harvey et al., 2014; Olimpo et al., 2016), whereas our model exposes students to multiple areas. The advantage is that application of the scientific process to different areas reinforces research skills and students’ understanding of the process of science. Furthermore, if students do not connect with one research area, they may appreciate another.

Using faculty instructors’ own research programs as bases for UMARL courses reduced our funding needs and kept the labs financially sustainable. In general, the cost of CUREs ranges widely; the few studies that have reported costs estimated from as little as $20 to as much as $500 per student (Rowland et al., 2012; Burnette and Wessler, 2013; Harvey et al., 2014; Russell et al., 2015; Dolan, 2016). Each UMARL section had a supplies budget of $1000, about $42 per student. Although laboratory equipment requirements could make start-up costs steep, we avoided that by using faculty instructors’ own specialized lab equipment. We purchased general equipment such as balances and micropipettes for the UMARL courses with grant and departmental funds, but the research areas of faculty were too diverse to purchase all the equipment necessary for each research theme. As to unanticipated supplies needed for particular student-designed projects, faculty often could supply small amounts of necessary reagents from their research lab, thereby avoiding the need to buy a large amount of a specialized reagent and wait for an order to arrive.

UMARL student access to sophisticated analytical instruments (e.g., microplate reader, CN analyzer, mass spectrometer, transmission electron microscope) that included trips to faculty members’ labs, had an unexpected beneficial effect. It effectively emphasized to students that they were doing authentic research in a far more convincing way than just telling them that. The use of sophisticated equipment also may contribute to publication-quality data (Weaver et al., 2008), a component of authentic CUREs. In contrast, the use of simple, unsophisticated measuring equipment, such as student-constructed respirometers, which we tried in one of the very first labs in an attempt to demystify complex lab equipment, was abhorrent to students and characterized by them as worse than what they did in their high school labs. Being allowed to use (with close supervision) complicated, often computer-controlled, research lab instruments invariably excited and engaged students.

Because each teaching team oversees four different research projects simultaneously, teaching UMARL can be demanding. A graduate teaching assistant and undergraduate peer facilitator alleviated some of that intensity for the faculty instructors, but the three-person teaching teams did make the labs personnel intensive. Furthermore, faculty were given reduced teaching loads to help compensate for the time demands of teaching UMARL. Despite the challenges, faculty were attracted to teaching the labs, partly because they could effectively “prescreen” and then recruit students for individual research in their labs beginning in those students’ second years. Moreover, research projects in UMARL often provided novel results that were informative for faculty research and that could be a basis for UREs. At least one such research project led to a publication with a UMARL student (e.g., Janos et al., 2008), for which the research conducted as part of the course was continued and expanded. In UMARL, faculty instructors most comfortable with the unscripted nature of the labs engendered the most authentic research learning experiences. Faculty must be comfortable with free-form, impromptu instruction and able to “think on their feet” in order to guide students with interpretation of unexpected results and problem solving.

Student frustration and dissatisfaction with the authentic research labs could be high when things did not work as anticipated. Even though students’ grades were not related to the conclusiveness of their experiments, first-year students fresh from high school cookbook labs are not prepared for setbacks. The teaching team had to help students understand that failures of technique, problem solving, and revision of experimental design are all realities of “real” science (and, while somewhat painful, can greatly impress upon students the authenticity of their experiences).

Another frustration for both students and faculty was the time constraint of 6 or 7 weeks (a half-semester for each research theme) of 3-hour labs. Not only were students trying to conclude a research project, but especially in the first semester, they also were developing proficiency with presentation software, mastering computational and spreadsheet skills, gaining an ability to interpret simple statistical tests, and learning the standard organization of a scientific paper and how to search the scientific literature. In response, some faculty and teaching assistants convened optional, “open” labs in lieu of office hours and facilitated by the availability of the UMARL classrooms for exclusive use by the UMARL course.

UMARL was designed knowing that there must be a compromise between the number of different research themes covered and the amount of time available to obtain conclusive data. Students often remarked that they wished a particular research project could run for an entire semester, but others, who did not like a particular theme, looked forward to the next one. Overall, our experience is that it is best to “leave them wanting more.” Thereby, students learn that research seldom is conclusive. Moreover, exposure to multiple research areas provides students with breadth. Four different research projects in two semesters also involve repetition that reinforces research skills and builds students’ self-efficacy.

Through feedback from students, we learned that many felt UMARL should confer more than one credit, because it is “more work” than the traditional biology labs. The traditional labs frequently have quizzes and homework assignments, so we suspect that the “more work” of the authentic research labs meant that they required student initiative, creativity, and analytical thinking. Because students might have had to work outside the assigned lab period (e.g., for off-campus sample collection or group poster preparation), additional credit could be justified. Nevertheless, we kept the authentic research labs on par with the traditional labs at 1 credit hour so as not to disrupt the requirements of the biology or other STEM majors.

Over our more than a decade of offering UMARL, we have found that the most successful and authentic research themes are relevant to topics of societal and scientific importance and have a high multiplication product of the numbers of independent and dependent variables that students can control and assess. High multiplication products allow very many possible investigations. Successful research themes have not relied on highly developed or extensive taxonomic, methodological, or conceptual expertise. They had easy access to adequate experimental subjects, rapid responses to experimentally manipulated variables, and closeness to the faculty member’s research. In contrast, our least successful projects involved excessive student downtime (e.g., waiting for access to an expensive, single, critical instrument or piece of equipment that all student groups needed to use; waiting for a slow response to experimental treatments), too much complexity, or too many steps. Although we have found these research theme design considerations indispensable, we also have found many research faculty eager and able to meet them.

CONCLUSIONS

UMARL-style labs are neither difficult nor fiscally expensive to implement, and their benefits can shape students’ academic paths and perhaps their eventual career choices, because CUREs are thought to have the potential to increase the number and diversity of STEM graduates entering STEM careers (NRC, 2003; American Association for the Advancement of Science, 2011; PCAST, 2012). The UMARL model, however, is inescapably personnel intensive. Nevertheless, the UMARL approach is relatively easily transportable to other STEM disciplines and institutions with available faculty researchers. Recently, we have extended these authentic research labs to introductory chemistry and to a nearby community college.

We suggest that the student learning gains and objective outcomes we have reported vouchsafe that our UMARL approach is well worth its costs in personnel, time, and effort. The weighted means of all 21 learning gains self-reported by UMARL students exceeded national means for every item, evincing strong student appreciation for our labs. Our outcome data are objective and embody relatively large (more than 500 students in each group), propensity score matched groups. Although the inclusion among our treatment group of the highly qualified PRISM students (who would be expected to persist in STEM majors and to excel academically) could have influenced our observed outcomes, our traditional biology lab comparison group students were similarly qualified in academic background and potential (i.e., SAT score and matriculation with a STEM major). Taken together, those highly qualified groups might have minimized our chances of finding detectable, beneficial effects of UMARL. Despite that, the four outcomes we analyzed had significant odds ratios of 1.5–1.7, indicating strong merit of the UMARL model to contribute to student academic success and persistence in STEM.

ACKNOWLEDGMENTS

REFERENCES