Development and Evaluation of the Tigriopus Course-Based Undergraduate Research Experience: Impacts on Students’ Content Knowledge, Attitudes, and Motivation in a Majors Introductory Biology Course

INTRODUCTION

For several decades, evidence has suggested that engagement in authentic research practices is of significant importance for novices’ development of reasoning and literacy skills in the science, technology, engineering, and mathematics (STEM) disciplines (Holt et al., 1969; Sundberg et al., 2005). In response, recent efforts within the bioeducation community have focused on the development and evaluation of course-based undergraduate research experiences (CUREs)—opportunities that extend beyond traditional laboratory exercises and instead often challenge students to collaboratively develop or identify appropriate researchable questions, experimental protocols, and analytical approaches to make meaning of collected data (Auchincloss et al., 2014; Spell et al., 2014; Corwin et al., 2015). Though the structure of these authentic research experiences has been observed to vary widely (for instance, building upon faculty research interests [e.g., Brownell and Kloser, 2015] or aligning with national models such as the Howard Hughes Medical Institute’s SEA-PHAGES program [Jordan et al., 2014]), the documented impacts of student participation in CUREs have largely been positive. Using both quantitative and grounded theory approaches, Brownell et al. (2012, 2015) have shown, for instance, that students enrolled in introductory cell/molecular and organismal biology CUREs report a deeper appreciation for and interest in scientific research as compared with their peers completing traditional laboratory coursework. Furthermore, students enrolled in these research-intensive opportunities exhibit marked postinstructional shifts in their confidence in conducting authentic scientific research and in their ability to “think like a scientist” (Brownell et al., 2015)—essential mind-sets for success in the STEM workforce (National Research Council, 2003; American Association for the Advancement of Science [AAAS], 2011).

Despite the significance of these findings, the majority of empirical studies to date have used student self-reported metrics to assess participants’ attitudinal, motivational, and skills-based outcomes as a result of participation in CUREs (e.g., CURE survey; see Lopatto et al., 2008; described also in Corwin et al., 2015), and few, if any, have provided a comparative account of traditional versus CURE student outcomes within nonvolunteer laboratory courses at the introductory level (Spell et al., 2014; Makarevitch et al., 2015). Consequently, recent research indicates that these practices, which likewise include the use of unpublished or nonvalidated instruments within reported studies (Beck et al., 2014), recruitment bias (Brownell et al., 2013; Corwin et al., 2015), and overestimation of learning and aptitude within student self-reported data sets (Boud and Falchikov, 1989; Falchikov and Boud, 1989), make it difficult to discern the true extent to which CUREs impact cognitive and noncognitive student attributes. To address these concerns, we adopted a mixed-methods approach to explore the following central questions:

1. What impact does participation in the Tigriopus CURE have on students’ content knowledge in the biological sciences as compared with a matched comparison group that participated in a parallel, traditional laboratory experience?

2. What impact does participation in the Tigriopus CURE have on students’ attitudes and motivation in biology as compared with a matched comparison group that participated in a parallel, traditional laboratory experience?

3. What differences, if any, exist in STEM versus non-STEM students’ shifts in attitude and motivation in biology following participation in the Tigriopus CURE?

4. To what degree were course and programmatic learning outcomes achieved as a result of implementation of the Tigriopus CURE?

We hypothesized that students participating in the CURE would exhibit greater expert-like shifts in attitudes, motivation, and content knowledge in the discipline than those students within the matched comparison group given the active- and inquiry-based nature of the authentic research opportunity. This hypothesis is supported by existent literature, which indicates a positive correlation between participant engagement in student-centered learning environments and affective and/or performance-based outcomes (Tai et al., 2006; Hansen and Birol, 2014). Because the academic interests of students enrolled in the Tigriopus CURE were anticipated to be diverse, given the dual function of the course as both a liberal arts core (non-STEM) option and a required survey course for several STEM disciplines on campus (see Supplemental Table S1; Batzli, 2005), we likewise found it imperative to assess for potential differences in affective shifts between STEM and non-STEM cohorts enrolled in the CURE. From a broader perspective, we believed the Tigriopus CURE allowed for greater targeting of programmatic learning objectives, namely, the enhancement of students’ scientific reasoning and professional skills in the domain, in a manner that had the potential to promote learning for all students (AAAS, 2011).

The CURE described herein, and the central research questions detailed above, are novel in several aspects. To the best of our knowledge, this is the first CURE to adopt a structure in which the initial hour of each week’s laboratory meeting is devoted to engaging students in an active learning-based supplemental instruction (SI) session designed to reinforce their understanding of content presented in the lecture portion of the course. This structure was adopted intentionally to ensure that a connection between lecture and laboratory experiences remained despite the fact that the CURE was no longer aligned with core content presented in the lecture. In addition, although the structure of the Tigriopus CURE (see Methods) precludes examination of associative or causal relationships between student engagement in SI and knowledge-based or affective outcomes within the context described, previous research suggests that voluntary student participation in SI results in increases in performance on summative course exams and more expert-like attitudes toward learning in the biological sciences (e.g., Batz et al., 2015).

Moreover, and with specific regard to the identified research questions, our approach offers a novel perspective on how students’ attitudes and motivation change not only in relation to participation in the course itself (whether traditional or research-intensive) but also macroscopically at the disciplinary level. As a result, our analyses provide insight into many aspects of traditional versus CURE laboratory experiences, including their impact on students’ career interest, self-efficacy, self-determination, problem-solving skills, and enjoyment in the biological sciences. These data are enhanced by students’ self-reported perceptions of learning gains related to the structure of the CURE itself. Collectively, such findings are of importance in understanding not only the immediate impacts of authentic research experiences on students’ growth in a cognitive and noncognitive sense but also how the structure of the CURE could be contributing to those outcomes as well.

CONCEPTUAL MODEL

The research presented here is situated within Corwin et al.’s (2015) conceptual model for evaluation of CUREs. In this model, several critical nodes are identified that exemplify short- and long-term outcomes associated with participation in CUREs and research-driven internship experiences previously documented within the literature. These nodes, which include increased communication skills and increased analytical skills, among other factors, are further linked to centralized hubs that serve as “gateways” to achieving longer-term outcomes (e.g., persistence in science). Within the parameters of our own research, we specifically sought to use the tenets of this model, including student self-efficacy (medium-term outcome), attitudes/motivation in biology (medium-term outcome), and increased content knowledge (short-term outcome), as dependent variables designed to compare student outcomes following participation in either a traditional laboratory course or the Tigriopus CURE. Although our objective was not to explicitly test the model proposed by the authors, we elected to focus on these short- and medium-term outcomes due, broadly, to their established relationships to student success and retention in the STEM disciplines (Seymour, 2000; Tai et al., 2006) and their posited links to student persistence resultant from engagement in CUREs (Corwin et al., 2015). Mixed-methods approaches were likewise adopted to provide a nuanced account of the potential intersection of these measured outcomes and students’ perceptions of the Tigriopus CURE. Collectively, these data are designed to provide insight not only into student outcomes obtained from engagement in either traditional or authentic research experiences but also the structural characteristics of the CURE that could be contributing to those outcomes.

METHODS

Participant Recruitment and Matching Procedures

Participants (n = 125; 97% of sampled population) represented a convenience sample consisting of all students enrolled in an introductory cell and molecular biology CURE at a midsized, doctoral degree–granting institution in the Spring 2015 semester. For comparative purposes, a matched comparison group (n = 125) consisting of students who had previously completed the traditional version of the laboratory course during the Fall 2014 term was likewise identified. Participants were matched on several demographic factors (e.g., index score [a measure of college readiness], major, and minority and first-generation statuses) demonstrated to be significant correlates of student success in introductory science coursework (Tai et al., 2006) so as to reduce confounding due to heterogeneity between groups. These demographic data were collected through the university’s system for institutional reporting (Supplemental Material). In an effort to further account for between-semester differences in student populations not captured explicitly through provided demographic information, only those students who were completing the course for the first time and who participated in all aspects of data collection were included in our analyses.

Specifically, CURE participants were first matched to one or more nonparticipants on those demographic variables referenced earlier. This resulted in exact matching on all factors excluding index score (Table 1). To account for the variation observed in this last variable, only those nonparticipants with an index score within one-half SD (± 5 units) of participants’ index scores were retained in the matching process. In instances in which multiple nonparticipants were identified as near matches to any single CURE participant, a random number generator was used to determine a one-to-one pairing. Once matching had been completed, an independent t test was performed to assess for differences in index score between groups. This analysis revealed no statistically significant, between-cohort difference in index score (t(248) = 0.065, p = 0.948). Although a high degree of homogeneity was achieved between the experimental and matched comparison groups, we acknowledge, as is also customary with similar matching procedures (e.g., propensity score matching; Stuart and Rubin, 2008), that it is only possible to account for covariates that are observable (and that were observed). In that same vein, however, it is important to note that participants were not selected on the basis of any other qualifying factors and did not receive course credit or compensation for their enrollment in this research.

CURE Learning Goals and Suboutcomes

The primary function of the CURE described herein was to increase student engagement and confidence in implementing authentic scientific practices, including both the processes of experimental design and science communication, and to improve students’ attitudes and motivation in the biological sciences. To accomplish these goals, we developed several suboutcomes in alignment with previous research (Auchincloss et al., 2014; Brownell et al., 2015). Specifically, these outcomes were 1) student development of a researchable question and subsequent hypothesis, the answer to which was previously unknown by the course instructor or other students participating in the CURE; 2) student–student and student–instructor collaboration throughout all aspects of the CURE, including defined roles for each student in a given laboratory team (Table 2; adapted from Luckie et al., 2004); and 3) iterative, constructive feedback on formative and summative assessments (e.g., weekly research updates, research proposals, and final oral presentations) provided by both the course instructor and students enrolled in the CURE. Specific details regarding course structure and implementation are described in the following section.

CURE Description and Implementation

Principles of Biology, an introductory cell and molecular course, is a mandatory class for students pursuing a degree in the biological sciences or a closely related field of study (e.g., chemistry, dietetics and nutrition, nursing). This course is typically completed by freshmen during their first semester at the university, and students are required to concurrently enroll in both the lecture (three 50-minute sessions weekly) and laboratory (one 3-hour session weekly) portions of the course. While this is the case, students retain the flexibility to complete the course in either semester and do so for numerous reasons (e.g., more equitable distribution of STEM courses across their first academic year, scheduling conflicts). Therefore, matching procedures were used to account for student demographic differences between semesters (see Participant Recruitment and Matching Procedures). In the Spring 2015 semester, the traditional laboratory exercises that had historically been used (Dickey, 2003) were replaced with the Tigriopus CURE (see Table 3 for a comparison of the traditional and CURE course learning objectives and laboratory curricula). This CURE was implemented across all sections of the laboratory portion of the course.

Building upon the expertise of faculty in the department, the planktonic copepod Tigriopus californicus was used as a model system for experimental investigations. This species is a common inhabitant of tidepools along the West Coast of the United States, where it serves as the foundation of many marine food chains. Although its congener, T. japonicas, has been well studied (e.g., Machida et al., 2002; Lee et al., 2007), relatively little is known about the biology of T. californicus. This feature provides an ideal platform for students to pose basic, unanswered questions about the organism, including inquiries about the impact of common abiotic factors, such as salinity, temperature, and diet, on survival. Furthermore, T. californicus is amenable to rearing under laboratory conditions with minimal cost, possesses a brief yet complex life cycle, and can tolerate a wide range of abiotic conditions, making it ideal for use in a large-enrollment introductory biology course.

In addition to being novel in its topical focus, the structure of the Tigriopus CURE is likewise noteworthy. Students spend the first 6 weeks of the semester mastering basic laboratory techniques essential for success in subsequently performing independent investigations, including microscopy, pipetting, serial dilutions, and statistical analysis. These modules are traditional in the sense that they possess prescripted procedures and predetermined outcomes similar to those exercises presented in the pre-existent laboratory manual (Dickey, 2003; Supplemental Material). While students are engaged in these exercises, they are simultaneously drafting a collaborative research proposal outlining the independent studies they intend to conduct. These proposals are critiqued and the experimental designs refined at multiple time points throughout the 6-week duration (Table 3). The remainder of the term is then allocated for students to conduct authentic research, with the final week devoted to oral group presentations of research findings. This structure was intentionally adopted to reduce the cognitive load associated with students’ assimilation and enactment of technical and scientific process skills.

Furthermore, and to the best of our knowledge, this is the first CURE to incorporate a format in which the initial hour of each 3-hour laboratory session is composed of an active learning–enhanced SI session intended to increase student interaction with lecture course content rather than experimental design principles (Brownell et al., 2015; Makarevitch et al., 2015; for sample activity, see Olimpo, 2015). This structural aspect was adopted in response to feedback on student course evaluations and surveys (J.T.O. and G.R.F., unpublished data), as well as departmental faculty input, which indicated a strong preference for retaining a bridge between lecture and laboratory portions of the course—a concern echoed within the literature (Temple et al., 2010; Banta et al., 2012).

CURE Student Research Projects

As evidenced in the preceding discussion (see also Table 3), a central hallmark of student engagement in the Tigriopus CURE was the development and enactment of an authentic independent research project. In selecting foci for these inquiries, student teams elected to pursue such questions as “What effect does temperature have on egg production in T. californicus?” and “What are the effects of exposure to low-sulfur diesel on T. californicus survival rates?” Content analyses of student research proposals (unpublished data) indicated a high degree of within-section variability in the type of question posed despite repetition of questions between sections. Regardless, the resultant experiments generally required students to grow and maintain copepods within a series of 24-well plates (for the purposes of replication), which were incubated at 20°C under a 14L:10D photoperiod and supplied with Tetraselmis algae (a common food source for T. californicus), unless otherwise dictated by the team’s experimental protocol. Data collected in accordance with students’ proposed research question(s) were analyzed using basic statistical approaches, including independent t test and analysis of variance procedures, and were included both in student laboratory notebooks as well as in final written and oral presentations of students’ experiments.

Measures of Content Knowledge

To determine the impact of the Tigriopus CURE on students’ development of content knowledge in the domain, we used a multivariate analysis of variance (MANOVA) procedure to compare midterm and final exam scores for CURE versus non-CURE participants. Summative assessments were identical between groups and consisted of 50 multiple-choice questions ranked at the recall, knowledge, and application levels of Bloom’s taxonomy (Bloom et al., 1956; Crowe et al., 2008; Supplemental Table S2). To maintain the integrity of test items between the Fall 2014 and Spring 2015 semesters, the course instructor retained all exams in a locked filing cabinet, and students were only permitted to review their exams in her office under strict observation. Student responses were recorded on Scantron forms and scored electronically, and a total percentage correct metric was obtained. Exam scores were subsequently entered into SPSS, version 22 (IBM, New York), for the purpose of future analysis.

Measures of Students’ Attitudes and Motivation in Biology

Student attitude and motivation data were collected using the Colorado Learning Attitudes in Science Survey–Biology (CLASS-Bio; Semsar et al., 2011) and the Science Motivation Questionnaire II–Biology (BMQ; Glynn et al., 2011), respectively. These instruments were selected based on both their previous validation for use in majors introductory biology contexts and existent literature detailing their use to examine shifts in majors and nonmajors students’ attitudes and motivation in traditional and active learning–based environments (Perkins et al., 2005; Semsar et al., 2011; Batz et al., 2015). The CLASS-Bio consists of 31 Likert-item questions designed to examine the extent to which students agree with expert responses on seven scales ranging from enjoyment to problem-solving strategies and reasoning in the domain. Percent favorable scores (0–100%) were generated for each participant on each scale (Semsar et al., 2011), indicating their degree of alignment with experts’ attitudinal beliefs on those factors. With regard to interpretation of these findings, previous research has demonstrated that students enrolled in introductory biology courses tend to exhibit negative, pre/postsemester shifts in attitudes, indicating a decline in expert-like thinking (Semsar et al., 2011). It is important to note, however, that student engagement in active learning–based STEM curricula has been found to result in no or more expert-like, pre/postsemester shifts in scientific attitudes among novices (Perkins et al., 2005).

In comparison, the BMQ consists of 25 Likert-item questions regarding intrinsic and extrinsic factors related to students’ motivation in the biological sciences (e.g., career and grade motivation, self-efficacy, self-determination). Each item is scored on a scale ranging from zero to four points (corresponding with “strongly disagree” to “strongly agree”), and a total score is tabulated for each factor. Both diagnostics were administered during the first and 14th weeks of the semester in one 45-minute block so as to assess for pre/postsemester shifts in student outcomes. Student responses were recorded on Scantron forms, scored electronically, and entered directly into SPSS, version 22, for future analysis.

Overview of CLASS-Bio and BMQ Quantitative Data Analyses

Psychometric analyses indicated a high degree of construct validity (as established through expert panel and participant review processes) and reliability for both the CLASS-Bio and BMQ (Cronbach’s α = 0.847 and 0.852, respectively). MANOVA procedures were used to assess student outcomes resulting from administration of these instruments, in accordance with previously published protocols (Glynn et al., 2011; Semsar et al., 2011).

Student Perceptions of Learning Gains (SPLG)

To better understand how the Tigriopus CURE might have impacted student learning and attitudes in biology, we asked CURE participants to complete a brief survey during the final week of the semester. This survey consisted of two open-ended questions (“What elements or characteristics of this semesters’ laboratory experience did you find most enjoyable, and why?” and “What do you believe you learned from taking part in this experience?”) and three Likert-item questions (Supplemental Material) adapted from Kloser et al. (2013) and based on semistructured and focus group interviews conducted with students, faculty, and graduate teaching assistants (GTAs) at the university at which this research occurred (J.T.O., unpublished data). We elected to focus on the construct of enjoyment, specifically, as a mechanism to 1) further expand upon data collected from the Enjoyment scale on the CLASS-Bio and 2) provide formative feedback on successful elements of the course itself. Such analysis was likewise motivated by existent literature, which suggests an integral role of laboratory investigations, particularly those of an authentic nature, in stimulating student enjoyment and success in the STEM disciplines (Thompson and Soyibo, 2002; Lunetta et al., 2007). Open-ended responses were analyzed using a descriptive interpretive approach (Tesch, 2013), with emergent themes identified following iterative cycles of open and axial coding. Each student response was coded independently by two researchers with expertise in the field of science education. High interrater reliability was observed between coders (κ = 0.97, p < 0.001), with all disputes resolved by a third researcher with similar expertise. Descriptive statistics were tabulated for Likert-item questions, with potential rankings ranging from 1 (strongly agree) to 5 (strongly disagree), and the data entered directly into SPSS, version 22, for future analysis.

RESULTS

To control for potential bias introduced as a result of variation in laboratory GTA instructor, we first analyzed the aforementioned outcome variables using a MANOVA procedure. Results from this analysis indicated no significant difference in student outcomes based on GTA instructor (F(108,1680) = 0.97, p = 0.575, Wilk’s Λ = 0.64, η_p² = 0.05). Furthermore, it is important to note that, although we acknowledge that natural variation will exist in how material is delivered across various semesters of a given course, the lecture instructor, PowerPoint lecture content, general pedagogical methods used (e.g., use of clicker questions and case studies), and summative assessments administered as part of the lecture courses referenced herein were identical between groups. Remaining assumptions for performing parametric tests were confirmed prior to data analysis.

Participation in the Tigriopus CURE Facilitates Students’ Development of Content Knowledge in the Discipline

To test our hypothesis that engagement in the Tigriopus CURE enhanced students’ content knowledge in the biological sciences relative to the matched comparison group, we compared students’ midterm and final exam scores between treatment groups using a MANOVA procedure. These data indicated a statistically significant difference in overall percentage grade obtained between CURE and non-CURE participants on the first and third midterm assessments (F(4,245) = 3.20, p = 0.014, Wilk’s Λ = 0.95, η_p² = 0.06; Table 4).

Importantly, though student performance on the second midterm and final course exams was not found to be statistically different between groups, CURE participants scored an average of 3–4% higher on these assessments than their non-CURE peers.

Student Engagement in the Tigriopus CURE Leads to More Expert-Like Attitudes and Motivation in Biology than Student Engagement in a Traditional Laboratory Course

Students’ overall shift on the CLASS-Bio was compared initially using an independent t test procedure to assess for between-group differences. Results were statistically significant (t(248) = −4.89, p < 0.001), suggesting that it was appropriate to proceed in further analyzing students’ responses on the remaining seven subscales of the diagnostic. This analysis, accomplished through use of a MANOVA procedure, revealed statistically significant differences between the CURE and matched comparison groups on all subscales with the exception of the Problem-Solving Difficulty factor (F(7,242) = 4.27, p < 0.001, Wilk’s Λ = 0.89, η_p² = 0.11; Figure 1 and Supplemental Table S3).

Similar analyses were subsequently used to assess for differences in CURE versus non-CURE students’ shifts in both internal and external motivation as measured via the BMQ. Results from a MANOVA procedure indicated a significant between-group difference on all factors, excluding grade motivation (F(5,244) = 5.23, p < 0.001, Wilk’s Λ = 0.90, η_p² = 0.10; Figure 2 and Supplemental Table S4).

Participation in the Tigriopus CURE Results in Parallel Shifts in STEM and Non-STEM Students’ Attitudes and Motivation in Biology

To further examine the impact of participation in the Tigriopus CURE on students’ attitudes and motivation in the discipline, we stratified data by type of major (STEM vs. non-STEM) in accordance with published classification guidelines (Louis Stokes Alliance for Minority Participation, 2015; Supplemental Table S1). Results from a MANOVA procedure revealed no statistically significant differences between STEM (n = 50) and non-STEM (n = 75) individuals who had participated in the CURE on either the CLASS-Bio or BMQ diagnostics (F_CLASS-Bio(7,117) = 0.57, p = 0.783, Wilk’s Λ = 0.97, η_p² = 0.03; F_BMQ(5,119) = 0.95, p = 0.445, Wilk’s Λ = 0.96, η_p² = 0.04; Figure 3; p ≥ 0.139 for all planned comparisons). A posteriori multiple linear regression analyses were performed, in which the 12 CLASS-Bio and BMQ factors served as individual output variables, and type of laboratory experience (CURE vs. non-CURE), major (STEM vs. non-STEM), and type of laboratory experience × major were included as input variables. This analysis indicated no significant interaction effect for any of the output variables measured (p ≥ 0.155 for all potential interactions). Collectively, these data suggest that there are no statistically significant within-group differences in attitudinal or motivational outcomes for STEM versus non-STEM participants enrolled in either the traditional or CURE laboratory experience.

Increased Autonomy, Opportunities for Collaboration, and Technical Training Identified as Critical Factors Related to Student Enjoyment and Learning Following Participation in the Tigriopus CURE

In addition to explicitly performing a comparative assessment of the impact of traditional versus authentic laboratory experiences on students’ development of content knowledge, attitudes, and motivation in the domain, we sought to understand what specific features of the CURE might be mediating those observed outcomes. Descriptive interpretive analysis (Tesch, 2013; Supplemental Table S5) of student responses to those open-ended questions presented on the end-of-term SPLG revealed four emergent themes within the data set: 1) an appreciation for autonomy in all aspects of the experimental design process; 2) increased understanding of and confidence in performing laboratory techniques, as well as a perceived improvement in acquiring both general (e.g., critical thinking) and laboratory-specific skills; 3) increased interest as a result of collaborative efforts in the laboratory, leading also to improvement in written and oral science communication skills; and 4) a deeper respect and appreciation for what it means to “do” science. Importantly, our analyses indicated that these themes, albeit to differing extents, were apparent in students’ responses to both of the open-ended prompts.

Specifically, among these themes, autonomy was cited most frequently (72.8% of participants; Table 5A) as contributing to student enjoyment in the CURE, with techniques/skills and collaboration cited in relatively equal proportion (11.2% and 14.4% of participants, respectively) thereafter, and appreciation for “doing” science cited least frequently (6.4% of participants). Conversely, in regard to those self-identified factors contributing to student learning, appreciation for “doing” science and techniques/skills were cited most frequently (47.2% and 39.2% of participants, respectively; Table 5B), with collaboration (23.2% of participants) and autonomy (15.5% of participants) cited thereafter. In interpreting these outcomes, it is important to note that, across both open-ended questions, more than 85% of CURE participants’ responses were identified as belonging to two or more of the aforementioned coding categories. One student stated, for instance, that “I enjoyed all of it—learning how to use microscopes, graphing, and, finally, working with a group to conduct our very own experiments with our own test subjects. I felt like I was actually being prepared for real world research.” Responses such as this demonstrate that, in agreement with Corwin et al.’s (2015) framework, both CURE-specific activities (e.g., collecting novel data, working collaboratively with peers) and short-term outcomes (e.g., increased project ownership, increased technical skills) are likely exerting a combinatorial influence on students’ perceptions of the CURE learning environment, including students’ attitudes in the discipline (e.g., enjoyment, a medium-term outcome).

Data Indicate That Course and Programmatic Goals and Outcomes Were Achieved

Owing to the novel nature of the CURE and the fact that it was developed by faculty at the institution where this research was conducted, we found it imperative to ensure that learning goals and outcomes were being achieved from both a course-based and programmatic standpoint. Student responses to the end-of-term SPLG Likert-item questions were first analyzed using Friedman’s test to assess for within-group differences on the following three categories: 1) confidence in developing scientific questions/hypotheses; 2) confidence in designing an experimental protocol to examine a research question of interest; and 3) confidence in interpreting and presenting one’s findings. Results were significant (χ²(4) = 37.54, p < 0.001), indicating a statistical difference in mean rank between categories. Pairwise comparisons were subsequently performed using the sign test, with Bonferroni correction applied. These analyses indicated no significant difference in students’ self-reported confidence in developing scientific questions or hypotheses (M = 1.95, SD = 0.73) and designing experimental protocols to test those hypotheses (M = 1.95, SD = 0.66, p = 0.742). However, a statistically significant difference was noted between students’ self-reported confidence in interpreting and presenting their results (M = 2.17, SD = 0.66) and their confidence in developing research questions and hypotheses (p = 0.007) as well as in students’ self-reported confidence in interpreting and presenting their results and their confidence in designing experimental protocols to test those hypotheses (p = 0.010). These findings suggest that, although data analysis and interpretation are integral components of the CURE described herein, future iterations of the CURE might benefit from additional attention being placed on these critical aspects of the experimental design process.

Because students from both the STEM and non-STEM disciplines were enrolled in the CURE, we wanted to confirm, furthermore, that the aforementioned course learning outcomes were being met equally between groups. To address this concern, we performed a Kruskal-Wallis test. This analysis indicated no significant difference between groups in any of the aforementioned categories (Table 6), suggesting that course learning objectives (as well as the first course suboutcome) were attained for both STEM and non-STEM participants.

For those interested in developing and evaluating CUREs at their own institutions, it is also important to note, from a broader standpoint, that the CURE described here not only was inclusive of those programmatic learning outcomes required for students majoring in the biological sciences at the institution at which this research was conducted (S. Keenan, personal communication), but also expanded upon those outcomes in novel ways as compared with the traditional laboratory experience previously in existence (Table 7).

DISCUSSION

For several decades, evidence has suggested that student engagement in authentic research opportunities is important for their development of scientific reasoning and process skills in the STEM domains (Holt et al., 1969; Sundberg et al., 2005). Current reports in the literature corroborate this assertion, indicating that not only do students participating in such experiences develop the skills necessary to “think like a scientist,” they also demonstrate a deeper appreciation for and interest in scientific research (Brownell et al., 2012, 2015; Kloser et al., 2013). Despite these critical benefits, little remains known about the broader impact of CUREs on students’ attitudes, motivation, and content knowledge within the discipline. This is especially true for CUREs implemented in nonvolunteer course contexts. Likewise, few studies have performed a comparative examination of these outcomes among STEM versus non-STEM students engaged in CUREs (e.g., Caruso et al., 2009). To address these concerns, we adopted a mixed-methods approach to broadly evaluate the implementation of a novel CURE in an introductory cell and molecular biology course, with specific assessment of the aforementioned cognitive and noncognitive outcomes conducted.

With regard to students’ development of content knowledge in the discipline, our findings indicate a statistically significant, between-group difference, in which CURE participants outperform the matched comparison group on the first and third midterm examinations by approximately half a letter grade. These data support a now well-described phenomenon within the science education community—student engagement in active learning–based exercises promotes intellectual growth (Freeman et al., 2014). Furthermore, this research is in agreement with current studies of the impact of SI on student learning (Rath et al., 2007; Batz et al., 2015; Snyder et al., 2015). Batz et al. (2015) have shown, for instance, that struggling students who regularly attend peer tutoring exhibit increased exam performance, better attitudes in biology, and greater course persistence relative to a matched control group that did not attend the tutoring sessions. Although our current analyses preclude us from establishing a causal link between SI implementation and increases in students’ content knowledge in the domain, our data indicate that the structure of the Tigriopus CURE, as a whole, was effective at promoting this latter outcome.

In addition to differential gains in content knowledge, we observed significant between-group differences in CURE versus non-CURE students’ shifts in attitudes and motivation following participation in the laboratory experience. In support of our original hypothesis, students enrolled in the CURE exhibited markedly expert-like shifts (or less novice-like shifts) on attitudinal constructs relative to the matched comparison group, whose responses were, collectively, more novice-like in nature. Similarly, more positive motivations were observed for those individuals participating in the CURE versus those individuals completing traditional laboratory coursework. It is important to note that such shifts were witnessed across a multitude of factors, including self-determination, career interest, and problem-solving strategies, suggesting that the CURE had a wide-reaching impact as compared with the traditional laboratory experience. These data are in accordance with previous reports, which have demonstrated that student engagement in authentic research opportunities results in increases in self-efficacy and intrinsic motivation in the sciences, including increases in students’ perceived ability to prepare for and conduct studies (Drew and Triplett, 2008), greater clarification of students’ academic and/or career paths (Lopatto et al., 2008; Shaffer et al., 2014), and a deeper understanding of how scientists engage in real-world problem solving (Shaffer et al., 2014).

In addition, the results presented herein are novel for several reasons. First, in comparison with previous studies, which have largely provided a subjective account of students’ self-reported perceptions of the CURE itself or of students’ competency in performing experimental design tasks (Lopatto et al., 2008; Brownell et al., 2012), the data presented herein speak more objectively to the broader, discipline-based outcomes associated with participation in authentic, course-based research opportunities. Though such metrics are not examined in this paper, we argue that they are of significant importance in offering a common foundation upon which to evaluate and compare CUREs and longitudinal student outcomes resulting from engagement in such opportunities, as they are designed independently of any one particular CURE. Second, the similarity in attitudinal and motivational outcomes observed between STEM and non-STEM participants enrolled in the CURE is critical given that introductory cell and molecular biology continues to remain a “gatekeeper” course for several other programs of study at institutions nationwide (including our own; Gasiewski et al., 2012) and is identified as a liberal arts core course at the university at which this research was conducted. Consequently, and in light of persistent concerns regarding student success and retention across academic disciplines (e.g., Seymour, 2000; Whalen and Shelley, 2010; Palmer et al., 2011), it is essential that the curricular and cocurricular elements incorporated into such courses serve to enhance the educational experience in a manner that maximizes opportunities for intellectual growth and professional development for all students (AAAS, 2011).

CONCLUSIONS: CONSIDERING COURSE AND PROGRAMMATIC OUTCOMES

The development and evaluation of novel CUREs requires considerable time and resources (Spell et al., 2014). As such, we found it imperative to assess the degree to which student engagement in the Tigriopus CURE allowed for both course and programmatic learning outcomes to be achieved. Mixed-methods analyses of end-of-term SPLG survey data revealed that, while students perceived themselves to be competent in constructing a researchable question and appropriate experimental protocol to address that question, they were less confident in their ability to interpret and present their results. We believe this difference in perception is associated with the statistics-based analytical techniques embedded within our CURE. Though students enrolled in the CURE receive a module dedicated specifically to statistical methods (and though these methods are considered basic in terms of statistical approaches), participants have not yet had an opportunity to complete any coursework in this area. In alignment with previous research (Feser et al., 2013; Makarevitch et al., 2015), we argue that a greater focus on quantitative reasoning skills and quantitative literacy is needed in contexts such as the one described herein, particularly as a mechanism to increase student awareness of the relevancy of mathematics to biological sciences research. Current efforts within our laboratory are focused on the impact of curricular and cocurricular interventions designed to address these concerns and on the relationship between students’ ability to analyze research data and their beliefs about mathematics in the context of biology.

In addition to students’ perceived competence in using scientific process skills, qualitative analyses of open-ended SPLG items indicated that student learning and affect, as they were related to the authentic research opportunity, were primarily associated with four characteristics or features of the experience, namely, 1) an appreciation for autonomy in all aspects of the experimental design process; 2) increased understanding of and confidence in performing laboratory techniques, as well as a perceived improvement in acquiring both general (e.g., critical thinking) and laboratory-specific skills; 3) increased interest as a result of collaborative efforts in the laboratory, leading also to improvement in written and oral science communication skills; and 4) a deeper appreciation for what it means to “do” science. These data are supported by previous evidence in the field (e.g., Brownell et al., 2012, 2015) and speak directly to Corwin et al.’s (2015) framework for evaluating CUREs. Specifically, we contend that these results provide novel insight into how CUREs impact student outcomes, with core structural elements of the Tigriopus CURE, including increased technical and communication skills, sharing a potential, positive relationship with both medium- and long-term outcomes (e.g., greater career interest in biology, increased self-efficacy and enjoyment) within Corwin et al.’s (2015) model. Though beyond the scope of the present study, ongoing research within our group seeks to more rigorously test these assumptions and Corwin and colleagues’ model across multiple CURE experiences and with larger participant samples—an area we strongly encourage other CURE education researchers to engage in as well.

Finally, and perhaps of greatest importance to those seeking to create CUREs at their own institutions, the CURE presented here addresses several programmatic learning outcomes not achieved previously through use of the traditional laboratory experience (see Table 7). While we acknowledge that each institution and program possesses its own complement of goals and measurable outcomes, we argue that authentic research experiences of this nature are essential, in a broader sense, because they provide all students with an opportunity to engage in the research process from question and hypothesis development to presentation of results (AAAS, 2011; Bangera and Brownell, 2014), further offering a potential solution to the limited number of opportunities available for students to engage in research in independent faculty laboratories (Desai et al., 2008). In addition, Corwin and colleagues (2015) state, importantly, that

As we, the educational community, continue to purposefully construct and embed authentic research experiences within our courses, we challenge this mindset by creating contexts in which students possess the autonomy to engage in real-world scientific practices that address questions of interest to them and to the scientific community at large.

FUTURE DIRECTIONS

Continued development and integration of CUREs in the biological sciences and other STEM disciplines predicates a need for objective assessment of outcomes associated with implementation of these experiences. At present, much of the research associated with CUREs, including that described herein, has focused on student outcomes following engagement in either traditional or authentic research-based laboratory coursework (see Table 1 in Corwin et al., 2015, for a summary of such studies). While such findings are imperative and continued studies of this nature are needed (e.g., identifying appropriate measures of students’ scientific process skills; Dasgupta et al., 2016), we contend, as have others (see Corwin et al., 2015), that a critical focus on alternate contextual features of CUREs is likewise warranted. For instance, though limitations in sample size within the present study preclude us from doing so, future research employing generalized linear mixed modeling approaches might seek to better understand the role GTAs have in promoting student learning and interaction in CURE versus traditional laboratory environments.

In addition, reported outcomes within the literature have historically been associated with implementation of an institution-specific CURE or, alternatively, multi-institutional implementation of a national CURE model (e.g., HHMI SEA-PHAGES [Jordan et al., 2014]; Genomics Education Partnership [see Shaffer et al., 2014, as an example]; Brownell et al., 2015). Because these experiences are each arguably unique in topic and structure, CURE–CURE comparisons using the same series of assessments and defined outcomes could be of great interest to the community (akin to Freeman et al.’s (2014) description of “second generation” research), particularly in allowing for better articulation of which features of CUREs are beneficial and for whom. Indeed, Spell et al. (2014) suggest that varying conceptions of what it means to engage in authentic research exist. Specifically, the authors indicate that these conceptions often focus either on the “products” of science, wherein the expectation is to generate novel questions leading to publishable results, or the “process” of science, in which students are expected to generate their own questions/hypotheses, conduct experiments, and analyze/present the findings of their study. It is not our intent, as Spell et al. (2014) contend, to suggest that all CUREs should encompass both conceptions. Instead, we propose that a collective evaluation of such experiences, both individually as well as comparatively, when coupled with existent, ongoing efforts in the field (e.g., CUREnet: http://curenet.cns.utexas.edu), offer a potential mechanism to better represent both the small- and large-scale outcomes associated with CURE implementation across a multitude of diverse contexts.

ACKNOWLEDGMENTS

REFERENCES