The Impact of Broadly Relevant Novel Discoveries on Student Project Ownership in a Traditional Lab Course Turned CURE
Abstract
Course-based undergraduate research experiences (CUREs) have been shown to lead to multiple student benefits, but much is unknown about how CUREs lead to specific student outcomes. In this study, we examined the extent to which students making “broadly relevant novel discoveries” impacted student project ownership by comparing the experiences of students in a CURE and a traditional lab course. The CURE and traditional lab were similar in most aspects; students were exposed to an identical curriculum taught by the same instructor. However, there was one major difference between the two types of courses: the type of data that the students produced. Students in the traditional lab characterized the immune system of wild-type mice, thereby confirming results already known to the scientific community, while students in the CURE characterized the immune system of a mutant strain of mice, which produced broadly relevant novel discoveries. Compared with traditional lab students, CURE students reported higher cognitive and emotional ownership over their projects. Students’ perceptions of collaboration and making broadly relevant novel discoveries were significantly and positively related to their cognitive and emotional ownership. This work provides insight into the importance of integrating opportunities for broadly relevant novel discoveries in lab courses.
INTRODUCTION
Future studies should seek to identify and measure the variables that explain why specific aspects of undergraduate research experiences have impact (or not) on the students participating in undergraduate research experiences.—National Academies of Sciences, Engineering, and Medicine (2017, p. 173)
A number of national reports have championed undergraduate research as a high-impact practice in which all science undergraduates should engage (American Association for the Advancement of Science, 2011; National Academies of Sciences, Engineering, and Medicine [NASEM], 2015, 2017). Undergraduate research experiences have been shown to positively impact students by enhancing critical-thinking skills, fostering student enculturation into the scientific research community, and improving undergraduate persistence in college (Hunter et al., 2007; Jones et al., 2010; Thiry et al., 2011; Hernandez et al., 2018). However, only a subset of undergraduate science students typically participate in research because of the limited number of positions available in faculty-member research labs (Wood, 2003; President’s Council of Advisors on Science and Technology, 2012).
Course-based undergraduate research experiences, or CUREs, offer an alternative way to engage students in undergraduate research (Auchincloss et al., 2014). Instead of students joining faculty-member research labs to conduct research, students in a CURE enroll in a formal course and conduct a research project, typically during a single academic term (Auchincloss et al., 2014; Brownell and Kloser, 2015). By offering research experiences to students in a course, CUREs amplify the total number of research opportunities available to students, thereby increasing the number of students who can participate in undergraduate research (Bangera and Brownell, 2014).
CUREs have been shown to lead to many of the same student benefits as undergraduate research experiences in faculty-member labs (Corwin et al., 2015a; Linn et al., 2015; NASEM, 2015, 2017). Some of these benefits include gains in content knowledge (Shaffer et al., 2010), learning to think like a scientist (Brownell et al., 2015), becoming a published author (Leung et al., 2015; Cooper and Brownell, 2018; Cooper et al., 2018b, 2019), and persistence in undergraduate science (Rodenbusch et al., 2016).
To define important elements of a CURE and distinguish how CUREs are distinct from other learning experiences, a group of education researchers met in 2013 and proposed five design features of CUREs. These design features are scientific practices, collaboration, iteration, discovery, and broad relevance (Auchincloss et al., 2014). Engaging students in scientific practices means having students do what scientists do, and can include tasks such as collecting data, proposing hypotheses, or communicating results. Collaboration refers to students working together to solve a scientific problem. Students engage in iteration by building upon prior work that was published by the scientific community, by continuing research that was started in a faculty-member lab, by repeating and revising their own work, or by replicating an experiment done by other students within the same course. The element of discovery refers to generating results that are novel to the student, the instructor, and relevant stakeholders outside the class (e.g., scientific community or local community). Broadly relevant work implies that the research findings will have potential impact beyond the classroom, which could mean that the research will affect the local community or is potentially publishable in a scientific journal; the term “broadly” is used to distinguish between relevance beyond the course and personal relevance (Leiserowitz, 2007; Maio and Haddock, 2007). This articulated set of five design features of CUREs was an important first step to help the CURE community create a working definition of a CURE and to establish a common language for CUREs.
Since the Auchincloss and colleagues’ (2014) meeting report was published, there have been a number of publications that have critiqued, modified, and expanded on the specific design features of CUREs, what comprises those design features, and which design features are essential for a course to be considered a CURE (Brownell and Kloser, 2015; Corwin et al., 2015b; Rowland et al., 2016). Notably, Brownell and Kloser (2015) argued that discovery and broad relevance are not two separate design features but rather act as a single construct. This assertion that the design features of discovery and broad relevance are not separate constructs in a CURE was corroborated by Corwin and colleagues (2015b) when they developed the Laboratory Course Assessment Survey (LCAS) to measure students’ perceptions of design features of lab courses. They found that questions measuring discovery and questions measuring broad relevance loaded onto a single factor, leading them to combine these two design features into one scale on the LCAS called “Discovery/Relevance.” Further, Cooper and colleagues (2017b) proposed that the defining feature of a CURE—what makes a CURE unlike other types of lab courses—is the combination of discovery and broad relevance. Specifically, the aspect of a CURE that makes it “real research,” similar to the type of research that happens in faculty-member research labs, is “broadly relevant novel discoveries.” While there have been discussions about the definition of authentic research and the extent to which CUREs need to be authentic for students to benefit (Spell et al., 2014; Rowland et al., 2016), we assert that, if the research project embedded in the CURE is neither novel nor broadly relevant, then it is not research and the course is not a CURE. Throughout this manuscript, we will use the phrase “broadly relevant novel discoveries” to describe novel discoveries made in a CURE that are of broad interest to stakeholders outside the course.
Researchers have hypothesized that making broadly relevant novel discoveries may be particularly important for students’ development of project ownership (Corwin et al., 2015a). Project ownership is defined as the extent to which students perceive that they have ownership over their work (Hanauer et al., 2012). The construct of project ownership is multifaceted and encompasses the following: 1) a connection between a student’s personal history and scientific inquiry, so students bring their past personal and educational experiences into their research; 2) practicing agency or actively seeking advice or direction to make progress on research; 3) expressing excitement toward doing science; 4) overcoming challenging moments in science; and 5) expressing positive emotions when achieving a specific goal in science (Hanauer et al., 2012; Hanauer and Dolan, 2014). An increased sense of project ownership has been suggested to help students become more tolerant of obstacles and to persevere when facing research-related challenges (Ward et al., 2002; Laursen et al., 2010; Hanauer et al., 2012; Alkaher and Dolan, 2014; Corwin et al., 2015a), which in turn has been hypothesized to increase students’ self-efficacy and motivation (Corwin et al., 2015a). Further, students who express more project ownership have demonstrated a better understanding of the unpredictability of scientific research (Hanauer et al., 2012). Importantly, project ownership has also been shown to predict students’ interest in pursuing science careers (Corwin et al., 2018b; Hanauer et al., 2012, 2016).
Hanauer and Dolan (2014) designed the Project Ownership Survey (POS) to measure project ownership. The POS measures both cognitive ownership, or the degree to which students feel as though they have intellectual responsibility over their work, and emotional ownership, or the strength of students’ emotions toward their work (Hanauer and Dolan, 2014; Corwin et al., 2018b). Multiple studies have demonstrated that students enrolled in CUREs have high project ownership (Hanauer et al., 2016), and some have found higher levels of project ownership for students who completed a CURE compared with students enrolled in traditional lab courses (Hanauer and Dolan, 2014; Hanauer et al., 2018). However, these studies did not determine what specific aspects of CUREs led to students’ enhanced feelings of ownership and thus were not able to conclude whether students working on broadly relevant novel discoveries enhanced their project ownership.
Recently, two research groups have explored how broadly relevant novel discoveries may lead to project ownership, but their findings conflict. Corwin and colleagues (2018b) conducted a nationwide study of ∼800 undergraduates enrolled in more than 23 different lab courses. They found that broadly relevant novel discoveries were significantly and positively related to students’ cognitive ownership of their lab work, but that broadly relevant novel discoveries were not related to students’ emotional ownership (Corwin et al., 2018b). In a different study, Ballen and colleagues (2018) surveyed ∼400 students in three different lab courses at a single institution that they defined as a CURE, an inquiry course, and a traditional lab course. This study concluded that there was no impact of broadly relevant novel discoveries on students’ project ownership (Ballen et al., 2018). There are notable caveats for these conflicting results, including possible validity issues in the Ballen study (Corwin et al., 2018a). First, to measure student project ownership, Corwin and colleagues used Hanauer and Dolan’s (2014) full 16-item POS, while Ballen and colleagues (2018) only used five modified items from the POS to measure project ownership. Further, Corwin and colleagues (2018b) measured student perceptions of broadly relevant novel discoveries as one construct using the LCAS Discovery/Relevance scale (Corwin et al., 2015b). In contrast, Ballen and colleagues did not formally measure student perceptions of discovery and broad relevance, but instead categorized the discovery and broad relevance for each course based on its general design.
Given these contradictory findings about the importance of broadly relevant novel discoveries, there is a need for additional studies to further understand the impact of broadly relevant novel discoveries on undergraduate science students. Further, the study designs of both the Ballen et al. (2018) study and the Corwin et al. (2018b) study made it impossible to control for some important aspects of lab courses that can differ between CUREs and traditional labs. These potentially different aspects, such as course instructors and the scientific practices that students engage in during the lab course, could impact project ownership. Prior literature has suggested that instructors who teach CUREs may be more innovative than instructors who teach traditional labs (Shortlidge et al., 2015, 2017), so it is possible that the differences between the CUREs and traditional labs are due to an instructor effect, or the extent to which a specific instructor influences students’ outcomes in a course. Additionally, students in different types of lab courses (e.g., traditional lab courses and CUREs) likely experience unique scientific practices, which could have a differential impact on project ownership (Corwin et al., 2015a). It is possible that the high project ownership demonstrated by CURE students, such as in the Corwin et al. (2018b) study, may be partially explained by engaging in specific types of scientific practices or the influence of CURE instructors.
To further clarify the potential impact of broadly relevant novel discoveries on undergraduates, we designed a study to compare the project ownership of students enrolled in two versions of the same upper-division undergraduate immunology lab course, with the only major difference between the courses being the extent to which students made broadly relevant novel discoveries. Students in the traditional lab version of the course characterized the immune system of wild-type mice, thereby confirming results already known to the scientific community, while students in the CURE version of the course characterized the immune system of a mutant strain of mice, producing broadly relevant novel discoveries. In this study, we compared the experiences of students in the two versions of the course to examine whether changing the design feature of broadly relevant novel discoveries impacted student cognitive and emotional ownership. The CURE and the traditional lab course were taught by the same faculty instructor (J.N.B.) and followed an identical curriculum. Thus, the courses were similar in all other lab course design features, including scientific practices, collaboration, and iteration; to our knowledge, the only major difference between the design of the traditional lab course and the CURE was whether the students worked with wild-type mice or mutant mice and the corresponding data that they produced. This unique research design allowed us to control for other aspects of the lab courses that may affect student project ownership, including the scientific practices that students engage in and the effect of the faculty instructor. Controlling for these aspects of lab courses has not been done before and allowed us to examine how the specific feature of broadly relevant novel discoveries impacted project ownership among college biology students.
Research Aims
Our research aims were as follows:
Identify to what extent, if at all, there are differences between the cognitive and emotional ownership of students in the CURE and students in the traditional lab version of the course.
Examine to what extent students’ perceptions of collaboration, iteration, and discovery/relevance are predictive of students’ cognitive and emotional ownership.
Study Focus and Context
This study was conducted in an upper-level immunology stand-alone lab course that was taught in the Spring semesters of 2016, 2017, and 2018. The 2016 version of the course was taught only as a traditional lab course, and the 2017 and 2018 versions of the course were taught only as a CURE.
Description of the Traditional Lab Course.
In the traditional version of this immunology lab course, students characterized the immune system of C57B1/6J wild-type mice. Scientists have already characterized the immune system of wild-type mice, so students confirmed previously published, known results (Blattman et al., 2016), thereby making it a traditional lab course. This course will be referred to as “the traditional lab course.”
Description of the CURE.
The traditional immunology lab course was backward designed to be a CURE (Cooper et al., 2017b) by changing only one component: broadly relevant novel discoveries. Instead of characterizing the immune system of wild-type mice, students conducted experiments with mutant mice that had an uncharacterized immune system. Students in the CURE evaluated whether the mutant strain of mice had differences in immune systems development and immune responses compared with wild-type mice, so all the student-generated results were novel and potentially publishable. The instructor explicitly told students in the CURE that they were working on a potentially publishable project that was novel and broadly relevant. This course will be referred to as “the CURE.”
Identical Characteristics of the Traditional Lab Course and the CURE.
All aspects of the courses, including the curriculum, the lab protocols, the assignments, and the faculty instructor,1 were the same between the two lab courses. To our knowledge, the only major difference between the traditional lab course and the CURE was in the type of mice examined and, thus, the novelty and broad relevance of the data produced by students. The traditional lab and CURE were 2-credit labs that met for 2 hours and 45 minutes twice a week for 14 weeks. Each year, there were two class sections with the potential to enroll 24 students each; a total of 40–48 students were enrolled each semester. The course title and description were the same for both versions of the course; students in the 2017 and 2018 courses were not told that the lab class was a CURE before enrolling. Students worked in small groups (n = 4) on all experiments and assignments in the course, although they had to turn in individual assignments. Course assignments included 1) daily quizzes before the start of each class that covered the material for that lab, which were meant to ensure that students understood the concepts and procedures they were doing that day; 2) lab notebooks that students filled out throughout the semester; and 3) five written laboratory reports for each of the major sections of the course. The five sections of the course were 1) anatomy and cells of the immune system, 2) innate immune responses, 3) B- and T-cell development, 4) adaptive responses to vaccination, and 5) immune memory and protection. Written laboratory reports included an introduction with hypotheses, materials and methods, results, discussion of how the results compared with hypotheses, and references; students in the traditional lab wrote about their confirmation results, and students in the CURE wrote about their novel results. A summary of the similarities and differences between design features of the traditional lab course and CURE is provided in Table 1.
| Traditional lab | CURE | |
|---|---|---|
| Scientific practices | Students were tasked with developing hypotheses; designing experiments using protocols in immunology, including flow cytometry, ELISA (enzyme-linked immunosorbent assay), cytolysis, and plaque assays; and analyzing data and writing lab reports. | Students were tasked with developing hypotheses; designing experiments using protocols in immunology, including flow cytometry, ELISA, cytolysis, and plaque assays; and analyzing data and writing lab reports. |
| Collaboration | Students worked in groups of four on all experiments and lab reports. | Students worked in groups of four on all experiments and lab reports. |
| Iteration | Students compared the data generated by their own groups to data generated by other groups. If an individual group had widely disparate results compared with other groups, students would need to include potential reasoning behind why their results did not match other groups’ results. | Students compared the data generated by their own groups to data generated by other groups. If an individual group had widely disparate results compared with other groups, students would need to include potential reasoning behind why their results did not match other groups’ results. |
| Broadly relevant novel discoveries | Students characterized the immune system of wild-type mice, so there were no broadly relevant novel results. | Students characterized the immune system of a mutant strain of mice, which have never been characterized before, and therefore a “broadly relevant novel discovery.” |
METHODS
This study was conducted with an approved Institutional Review Board protocol (#4249) from Arizona State University.
Participants
We collected data from students enrolled in the Spring 2016 traditional lab course and the Spring 2017 and Spring 2018 CURE courses. Thirty-two of the 40 students (80.0%) enrolled in the 2016 traditional lab course, and 72 of the 94 students (76.6%) enrolled in the CURE courses consented to participate in the study and were included in the data set. We collected data from two iterations of the CURE course to maximize the number of students in the study and combined the data; we compared the demographics of students in the two CURE courses and found no statistically significant differences. We also compared the demographics of students in the traditional lab course and students in the CURE using chi-square tests of independence and found no statistically significant differences (see the Supplemental Material for analyses). Student demographics are listed in Table 2.
| Demographics | Traditional lab course students (n = 32) n (%) | CURE students (n = 72) n (%) |
|---|---|---|
| Gender | ||
| Female | 17 (53.1) | 42 (58.3) |
| Male | 13 (40.6) | 28 (38.9) |
| Other | 2 (6.3) | 1 (1.4) |
| Declined to state | 0 (0.0) | 1 (1.4) |
| Race/ethnicity | ||
| Asian/Pacific Islander | 5 (15.6) | 7 (9.7) |
| Black or African American | 2 (6.3) | 5 (6.9) |
| Hispanic, Latino, or Spanish origin | 6 (18.8) | 16 (22.2) |
| White | 16 (50.0) | 39 (54.2) |
| Other | 2 (6.3) | 2 (2.8) |
| Declined to state | 1 (3.1) | 3 (4.2) |
| College generation status | ||
| First generation | 11 (34.4) | 29 (40.3) |
| Non–first generation | 21 (65.6) | 42 (58.3) |
| Declined to state | 0 (0.0) | 1 (1.4) |
| Previous research experience | ||
| No | 15 (46.9) | 27 (37.5) |
| Yes | 17 (53.1) | 45 (62.5) |
Measures
During the last week of the term, students in each course completed the same in-class survey, which consisted of 1) the LCAS, 2) a single item measuring to what extent students perceived they were participating in scientific research during their lab courses, 3) the POS, and 4) a short demographic survey.
Laboratory Class Assessment Survey.
The LCAS is a 17-item survey instrument that consists of three scales developed to measure students’ perceptions of three design features of biology lab courses: Collaboration, Iteration, and Discovery/Relevance (Corwin et al., 2015b). The LCAS does not measure students’ experience with scientific practices. The LCAS Collaboration scale measures students’ experience with collaboration in the context of a lab course using six items that evaluate the frequency with which students engage in activities related to collaboration (such as discussing work with other students); the response options for the collaboration items are never, one or two times, monthly, and weekly. The LCAS Iteration scale measures students’ agreement with six statements about the extent to which they have the time to experience iterative processes (such as repeating aspects of their work or revising their work); the response options for the iteration items are on a six-point scale ranging from strongly disagree to strongly agree. Finally, the LCAS Discovery/Relevance scale measures students’ experience with broadly relevant novel discoveries (Discovery/Relevance in Corwin et al., 2018b) with five items that ask students to rate the extent to which they agree that their work in the lab could lead to new discoveries and whether their data are of interest to the scientific community; the response options for discovery/relevance are on a six-point scale ranging from strongly disagree to strongly agree. We used Cronbach’s α to calculate reliabilities for the collaboration (α = 0.72), iteration (α = 0.77), and discovery/relevance (α = 0.86) scales, all of which were at an acceptable level (Nunnally, 1978). A copy of the LCAS can be found in the Supplemental Material.
Perception of Scientific Research.
We were interested in measuring the extent to which students perceived they were engaging in scientific research in the context of the lab course. We defined scientific research for the students as the type of research that is done in faculty-member labs and asked students to rate their agreement with the statement “I conducted scientific research in my experimental immunology lab course” on a 10-point scale from strongly disagree to strongly agree. We also asked students to explain their answers to this question in three to four sentences. To check whether students interpreted this question the way we intended, we conducted think-aloud interviews with four undergraduate biology students before administering the first survey (Trenor et al., 2011). A copy of this item can be found in the Supplemental Material.
Project Ownership Survey.
The POS is a 16-item survey instrument developed to measure students’ ownership of their research projects (Hanauer and Dolan, 2014). The POS consists of two subscales. The Cognitive Ownership subscale is composed of 10 items that ask students to what extent they agree that they had intellectual ownership of or responsibility for their lab work (e.g., “I was responsible for the outcomes of the work I did [in my experimental immunology lab course]”) with a five-point response scale ranging from strongly disagree to strongly agree. The Emotional Ownership subscale is composed of six items that measure the strength of students’ emotion toward their lab work (e.g., “To what extent does ‘astonished’ describe your experience of the laboratory course?”) with a five-point response scale ranging from very slightly to very strongly. We calculated reliabilities (Cronbach’s α) of the Cognitive Ownership (α = 0.86) and Emotional Ownership (α = 0.85) subscales and found both to be at an acceptable level (Nunnally, 1978). A copy of the POS can be found in the Supplemental Material.
Demographic Questions.
At the end of the in-class survey, students completed a set of demographic questions asking about their gender, race/ethnicity, college generation status, and prior research experiences. A copy of the demographic questions can be found in the Supplemental Material.
Data Analysis
Student Experience with Collaboration, Iteration, and Discovery/Relevance.
Using the results from the LCAS survey, we performed preliminary tests to ensure all statistical assumptions of our t tests were met. For each scale, we summed students’ responses to the respective questions. Bartlett’s test indicated that the assumption of homogeneity was met for the Collaboration and Iteration scales. However, it was not met for the Discovery/Relevance scale, and thus Welch’s df adjustment was made for the Discovery/Relevance scale only (Welch, 1947). We conducted independent-samples t tests to compare traditional lab student and CURE student mean scores on the Collaboration, Iteration, and Discovery/Relevance scales of the LCAS.
Student Perception of Engaging in Scientific Research.
Students rated the extent to which they agreed that they had conducted scientific research in the context of their immunology lab course from strongly disagree to strongly agree. Bartlett’s test indicated that the assumption of homogeneity was not met, and thus Welch’s df adjustment was made; we conducted independent-samples t tests to compare traditional lab student and CURE student mean scores.
After the students rated the extent to which they agreed that they were conducting scientific research in their lab courses, they were asked to explain their ratings. Together, two authors (K.M.C. and T.H.) reviewed all student responses about why students agreed or disagreed that they conducted scientific research in their immunology lab courses and used open-coding methods to identify common ideas in students’ reasoning (Strauss and Corbin, 1990). We used constant comparison methods to organize student responses into specific categories; quotes were assigned to a category and were continuously compared to ensure that the description of the category was inclusive of all quotes and that student quotes were not different enough from one another to warrant a different category (Glesne and Peshkin, 1992). We created a rubric describing each category after reviewing every student response (see the Supplemental Material for a copy of the coding rubric). A single student’s response could comprise multiple quotes that were each coded as a different category. Both researchers (K.M.C. and T.H.) used the rubric to independently code each student response, then compared their codes and discussed any discrepancies until they came to agreement. To see whether CURE students were more likely to report out a specific category than traditional lab students, we used chi-square tests of independence to compare the proportions of traditional lab students and CURE students who reported each category.
Student Cognitive and Emotional Ownership.
Using the results from the POS, we performed preliminary tests to ensure that all statistical assumptions of our t tests were met. For each subscale, we summed students’ responses to the respective questions. Bartlett’s test indicated that the assumption of homogeneity was met for both the Cognitive Ownership and Emotional Ownership subscales. We conducted independent-samples t tests to compare traditional lab student and CURE student mean scores on the Cognitive Ownership subscale and the Emotional Ownership subscale.
Relationship between Course Design Features and Student Cognitive and Emotional Ownership.
We were interested in examining the extent to which student perceptions of collaboration, iteration, and discovery/relevance varied within and between the traditional lab course and CURE course. We began by exploring the variance of student perceptions of each course design feature within each course type and generated density plots to visualize the distribution of traditional lab and CURE students’ scores on the Collaboration, Iteration, and Discovery/Relevance scales of the LCAS. A density plot allows for visualization of data over a continuous interval and is a variation of a histogram that uses kernel smoothing. Unlike histograms, density plots are not affected by the number of bins used (in this case, possible scores on a scale of the LCAS) and allow for comparisons of distributions across groups of unequal sizes. After visualizing the variability in students’ perceptions of collaboration, iteration, and discovery/relevance among students in the same course type, we used linear regression to identify how students’ perceptions of collaboration, iteration, and discovery/relevance influenced their cognitive and emotional ownership. We controlled for whether students were enrolled in the traditional version of the lab course or the CURE version of the course to better understand how students’ varied perceptions of collaboration, iteration, and discovery/relevance affect students in the same type of course. The full models that we tested were model A: cognitive ownership ∼ course type + collaboration + iteration + discovery/relevance; and model B: emotional ownership ∼ course type + collaboration + iteration + discovery/relevance. Additionally, we repeated these analyses, controlling for student demographics including gender, race/ethnicity, college generation status, and whether or not a student had previously participated in undergraduate research.
RESULTS
Students Perceived the Traditional Lab and CURE Versions of the Course Differently
This study was designed so that students in the CURE and students in the traditional lab would engage in similar scientific practices, collaborate with other students in a similar way, and experience similar amounts of iteration by comparing their results with those of other groups. However, the CURE was structured so that students would perceive a greater level of broadly relevant novel discovery. To confirm that students perceived these design features, we compared the extent to which students in the traditional lab and CURE courses perceived that they experienced collaboration, iteration, and discovery/relevance2 (each measured by the scales of the LCAS), as well as the extent to which they perceived they were participating in scientific research. We expected that CURE students would have significantly higher ratings on the LCAS Discovery/Relevance scale and would be more likely to perceive that they participated in scientific research than students in the traditional lab, but we did not expect differences in students’ LCAS Collaboration or Iteration scores.
Our results supported our hypotheses and confirmed that students perceive that the two courses differed in discovery/relevance, but not collaboration or iteration. We found that there were no significant differences between traditional lab student ratings (M = 20.84, SD = 2.73) and CURE student ratings (M = 21.15, SD = 3.04) of collaboration (p = 0.62, Hedges’ g = 0.10; Figure 1A). There was also no significant difference between traditional lab student ratings (M = 20.50, SD = 5.11) and CURE student ratings (M = 21.54, SD = 6.18) of iteration (p = 0.41, Hedges’ g = 0.18; Figure 1B). However, compared with students in the traditional lab course (M = 18.06, SD = 5.09), CURE students had significantly higher ratings on the Discovery/Relevance scale (M = 26.54, SD = 3.18; p < 0.0001, Hedges’ g = 2.18; Figure 1C). See the Supplemental Material for a table of all statistics.
FIGURE 1. Comparison of traditional lab student and CURE student mean scores on the (A) Collaboration, (B) Iteration, and (C) Discovery/Relevance scales of the LCAS. Bars represent 95% confidence intervals; ****, p ≤ 0.0001.
Compared with students in the traditional lab course (M = 6.71, SD = 2.66), students in the CURE (M = 8.57, SD = 1.69) were also more likely to agree with the statement that they had conducted scientific research in their lab course (p < 0.001, Hedges’ g = 0.91; Figure 2). See the Supplemental Material for a table of all statistics.
FIGURE 2. Comparison of traditional lab student and CURE student mean agreement that they conducted scientific research in their immunology lab courses. Students rated their agreement from 10 = strongly agree to 1 = strongly disagree. Bars represent 95% confidence intervals; ***, p ≤ 0.001.
Students were asked to explain their reasoning for their rating of whether they had conducted scientific research in their lab course, and four distinct themes emerged. Students in the CURE, but not the traditional lab, highlighted that their results were novel and broadly relevant. Conversely, students in the traditional lab course, but not the CURE, recognized that their results were confirming what had previously been investigated by scientists and what was well understood by the scientific community. Despite working on research questions that were neither novel nor broadly relevant, on average, students in the traditional lab course still somewhat agreed that they had conducted real research; according to students’ open-ended responses, they perceived that engaging in authentic scientific practices was part of real research. Scientific practices have been previously reported as an important element of authentic research experiences; biology faculty surveyed nationally indicated that engaging in scientific processes (e.g., generating research questions, forming hypotheses, designing experiments, and analyzing data) is a defining experience of an authentic research experience in a lab class (Spell et al., 2014). Additionally, even though students in the CURE were working on a real research project, not all CURE students strongly agreed that they were conducting scientific research. In fact, both CURE and traditional lab students highlighted that they lacked autonomy in the lab; that is, students mentioned that they did not develop their own research question, choose which analyses to do, or decide how to analyze the data, which they perceived to be unlike how scientific research is conducted in faculty-member labs (Table 3). Autonomy, or the opportunity for students to direct and make decisions about their work, has been identified as a potentially important lab design feature that could increase students’ research self-efficacy and the extent to which they invest in their research (Gin et al., 2018). Overall, students’ responses to this question indicated that they are able to identify specific nuances of what makes a real research project and further supports that, compared with students in the traditional lab, students in the CURE more strongly agreed that their work closely resembled the authentic scientific research conducted in faculty-member labs.
| Topic | Description | Traditional lab students (n = 27) % (n) | CURE students (n = 57)a % (n) | Example quote from traditional lab student (extent student agreed that he or she conducted scientific research in lab) | Example quote from CURE student (extent student agreed that he or she conducted scientific research in lab) |
|---|---|---|---|---|---|
| Research question was novel or broadly relevant | Students described working to answer a novel or broadly relevant research question. | 0.0 (0) | 54.4****(31) | NA | “The research that we did in [this course] was directly relevant to the research being done by [the course instructor] and his lab faculty, and the experiments that we did had never been done before. They were done with the intention of discovering something new that can be applied to a broader understanding of immunology and the genetic components governing innate and adaptive immunity.” (rating 10) |
| Research question was not novel or broadly relevant | Students described the research question they were working on as having a known answer. | 63.0 (17) | 0.0****(0) | “I don't believe that this was scientific research because the answer to the questions posed in lab had already been answered many times. Nothing new was discovered from this research and no quality material was added to the scientific community.” (rating 3) | NA |
| Engaged in scientific practices | Students described engaging in scientific processes, including following the scientific method, making hypotheses, designing experiments, following protocols, and analyzing data or interpreting data. | 59.3 (16) | 56.1 (32) | “I believe we do conduct scientific research because [at] any time an individual needs to put on their PPE, follow a protocol and analyze data.… Also, before each lab we are required to ask questions and form hypotheses whether we know the end result or not, which means the ‘scientific method’ is in full swing.” (rating 7) | “I used tools that are commonly used in most research labs. I had come up with a question based on observations or background information found and formed a question and hypothesis based on it. My lab group and I performed an experiment to test the hypothesis and discussed and analyzed this data in a lab report.” (rating 9) |
| Lack of autonomy when engaging in scientific practices | Students described a lack of autonomy when engaging in a specific scientific practice. For example, not developing their own research questions or not setting up their own experiments. | 7.4 (2) | 19.3 (11) | “Yes we created hypotheses and tested them, however it was already planned out for us. We didn't have to design anything.” (rating 4) | “[This immunology lab course] was also different from scientific research because we did not have to decide which experiments to perform.” (rating 8) |
Students in the CURE Developed Higher Cognitive and Emotional Ownership Than Students in the Traditional Lab
Previous research has suggested that students enrolled in CURE courses are predicted to have higher project ownership than students in traditional lab courses (Hanauer and Dolan, 2014). In this study, we were specifically interested in determining to what extent the specific design feature of broadly relevant novel discoveries was sufficient to develop high student project ownership. We found that CURE students reported significantly higher cognitive ownership (M = 40.71, SD = 5.89) than students in the traditional lab course (M = 36.72, SD = 5.24, p = 0.001, Hedges’ g = 0.69; Figure 3A). Similarly, CURE students also reported significantly higher emotional ownership (M = 20.60, SD = 5.19) compared with students in the traditional lab course (M = 17.84, SD = 4.15, p < 0.01, Hedges’ g = 0.56; Figure 3B). See the Supplemental Material for a table of all statistics.
FIGURE 3. Comparison of traditional lab student and CURE student (A) mean cognitive ownership score and (B) mean emotional ownership score. Bars represent 95% confidence intervals; **, p ≤ 0.01; ***, p ≤ 0.001.
Students’ Perceptions of Collaboration and Discovery/Relevance Influenced Their Cognitive and Emotional Ownership
The traditional lab and CURE courses were designed to provide students in each type of course with similar levels of collaboration, iteration, and discovery/relevance. However, we predicted that individual students would experience each construct slightly differently, so there would be natural variation in each construct for each version of the course. To visualize the variability of students’ perceptions of collaboration, iteration, and discovery/relevance within and between the types of courses, we created density plots for each factor (Figure 4). We found that the distributions of traditional lab and CURE students’ perceptions of collaboration were remarkably similar between the two versions of the course, yet individual students’ perceptions of collaboration varied within each of the courses (Figure 4). The distributions of traditional lab and CURE students’ perceptions of iteration were also similar between the two courses, yet individual students’ perceptions of iteration varied within each of the courses (Figure 4). In contrast, students in the traditional lab had perceptions of discovery/relevance that were highly variable, while CURE students on average perceived a higher amount of discovery/relevance, and their perceptions were less variable than students’ perceptions in the traditional lab course (Figure 4).
FIGURE 4. Density plots for each course design feature: collaboration, iteration, and discovery/relevance. The degree of curve overlap, illustrated by the green color of overlapping blue and yellow, indicates how similar the course types were for each element. Broad curves illustrate high variability among student responses, while narrow peaks indicate lower variability.
To understand how the variability in students’ perceptions of these factors relates to project ownership, we used linear regression to test whether students’ perceptions of collaboration, iteration, and discovery/relevance were significantly and positively related to their cognitive and emotional ownership. Figure 5 depicts the results from our analyses. Model A estimates the relationship between collaboration, iteration, discovery/relevance, and students’ cognitive ownership. Model B estimates the relationship between collaboration, iteration, discovery/relevance, and students’ emotional ownership. In both models, solid arrows indicate statistically significant relationships, while dashed paths are not statistically significant. All numerical values are standardized correlation coefficients (β) on a scale of −1 to +1 to allow for comparisons among the influence of design features on students’ cognitive and emotional ownership.
FIGURE 5. Relationships among course design features, collaboration, iteration, and discovery/relevance and students’ cognitive ownership (model A) and emotional ownership (model B). All significant relationships are solid arrows; and nonsignificant relationships are dashed arrows. Collaboration and discovery/relevance significantly and positively predicted students’ cognitive and emotional ownership, while iteration did not significantly predict either type of ownership. Discovery/relevance had the largest effect on both types of ownership compared with the other lab course design features. Altogether, the type of class a student was enrolled in (traditional lab or CURE) and the course design features explained 51% of the variance in students’ cognitive ownership (adjusted R2 = 0.51) and 33% of the variance in students’ emotional ownership (adjusted R2 = 0.33). *p < 0.05, **p < 0.01, ****p < 0.0001.
We found that students’ perceptions of collaboration and discovery/relevance were significantly and positively related to students’ cognitive ownership (Table 4 and Figure 5, model A). Discovery/relevance explained more variation in students’ cognitive ownership than collaboration. Altogether, the model explained just over half of the variance in students’ cognitive ownership. Similarly, collaboration and discovery/relevance were also significantly and positively related to students’ emotional ownership (Table 4 and Figure 5, model B). Discovery/relevance explained the most variation in students’ emotional ownership. This model explained about a third of the variance in students’ emotional ownership. Iteration was not significantly related to either cognitive or emotional ownership. Both models controlled for the type of class a student was enrolled in, either the traditional lab or CURE, which was not significant in either model. We also ran both regression models controlling for student gender, race/ethnicity, college generation status, and whether a student had previously participated in undergraduate research. Our findings did not change with the addition of these student-level characteristics (see the Supplemental Material for the additional analyses).
| Model A: Cognitive ownership | Model B: Emotional ownership | |||||||
|---|---|---|---|---|---|---|---|---|
| Variable | B | SE B | β | p | B | SE B | β | p |
| (Intercept) | 12.04 | 3.19 | <0.0001 | 0.12 | 3.16 | 0.96 | ||
| Course type: CURE (reference: traditional) | −1.74 | 1.38 | −0.14 | 0.21 | −0.94 | 1.36 | −0.09 | 0.49 |
| Collaboration | 0.48 | 0.17 | 0.24 | <0.01 | 0.41 | 0.13 | 0.24 | 0.02 |
| Iteration | 0.15 | 0.09 | 0.15 | 0.102 | 0.09 | 0.09 | 0.11 | 0.32 |
| Discovery/relevance | 0.64 | 0.13 | 0.59 | <0.0001 | 0.41 | 0.13 | 0.45 | <0.01 |
| Adjusted R2 | 0.51 | 0.33 | ||||||
DISCUSSION
This study used a unique study design to compare two versions of the same course that differed only in the design feature of broadly relevant novel discoveries, so that one was a CURE and one was a traditional lab. We found that students in the CURE garnered higher levels of cognitive and emotional project ownership than students in the traditional lab course. We also identified that students’ conceptions of both collaboration and discovery/relevance positively and significantly predicted their cognitive and emotional ownership across both versions of the course.
The Relationship between Course Design Features and Project Ownership
At the time that we started the study, project ownership was thought to be an important outcome of CUREs because it had been suggested to cause students to persevere when facing research-related challenges, which in turn had been hypothesized to increase students’ self-efficacy and motivation (Ward et al., 2002; Laursen et al., 2010; Hanauer et al., 2012; Alkaher and Dolan, 2014; Corwin et al., 2015a). Only recently was project ownership shown to be positively and significantly related to students’ intentions to pursue research-related scientific careers (Corwin et al., 2018b). However, a recent study by Ballen and colleagues (2018) found no significant relationship between discovery/relevance and students’ project ownership. The Ballen study was conducted with nonmajors, did not use the full POS, and its study design and interpretations have been critiqued in the literature (Corwin et al., 2018a). Conversely, Corwin and colleagues (2018b) used the full POS (Hanauer and Dolan, 2014) to explore the impact of discovery/relevance on cognitive and emotional ownership; they identified a significant and positive relationship between discovery/relevance and cognitive ownership, but found that discovery/relevance was not significantly related to students’ emotional ownership (Corwin et al., 2018b). Our study findings more closely align with the findings of Corwin and colleagues (2018b), although we found that discovery/relevance was positively and significantly related to students’ cognitive and emotional ownership.
There are many possible reasons for the differing observations made in the Corwin and colleagues’ (2018b) study and our study regarding the impact of broadly relevant novel discoveries on emotional project ownership. First, the Corwin study surveyed many different types of courses, whereas our survey focused on a specific course, so it is possible that some CUREs lead to emotional ownership, whereas others do not; we recommend additional replication studies with different types of CUREs to establish whether discovery/relevance is broadly related to emotional ownership. One specific hypothesis for why this particular CURE led to emotional ownership is that there was very little failure in this course. When making novel scientific discoveries, students can experience frustration and failure, especially when experimental protocols have not been piloted in previous courses (Gin et al., 2018). We hypothesize that associating frustration or failure with making discoveries could have a negative impact on students’ emotional ownership. The traditional lab course had been taught by the same instructor for 3 years before the first data collection in 2016. Thus, the protocols used in both the traditional lab and the CURE had been well established; much of the failure associated with the troubleshooting of experiments that students initially encountered had been resolved by the time students in this study enrolled in the course. Students in this CURE may have experienced fewer feelings of frustration toward the project and failed less when making discoveries than students enrolled in other CUREs because they were working with such well-established protocols, which may have led to an enhanced sense of emotional ownership.
Additionally, we found that students’ perceptions of collaboration were significantly and positively related to their cognitive and emotional ownership but that iteration was not related. This was particularly interesting, given that Corwin and colleagues (2018b) found that iteration positively impacted both cognitive and emotional ownership more than any other course design feature. Gin and colleagues (2018) highlighted that instructors may troubleshoot anticipated issues in advance of a CURE, which could limit opportunities for meaningful iteration in which students engage in the process of troubleshooting their own work. Because many of the obstacles experienced by students in earlier offerings of this course had been resolved before the data collection in this study, the amount of true iteration that students experienced was likely low, and in fact, students in this study reported levels of iteration on the LCAS that were lower than iteration values reported from a national sample of other CURE courses (Corwin et al., 2015b). However, students in this course did experience some iteration by having the opportunity to compare the results of their groups’ experiments with the results of other groups. This was meant to teach students that they cannot interpret their results in isolation, but that science is iterative and requires experiments to be replicated, a concept that students have known difficulties mastering (Brownell et al., 2013b). Further, only one of the six items on the LCAS Iteration scale measured this type of iteration.
An important distinction about the study presented here compared with the previous studies is that it controlled for the influence of an instructor effect, because the same instructor designed and taught both the traditional lab and the CURE. Neither of the previous studies that have explored the relationship between discovery/relevance and project ownership were able to control for whether courses that integrate broadly relevant novel discoveries might be more likely to be designed or taught by a certain type of instructor who is different from the type of instructor who designs or teaches traditional lab courses. Our quasi-experimental study design with the same instructor teaching both the traditional lab and the CURE allowed us to explore specifically the impact of changing the design feature of broadly relevant novel discoveries while keeping everything else essentially the same.
Students’ Varied Perceptions of Course Design Features
In this study, we identified that students enrolled in the same course can have varying perceptions of course design features. Why might students in the same course have such different perceptions of collaboration, iteration, and discovery/relevance? In both the CURE and the traditional lab course, students worked in groups of four. Their relationships with the other students in their group likely influenced the extent to which they experienced collaboration. For example, some students may work well with their groups, regularly give and receive help from others, and frequently share their ideas with the group, while other students may struggle to get along with others or be reluctant to share their thoughts with the group (Cooper et al., 2017a, 2018a). Additionally, each group conducted its own experiments, and we would expect that each group of students experienced different levels of iteration depending on how often they needed to repeat experiments, or how frequently they compared their data with other students’ data. Students’ varied perceptions of discovery/relevance are more surprising, especially the highly varied perceptions of students in the traditional lab; some students in the traditional lab rated the amount of discovery/relevance high and some rated it low. One factor that could influence students’ varied perceptions about discovery/relevance is whether a student had previously participated in undergraduate research in a faculty-member lab. Students who have participated in undergraduate research are likely to be more accurate in identifying the extent to which their work in the lab course is similar to a real research lab. Importantly, students in the traditional lab who perceive a high amount of discovery/relevance in the traditional lab may benefit from this perception even if they believe that their work is more novel and broadly relevant than it actually is. This presents a question of whether students have to actually make a novel discovery in the lab in order to benefit, or whether it is sufficient for students to merely perceive they are making a unique discovery. This would be an interesting hypothesis to probe in future studies. Additionally, it is important to note that student perceptions may help explain the different findings between the Corwin et al. (2018b) and Ballen et al. (2018) studies, because the Corwin group measured student perceptions of discovery/relevance, whereas experts characterized the amount of discovery/relevance in the paper by the Ballen group.
The Ease of Creating a Biology CURE
There has been a national push for biology faculty either to transition existing lab courses into CUREs, to develop CUREs from scratch, or to implement CUREs designed by someone else (Shortlidge et al., 2017). However, faculty have reported that developing CUREs can take additional time and effort (Shortlidge et al., 2015), and some faculty may be resistant to the idea of developing a CURE because of the perceived time and effort required. Here, we demonstrate that a faculty member was able to transition his traditional lab course into a CURE by exchanging wild-type mice for mutant mice and that this minor change resulted in increased student project ownership. We encourage biology faculty to consider the possibility of using transgenic organisms in lieu of wild-type organisms as a way to generate broadly relevant novel discoveries. This affords students the opportunity to still conduct experiments with well-defined protocols or even classic experiments, but with the advantage of giving students the opportunity to collect novel data and engage in research.
Using CUREs to Explore the Impact of Specific Elements of Undergraduate Research on Student Benefits
While undergraduate research is undoubtedly a high-impact practice that positively affects students (National Research Council, 2003; AAAS, 2011; NASEM, 2015, 2017), to our knowledge, no studies on undergraduate research experiences in faculty-member labs have been able to disentangle the effect of the broadly relevant novel discoveries component of the experience from the effect of other aspects of the research experience, including opportunities for collaboration, iteration, involvement in different types of scientific practices, and mentoring. However, we propose that CUREs may provide a better study system for exploring the importance of students making broadly relevant novel discoveries in research. Although the specific format of CUREs varies, many CUREs operate such that all students use the same experimental protocol to identify something novel, but each lab group produces or analyzes a different set of data that will lead to unknown results (e.g., Jordan et al., 2014; Brownell et al., 2015). There are logistical reasons for having all students work on the same protocol (e.g., prepping materials, creating lab protocols for students to follow so everyone is on the same step); this homogeneity also means that there will likely be less variation in the individual experiences of students in a CURE compared with the individual experiences of students working on different research projects in different faculty-member research labs, which makes CUREs a more controlled environment for education studies. Additionally, there are usually more students who are enrolled in a CURE than the number of students in an individual faculty member’s research lab, which adds statistical power to any analysis. Therefore, CUREs may be a better setting than undergraduate research experiences in faculty-member labs to explore the specific impact of working on projects that yield broadly relevant novel discoveries. Importantly, the results from this study may provide some insight into how important it may be to ensure that undergraduates in faculty research labs understand the novelty and broad relevance of the research project that they are working on. This may be difficult for a student to grasp if he or she is too focused on an assigned “task” as opposed to the larger research project or is participating in the beginning or middle stages of the project, when it is sometimes difficult to conceptualize the ultimate impact of his or her work.
Limitations
This was a quasi-randomized study design in which all students who enrolled in the immunology lab course in 2016 took the traditional lab course version and all students who enrolled in the immunology lab course in 2017 and 2018 took the CURE version. It was not logistically possible to randomize students into the two courses to create a truly randomized study design, which is a limitation of the study (Brownell et al., 2013a). However, we did not find demographic differences among students in the two versions of the course based on gender, race/ethnicity, college generation status, and prior participation in undergraduate research, and we have no reason to think that students who decided to take this course in 2016 would be different from students in 2017 or 2018 in terms of other demographics.
We worked with the instructor of the course to ensure that the traditional and CURE iterations of the immunology lab course were as similar as possible with the exception of students working on broadly relevant novel discoveries in the CURE. We tried to limit any differences between the two versions of the course, but it is possible that there were other differences between the two courses that we were unaware of and that could have influenced students’ cognitive and emotional ownership. However, we did control for course type in our regression models exploring the relationship between collaboration, iteration, discovery/relevance, and cognitive and emotional ownership; the course type was not significant in the model.
The instructor discussed the design of the course with the students at the beginning of the semester; students in the traditional lab knew that their experiments were confirmation experiments and students in the CURE knew that they were working on broadly relevant novel discoveries. The instructor specifically told students in the CURE that their data would be used in a scientific research publication. We interviewed a subset of students from the traditional lab and the CURE to corroborate what the instructor had claimed to say; the majority of students who were interviewed from the CURE remembered the instructor saying that the data could be publishable, and the majority of students who were interviewed from the traditional lab said that they were confirming known results. Our data from the LCAS and the open-ended question about whether they were participating in real research also support the assertion that students in the two versions of the course perceived these differences. We did not audio-record the class sessions, so we cannot be sure exactly what language was used, but besides the differences in the framing of the lab course, the instructor language was intended to be similar between the two courses.
It could also be possible that the specific design of this type of CURE, a more prescriptive CURE in which students worked on predetermined protocols, was relevant to these results, and we caution against making generalizations to CUREs more broadly, particularly CUREs in which students have much greater autonomy.
Finally, it would be interesting to collect other affective measures (e.g., self-efficacy, interest in science) in the future to identify whether students with differing levels of these affective factors may have enhanced project ownership in a CURE.
CONCLUSION
In this study, we compared the experiences of students who were enrolled in two versions of an upper-division immunology lab course: a traditional lab and a CURE. There was only one notable difference between the courses; the traditional lab characterized the immune system of wild-type mice, while the CURE integrated elements of broadly relevant novel discoveries by characterizing the immune system of a mutant strain of mice. Students in the CURE perceived greater discovery/relevance and reported higher cognitive and emotional ownership than traditional lab students. Additionally, students’ perceptions of collaboration and discovery/relevance were significantly and positively related to their cognitive and emotional ownership. This work highlights discovery/relevance as an important component of CURE courses, because it has potential to enhance students’ project ownership, which has implications for the design of lab courses.
FOOTNOTES
1The course had two graduate teaching assistants who changed from year to year. However, the teaching assistants used the same materials to teach, including grading rubrics, and were trained to teach in the same way.
2The LCAS uses the term “Discovery/Relevance” to measure what we refer to as “broadly relevant novel discoveries.”
ACKNOWLEDGMENTS
We thank the students enrolled in the traditional lab and CURE versions of the course for taking the time to complete the surveys. We thank the Biology Education Research lab, especially Logan Gin and Rachel Scott, for feedback on earlier versions of this article and Lisa Corwin and Sarah Eddy for their feedback on the research project and analyses. We thank the teaching assistants involved in the course, as well as the Arizona State University School of Life Sciences for providing support for this course.
