Short-Term Research Experience (SRE) in the Traditional Lab: Qualitative and Quantitative Data on Outcomes

INTRODUCTION

Driven by the economic needs for a science, technology, engineering, and mathematics (STEM)-educated workforce in the United States and the calls for enhanced diversity among STEM researchers, there is increasing pressure on faculty, science departments, and higher education institutions to revise and reform the traditional laboratory course (American Association of University Women [AAUW], 2010; American Association for the Advancement of Science [AAAS], 2011; National Research Council [NRC], 2012; President’s Council of Advisors on Science and Technology [PCAST], 2012). There is growing evidence that the quality of the early undergraduate research lab experience is an important component in the persistence of students from a variety of educational backgrounds, ethnicities, and genders in science majors (Nagda et al., 1998; PCAST, 2012; Graham et al., 2013; Auchincloss et al., 2014; Hanauer et al., 2017; Hernandez et al., 2018). Traditional laboratory courses have focused on helping students develop the research skills and procedures required for later, more advanced courses by providing explicit directions for experiments with predetermined and well-established outcomes (Domin, 1999; Weaver et al., 2008; Auchincloss et al., 2014). This approach of providing a safe, known set of procedures with guaranteed outcomes has a pedagogic logic to it. A particular experimental result is evidence that the students have executed the skills and procedures correctly. The process is not complicated by the possibility of an unexpected outcome, and students generally understand the expectations. But this mainstay of undergraduate science education may not promote persistence in the sciences for a variety of students with different demographics (PCAST, 2012; Graham et al., 2013; Hanauer et al., 2017; Hernandez et al., 2018). Students risk developing a naive view of science that does not address the realities of uncertain outcomes, ambiguity of results, and the presence of failure in science (Bencze and Hodson, 1999; Rahm et al., 2003; NRC, 2005).

In response to the need for keeping a range of different student demographics interested in the sciences and for accurately representing the nature of science, one widely supported solution is the involvement of undergraduate students in authentic research experiences (Hanauer et al., 2006, 2012, 2016, 2017; Hanauer and Dolan, 2014; PCAST, 2012; Graham et al., 2013; Auchincloss et al., 2014; Hernandez et al., 2018). While there has been both discussion and contention over what defines and characterizes an authentic research experience, there has recently been some convergence about what this experience is and is not (Spell et al., 2014). Although engaging in the critical thinking required for the process of science is valuable, this in itself is not sufficient to make students’ work in the laboratory authentic (Auchincloss et al., 2014; Hanauer et al., 2017). As explicated in a series of studies, a primary feature of authentic research is the requirement that the research conducted by the student is of significance beyond the laboratory classroom (Hanauer et al., 2006, 2012, 2016, 2017; Hanauer and Dolan, 2014; Graham et al., 2013; Auchincloss et al., 2014; Brownell and Kloser, 2015; Rowland et al., 2016, Shortlidge et al., 2017). This significance can take different forms, such as generating new data or analyses to address a scientific question or a social/environmental issue, but the common theme is that the student’s research could be used beyond the social, physical, and temporal setting of the laboratory classroom as part of a broader scientific research or social agenda (Hanauer et al., 2006, 2012, 2016, 2017; Hanauer and Dolan, 2014; Graham et al., 2013; Auchincloss et al., 2014). In this sense, an authentic research experience is one in which the research conducted by the student becomes part of a broader effort to advance scientific knowledge, address public policy, or change behaviors.

While the broader contextualization of the student’s efforts is central to the definition of course-based authentic research, previous work suggests that a project does not become an authentic research experience unless the student is engaged with this process (Hanauer et al., 2012; Hanauer and Dolan, 2014). In principle, it would be possible to complete a laboratory course designed as an authentic research experience by following the procedures required but without actually understanding or engaging with the broader scientific questions of the study (Hanauer et al., 2012). In other words, for this to become an authentic research experience for the student actually doing the research, there needs to be a sense of personal connection and engagement with the research and an understanding of the connection of this work to the broader scientific community (Hanauer et al., 2012; Rowland et al., 2016). To address this aspect of the student’s self-positioning in relation to the research he or she is doing, we introduced the concept of student project ownership as an additional feature differentiating traditional lab course work from authentic research experiences (Hanauer et al., 2012; Hanauer and Dolan, 2014). As a criterion for measuring the presence of a research experience, project ownership explains the degree to which a student feels engaged with and connected to the research he or she is doing and the degree to which he or she feels that this research is connected to the broader scientific community. High levels of project ownership are indicative of a student who has the experience of being involved in authentic research (Hanauer et al., 2012, 2017; Hanauer and Dolan, 2014; Dounas-Frazer et al., 2017; Corwin et al., 2018).

The scholarship on providing undergraduate students with research experiences has proposed three central models: 1) the undergraduate research experience (URE), in which an individual student works as a research apprentice in the lab and on the agenda of an established researcher (Seymour et al., 2004; Brownell and Kloser, 2015); 2) the course-based research experience (CRE), in which a whole class of students is involved with a professor’s research agenda (Auchincloss et al., 2014); and 3) the inclusive research and education community (iREC), in which a large-scale, multi-institutional research project situates a CRE within a broad community of researchers (Hanauer et al., 2017). While all these approaches involve an authentic research experience—including results that may be important beyond the classroom, the potential to develop student project ownership, and the experience of thinking and acting as a scientist—there are differences among the approaches. In relation to scale, the iREC provides access to authentic research for thousands of students (Hanauer et al., 2017), any individual CRE to all students in a lab course (60 students in one example; Rowland et al., 2016), and a URE to an individual student (see Seymour et al., 2004). Costs per student for implementation are inversely related to scale, with the URE being the costliest of the three (Auchincloss et al., 2014; Hanauer et al., 2017). Finally, there are differences in the timeline for each: the iREC can last up to a year, consisting of multiple terms of work (Hanauer et al., 2017); the CRE is a course that typically lasts for a single term but can be a year (Auchincloss et al., 2014); and the URE can be from several weeks to multiple years (Seymour et al., 2004).

While broad agreement on the need to reform the traditional laboratory course exists (AAUW, 2010; AAAS, 2011; NRC, 2012; PCAST, 2012), and a series of existing models concerning how to transform an entire course have been proposed (Seymour et al., 2004; Auchincloss et al., 2014; Hanauer et al., 2017; CUREnet, https://serc.carleton.edu/curenet/index.html), several barriers may impede actual implementation. Spell et al. (2014) identified barriers perceived by the faculty and departments, including the lack of time, the lack of facilities and personnel to develop a CRE in some institutions (especially those that do not have an existing research infrastructure), large class sizes, and faculty resistance due to concerns about the coverage of required aspects of the curriculum and ways in which they and their students will be evaluated. Shortlidge et al. (2016) also reported on challenges of CRE development, citing the amount of additional time and work required, financial constraints, and student resistance to a relatively ambiguous and uncertain process. For many institutions that do not have primary research roles, such as 2-year colleges and 4-year teaching schools, the internal development of authentic research courses for students can be too difficult to implement (the Community College Undergraduate Research Initiative, www.ccuri.org, is an exception, however). The findings reported by Spell et al. (2014) indicate that adding a research experience to an existing departmental program can be disruptive and can involve a series of changes on the levels of individual courses, the program, and personnel.

Considering the importance of authentic research experiences for STEM students and the potential difficulties that some institutions may face in developing CREs, the current project explores an additional model of science education designed to provide a quality research experience for undergraduate science students. In the project studied here, a traditional laboratory course was augmented with a monthlong authentic research project that was implemented at multiple institutions. In this model, conducted at both 2- and 4-year schools, the lead institution’s research center collaborates with other institutions by sharing a research agenda. The lead institution develops a set of research protocols and centrally stores and uses the data generated across the entire group. However, unlike the CRE and the iREC, which cover an entire course, the research experience is a module situated within the traditional laboratory class, usually the final month of the course. In this way, it is a hybrid model: students are involved in a traditional laboratory experience for two-thirds of the term, with controlled, predefined experiments, and during the last third of the term, they are exposed to a multi-institutional, authentic research experience. This model, a short-term research experience (SRE), is similar to the CRE model but takes less time. The concept of the SRE presented here builds upon previous pedagogical attempts at integrated mini-research experiences within lab courses. The specific SRE project described here also shares similarities with the iREC model, in that it involves collaboration between multiple institutions, although this may not always be the case, and we can foresee SRE modules implemented at a single institution. The SRE is a planned and scheduled aspect of the course syllabus designed to augment and extend the traditional lab course and provide an authentic research experience.

This SRE approach has several advantages. For those institutions that do not have a developed research infrastructure, it allows students to become involved in a research experience. In the example described here, the research was a collaboration between faculty at a research university and several neighboring community colleges that did not have research labs with URE opportunities for students. The SRE is relatively easy to implement within the current program structure, as it merely changes a portion of an existing class, rather than adding a new course. It should elicit fewer objections from faculty, as it does not involve relinquishing existing beliefs about the importance of structured and safe introductions to scientific skills and procedures. If successful, this approach would allow additional flexibility for schools that would like to include a research experience for students but do not wish to completely overturn existing lab course structures.

The crucial question here is really whether an SRE does indeed provide educational benefits leading to the potential for enhanced student persistence. A series of assessment tools have been developed to explore the values of a research experience. At the forefront of this assessment effort is a recent, psychometrically tested suite of psychosocial tools termed the Persistence in the Sciences (PITS) survey (Hanauer et al., 2016, 2017). Psychosocial variables measure psychological responses in the individual as a result of experiences in the social environment. For example, the PITS survey assesses the degree to which a variable such as Science Identity develops as a result of particular science education experience. This PITS survey has been used to evaluate the relative value of different types of research experiences and offers the current study an approach for looking at the relative value of the SRE following the traditional laboratory course experience (Hanauer et al., 2017).

However, because the SRE is an approach that has not been thoroughly tested, it is prudent to use a mixed-methods design involving qualitative and quantitative data to really explore how students understand and experience the SRE. Hence, beyond the use of the PITS survey, qualitative interview data have been collected and analyzed. The aim here is to see whether the SRE really was different from the traditional laboratory course and determine ways in which this experience might offer benefits beyond the existing traditional lab experience. Specifically, the following research questions were addressed:

1. In what ways do students understand and differentiate between the educational experiences of the traditional lab and the authentic SRE?

2. Are there differences between the authentic SRE and a CRE research experience?

THE SAN DIEGO BIODIVERSITY PROJECT

The specific SRE that was studied in this paper was the San Diego Biodiversity Project (Butler et al., 2014; Henter et al., 2016). This project aligned with current thinking about authentic student research in that the students’ work was part of a broader research effort. Students contributed data to a worldwide effort to document biodiversity. Specifically, students created inventories of invertebrate biodiversity on their campuses or in neighboring ecological reserves; they generated new data on the presence and geographic distribution of hundreds of invertebrate species.

These data are valuable because biodiversity remains poorly understood: despite 250 years of modern taxonomy, most species on Earth have never been named or described. Estimates of the unknown range from 70% of arthropod species (Hamilton et al., 2010; May, 2010) to 86% of all eukaryotes (Mora et al., 2011). This knowledge gap is a huge obstacle for environmental stewardship. After all, how can we conserve what we do not know we have? In particular, better documentation of biodiversity is important for Southern California as the area has been designated a hot spot of global conservation importance (Myers et al., 2000).

Students investigated local biodiversity with a technique known as DNA bar-coding. Just as a bar code is used to identify products in a store, a short standardized region of the genome (a “DNA bar code”) is used to differentiate taxa (Hebert et al., 2003). This region is variable enough to distinguish species in most cases, yet short enough to be sequenced cost-effectively (about $10 per individual organism). An international effort to create a reference library of bar codes is ongoing, with more than half a million species already in the Barcode of Life Database (BOLD, https://v4.boldsystems.org; Ratnasingham and Hebert, 2007). Because it is part of a larger research effort, this project shares similarities with the iREC model.

As described earlier, this SRE was conducted at three 2-year colleges and one 4-year university in Southern California. The 2-year colleges are all designated as Hispanic-serving institutions. All of the participants were enrolled in biology lab or lecture/lab courses. The classes at the different institutions shared many similarities but were not identical. The students were largely biology majors, in their second, third, or fourth year of the major. The lab sections had enrollments from 24 to 36 students, but there were often multiple sections within a course. The courses were either general biology labs required for the major or were one of several upper-division courses that would fulfill major requirements, and in one case, the SRE was conducted in a smaller honors section of a general biology lecture/lab course. The traditional portion of the course included activities such as repeating experiments from the published literature; developing skills required for future lab work, such as pipetting; and activities that illustrated concepts covered in lecture. The students were not informed ahead of time that their courses would involve authentic research, nor were they given a choice of different research projects. All students who participated in the surveys and interviews participated in both aspects of the course—the traditional part of the lab and the SRE.

During the SRE module, students working in groups of two or three collected arthropod specimens from the field. Although each group collected its own specimens, the latitude students were given in collecting protocol differed from course to course. In some courses, students could collect whatever they found; in other classes, students focused on a specific taxonomic group or habitat; and in yet other courses, the collections were aligned with an ecological experiment. Universally, however, students recorded critical data about their specimens, such as collection date, GPS coordinates, and a photograph. In the lab, they extracted and amplified DNA and sent the DNA to a commercial facility for sequencing. Once they received the sequence data from the facility, they compared their data with the reference library to potentially identify the specimens. With their instructor, students could contribute the resulting data to the professional public database (BOLD) via a student portal specifically designed for the college classroom (BOLD University Student Data Portal, https://uni.boldsystems.org/index.php/SDP_Home). Through the University Student Data Portal, students shared all of the class data and could see other class’s data as well, if the instructors within the project chose to do so. Students were given attribution as they were named as “collectors” in the data portal and subsequently in BOLD.

Although every group in a class attempted to bar-code a specimen, the results varied from group to group. Occasionally, the students were the first to submit a particular taxon to the database. There were always some groups that failed to generate any DNA sequence data, however, and this was discussed as a possibility when conducting original research. After some troubleshooting, groups usually shared data when this happened. Classes differed in how they used the data, from simply looking at the taxonomic diversity of the collections to comparing lab techniques, generating phylogenetic trees, examining DNA polymorphisms, or addressing an ecological hypothesis. In all classes students could compare their specimen with all of the data for that taxa on BOLD. All BOLD entries require GPS coordinates, so students could see the geographic distribution of bar-coded collections of their taxa. It was not uncommon for a student’s specimen to be the first record in the region. Similarly, all entries in BOLD require a photograph, so students could see the range of variation in morphology for a taxa. Greater detail about the field, lab, and data-handling techniques can be found in Butler et al. (2014). In sum, the emphasis of the research module was how little is known about biodiversity and how, by documenting invertebrates around their campus, students could contribute original data to the critical scientific effort of addressing this biodiversity knowledge gap.

RESEARCH APPROACH

The main aim of this research project was to evaluate the potential changes in undergraduate student science education as the result of the addition of an SRE module to an existing traditional course. The study was conducted in three stages: 1) the qualitative understanding of student experiences in the traditional lab and in the SRE; 2) the quantitative evaluation of the psychosocial variables characteristic of an authentic research experience; and 3) a quantitative comparison between the SRE group and a random sample of students studying in a CRE. A CRE was defined through its structural features of being a lab course organized around an authentic scientific question, directed by a central researcher, in which students conducted research and the data were used by the researcher and scientific community for research purposes. To explore these issues, we used the following design for data collection and analysis:

1. Qualitative study: After the completion of the research experience, interview data were collected using a semistructured research-experience interview protocol (Appendix A in the Supplemental Material). The interviews were transcribed and analyzed using a team-developed and validated coding system. In accordance with content analysis approaches to interview data, an emergent coding system was developed, tested for reliability of coding, and used for coding. Coded interviews were tabulated and summarized, offering an analysis of the students’ collective understandings of the traditional lab and research experience (Neuendorf, 2017).

2. Quantitative study: Using the PITS survey (Hanauer et al., 2016), longitudinal data were collected at two points: the end of the traditional lab experience (or before the starting point of the research experience) and then at the end of the research experience. Data were analyzed using repeated-measures, one-way analyses of variance (ANOVAs) for six variables of the PITS survey (Project Ownership Content, Project Ownership Emotion, Self-Efficacy, Science Identity, Scientific Community Values, and Networking). This analysis was followed by an analysis of the individual scales that comprise the construct of Project Ownership Content.

3. SRE versus CRE: Using the PITS survey (Hanauer et al., 2016), a random sample of students from the SRE group was compared with a random sample of students from a data set consisting of nine semester-long CRE courses at a research (R1) university not involved in this project. The collection of semester-long research experiences was developed within the context of a single university setting and within the context of a biology department. The CREs covered a range of topics within biology. For comparisons across research experiences and university settings, a propensity score matching approach was used with random student samples.

METHODS

Participants

Two samples of students who had completed both the traditional lab and the SRE participated in this study. Both of these groups were drawn from the overall population of students enrolled in the courses. First, 21 undergraduate students (nine women; 12 men) were interviewed for the qualitative study of SREs. These were students who volunteered after receiving an email invitation sent to all students in the course. Second, 124 students participated in the quantitative study, completing the PITS survey before and after the SRE self-reported demographic characteristics of all groups are in Table 1 (full detail demographic data by class section are presented in Supplemental Table 1). In addition, a comparison group that consisted of a random sample of 61 participants from a data set composed of nine semester-long authentic research courses (CRE) was used for the propensity scores matching analysis. Data were collected in accordance with the institutional review boards of Indiana University of Pennsylvania (Log no. 14-302, obtained by D.I.H.) and the University of California, San Diego (Log no. 161292, obtained by H.H.).

Data-Collection Procedures

Qualitative Study.

The first study was qualitative and involved a semistructured interview protocol. The protocol used for this study consisted of eight central questions in which students were asked to reflect upon their experiences in the traditional lab, and the research experience (SRE), and the comparison between the two. A full version of the interview protocol can be found in Appendix A in the Supplemental Material. The protocol included an equal number of questions directed at the traditional and research portions of the courses, but no attempt was made to equalize the amount of time respondents chose to spend describing either type of experience. Each of the 21 participants was interviewed individually and data were audio-­recorded. Interviews were conducted at the end of the course by H.H., who was not an instructor for any course involved in the study.

Quantitative Study .

Quantitative data were collected using the PITS survey (Hanauer et al., 2016) via the Web-based survey platform Qualtrics. Students completed the survey during class.

Analysis

Qualitative Study.

The qualitative interview data were initially transcribed and then analyzed using a content analysis approach. As an initial stage, all interviews were read by the four members of the educational research team (D.H., J.N., F.L., and A.B.), and a provisional coding system was proposed. The coding system captured statements relating to students’ understandings of their lab experiences in both sections of their course. In accordance with content analysis approaches, the coding system was developed by identifying responses, proposing generalized code categories and specifying the definition of the code. Through a series of readings and rereadings of interview transcripts, a coding system relating to research experiences was developed. At the stage of a full coding system, interrater reliability was assessed. Three raters independently rated responses from six interviews (25%), and an intraclass correlation coefficient was calculated to establish interrater reliability. The results (intraclass correlation coefficient [ICC] = 0.99) show very high levels of reliability for the coding system when used by the three raters. Areas of coding difficulty were discussed, and a final version of the coding system was established. Table 2 presents the full set of codes, their definitions, and examples for each code taken from the current data set. Following establishment of the stability of the coding system, the full set of 21 interviews was reanalyzed and coded individually by two raters for each interview transcription. Results were tabulated in relation to the total frequency of codes used by participants in relation to their traditional lab and research experiences. Once results were tabulated, a Wilcoxon signed-rank test was calculated to explore whether there were systematic differences between the two experiences in terms of the frequency of codes used to describe each experience. This test functions as a paired-difference test for rank-­order data and determines whether two dependent samples are significantly different. This nonparametric test does not assume equal intervals between the frequencies of codes and thus does not take into account the scale of difference in the number of times different code categories were stated, but it does offer information on systematic rank differences of codes between the two groups.

Quantitative Study.

The central aim of this analysis was to consider the potential differences between the research experience at the end of the traditional lab and at the end of the SRE. As such, the quantitative data consist of the PITS survey items collected at two different time points from each participant and indicated that a repeated-measures approach be taken. Because the sample was relatively small and fell below recommended guidelines of a repeated-measures multivariate analysis of variance, a repeated-measures ANOVA for each of the PITS variables was used. Each of the PITS survey variables was considered a within-subjects factor and analyzed using a single-­factor, repeated-measures ANOVA. This approach, which has the same participants in both the traditional lab and SRE sections of the course, neutralizes the need for analysis of potentially confounding variables at the level of participant characteristics. The covariables that are not covered by this approach relate to course section and institution type. These were not analyzed in the current study, primarily because of the size of the sample. Not including these covariates in the analysis would not change the outcomes of the comparison of the traditional lab and SRE experiences in a repeated-measures design.

SRE versus CRE.

As a final analysis, PITS survey variables for the students in the post (SRE) condition were compared with results from students who studied in a full-semester CRE. Because the comparison is between two groups with possible demographic differences, a propensity score matching technique was employed. A random sample of 61 students was elicited from a data set consisting of 162 students studying in nine different CRE courses. Another random sample of 35 students who had completed the post PITS survey was elicited from the SRE group. The propensity score matching technique used here compared the outcome for each student in the SRE group with the average outcomes of students deemed similar using propensity scores from the CRE group. In this analysis, the average treatment effect for the treated (ATET) was calculated for the matched groups on the six PITS survey scores. Participants were matched on the covariates of gender, grade point average (GPA), class level, parents’ educational level, parents’ occupation, and ethnicity.

RESULTS

Qualitative Study

Table 3 presents the code frequencies for interview data organized by total number of responses (across all participants) made in relation to traditional lab and research experiences. The Wilcoxon signed-rank test shows that there is a significant difference in the rank coding of these educational experiences (W = 6, Z = −2.051, p < 0.04, r = 0.403). This result suggests that, when discussing the two modes of the lab course, participants systematically used different frequencies of code indicating a subjective perception that the actual experience was dissimilar and distinguishable. According to this content analysis, participants described each of the educational modes differently.

The most common code used to describe the traditional lab was: “developing foundational knowledge of skills and procedures.” Some characteristic statements include “I got a lot of technical skills when I was working in this section” and “It was just kind of having—developing a toolset that you would need.” Another common code was: “results are clearly defined and tied to classroom outcomes such as tests, assignments and grades and are not important in themselves.” For example, one participant stated, “I felt like it was more like you’re copying the real thing. You’re not actually doing something that contributes outside of yourself.” Students were largely positive about the traditional section of the lab. The codes “providing an exciting introduction to biology” and “providing hands-on experience and greater understanding of lecture material” were common and were exemplified by the statements “I think getting that lab experience with biology was a really cool experience” and “It was a more practical application of lectures,” respectively. The statement “The pressure wasn’t as intense” was typical for the code “lab work involved low levels of pressure.” Overall, the picture that emerges of the participants’ understanding of the traditional lab is that it develops scientific skills and procedures that are mainly relevant for the course itself. This does not involve high levels of anxiety, and students perceived it as a suitable and enjoyable introduction to the field of biology.

In contrast, one of the participants described the research experience by stating, “You feel more of a, I don’t know, a responsibility or something to do it for others, not just for yourself. It’s not personal. It’s a bigger picture you’re part of,” which was categorized by the code “providing accurate results and being careful about procedures is a personal responsibility.” The most common code to describe the research experience was “providing a sense of conducting authentic research.” “It is true research and it is being used” is an example of statement included in this code. “Expression of excitement toward research” was also a common code, exemplified by the statement “It was exciting to see that the research actually worked.” But students expressed the opposite emotion as well: “It’s just nerve-wracking every single time because you don’t know if its going to work or not,” which was included in the code “lab work involved the possibility of failure.” The comment “We’re actually contributing information that may not be so well known already. We are providing contributions to the scientific community” exemplifies the code “personal lab work is important to future research, the scientific community and contributes to society.” Another student stated a similar reaction this way, “When we uploaded to BOLD, that was not just something personal like writing in my own notebook. It was uploading so that others can be able to access that data as well, so it’s kind of a bigger community.” Overall, the participants’ descriptions of the research module suggest that this was an authentic research experience for them. The group reported excitement and engagement at conducting research, took on both the personal responsibilities and anxieties involved in conducting original research, and saw this work as part of a broader scientific activity to which they could contribute. They also found it valuable for their future careers and reported higher levels of interest in STEM and feeling like a scientist.

The activities in both the traditional section and the research section of the course were found to contribute to procedural knowledge and understanding of the lecture material as seen in relatively close frequencies for the code “providing hands-on experience and greater understanding of lecture material.”

Quantitative Study

For further insight into the potential differences between the traditional laboratory and research experiences, a series of repeated-measures ANOVAs were calculated for each of the component variables of the PITS survey. As explained earlier, data were collected at the end of the traditional section of the course and at the end of the research experience. All data were matched across participants, and only those students who had completed surveys at both time points were included in the analysis. Table 4 presents the means and standard deviations for all PITS survey variables after the traditional lab and the research experience. For all variables, the research experience involves a slight increase ranging from 0.04 to 0.22 on a five-point scale over the traditional lab. The biggest difference concerns the Project Ownership Content variable, with a 0.22 increase for the research experience condition. For further evaluation of the differences in the responses to the two experiences, a series of repeated-measures ANOVAs were calculated. The assumptions of normality and sphericity were tested, and no assumption was violated. Table 5 presents the results of this analysis. Only the variable of Project Ownership Content reached significance, F(1,69) = 3.919, p < 0.05. All other variables were nonsignificant.

Because the variable of Project Ownership Content was found to be significantly different between the traditional lab and research experience sections of the course, and to evaluate the validity of the variable of Project Ownership for the traditional lab students, it was decided to further analyze potential differences between the educational experiences in relation to the 10 individual scales that comprise the construct of Project Ownership Content. A series of repeated-measures ANOVAs were calculated for each of these. The assumptions of normality and sphericity were tested, and no assumption was violated. Table 6 presents the descriptive data for the comparison of the two educational experiences across the scales. The research experience had higher ratings than the traditional laboratory across all the items, except for the item, “I had a personal reason for choosing the research project I worked on,” which stayed the same. Table 7 presents the results of the repeated-measures ANOVAs. Significant differences between the educational experiences with increased ratings for the research experience were found for the following four items: “My findings were important for the scientific community,” F(1,69) = 4.99, p < 0.03; “In conducting my research project, I actively sought advice and assistance,” F(1,69) = 4.68, p < 0.03; “My research project was interesting,” F(1,69) = 5.74, p < 0.02; and “My research project was exciting,” F(1,69) = 4.68, p < 0.03. These items should be comprehensible and valid within the context of the traditional laboratory. None of the other rating scales were found to be significantly different for the two educational sections of this course.

SRE versus CRE

As a final analysis and to evaluate the potential value of an SRE in relation to longer-term research experiences, PITS survey variables for the students in the post (SRE) condition were compared with results from students who studied in a full-semester CRE at an R1 research university. Table 1 presents a demographic comparison of the two groups. All participants completed the same version of the PITS survey (Hanauer et al., 2016). Following the elicitation of a random sample, a propensity score matching technique was used to address imbalances in underpinning demographic variables. Groups were matched on the covariates of gender, GPA, class level, parents’ educational level, parents’ occupation, and ethnicity. For evaluation of the quality of the propensity score matching, standardized differences and variance ratios were calculated for the matched groups. The majority of the covariates had standardized differences that were close to 0 (±0.1) and variance ratios that were close to 1 (±0.10). The exception was for the covariates of class level (standardized difference: −0.37; variance ratio: 0.77) and parents’ education (standardized difference: −0.48; variance ratio: 0.71). As such, the groups were matched, except for the covariates of class level and parents’ education. Potentially class level and parents’ education could function as confounding variables in comparative analyses of the two educational groups, and some subsequent analysis was required.

For evaluation of whether these unmatched variables had a significant impact on the PITS variables (and could be confounding variables), a series of linear regressions were conducted with each of the PITS variables as a dependent variable and class level and parents’ education as independent variables. The results of the regression analyses reveal that none of the models were significant, and very minimal levels of variance were explained by the independent variables of class level and parents’ education (Project Ownership Content: R² = 0.04, F(2,92) = 1.66, p = 0.19; Project Ownership Emotion: R² = 0.01, F(2,93) = 0.58, p = 0.56; Self-Efficacy: R² = 0.03, F(2,93) = 1.22, p = 0.29; Science Identity: R² = 0.01, F(2,72) = 0.44, p = 0.65; Scientific Community Values: R² = 0.02, F(2,93) = 0.94, p = 0.39; Networking: R² = 0.05, F(2,93) = 2.6, p = 0.08). Although none of these models was significant, further consideration was given to the individual predictors of class level and parents’ education in relation to each of the PITS survey variables. A consideration of the coefficients in each of the analyses revealed that only parents’ education had a significant impact on the outcome of the Networking variable (β = −0.23, p < 0.03). For all the other PITS survey variables, parents’ education did not have any significant impact. Class level did not have a significant impact on any of the PITS survey measures. Overall, the results of this analysis suggest that class level and parents’ education did not function as confounding variables in the propensity matched groups, even though matching on these variables was not optimal.

Having evaluated the potential confounding of the variables class level and parents’ education, it is possible to address the broader issue of matched-group comparisons. Table 8 presents the results of the propensity score analysis. There are significant differences between the groups on all measures, except for Scientific Community Values. A consideration of the descriptive data shows higher ratings for participants from the CRE than for participants from the SRE group.

DISCUSSION

This study aimed to evaluate the benefit of the SRE, to investigate how students understand this experience, and to measure any developments in psychosocial variables that may occur as a result of participation in this research experience. The outcomes of the study suggest that the addition of the SRE module did indeed change students’ perceptions of their educational experience. The qualitative data, which reflect students’ reporting of their experiences and their understandings of the two sections of the course, present a very different profile for the two educational experiences. The traditional lab was presented by students as a self-enclosed educational experience that can develop knowledge of biology and relevant skills and procedures. The SRE was presented as an authentic research experience in which students were engaged and excited and had a real sense of both participating in science and making a contribution to the scientific community. As expressed in students’ verbal statements, the SRE involved personal investment, responsibility, accuracy, and a relationship with real science. Whereas the traditional laboratory experience was seen as important for future classes, the research experience was seen as tied to a real-world scientific community and important for becoming a true STEM professional. The qualitative data and subsequent inferential analysis suggest that the traditional laboratory and SRE were experienced differently by participants.

While the qualitative data provide insight into the differences in students’ understanding of the two educational components of this course, the quantitative data measure the exact changes between the two sections. In this sense, the quantitative data provide an indication of specifically where and to what degree the addition of an SRE module changed results of the psychosocial variables used to measure research experiences. The results specify that it was only the variable of Project Ownership Content that reached significant levels of difference between the educational experiences. The results suggest that the SRE does increase a sense of project ownership. A detailed analysis of the individual scales of the Project Ownership Content variable pinpoints the specific site of the changes between the two sections. Specifically, students feel increased levels of interest and excitement about doing SRE laboratory research and see this work as important to the broader scientific community. Moreover, while doing this work (and probably as an outcome of the importance of the data for the scientific community), they had enhanced levels of seeking advice and help with their research. These results seem to reinforce the expressed understandings of the research experience found in the interview data.

The final analysis presented in this study compares random samples of students from the SRE with students who studied in a full-semester research experience (CRE). The results show an advantage for the CRE on all the psychosocial variables of the PITS survey, except for Scientific Community Values. There was a potential confounding effect of parents’ education on the variable of Networking that cannot be discounted. However, for the variables of Project Ownership Content, Project Ownership Emotion, Self-Efficacy, and Science Identity, the CRE seems to have measurable advantages over the SRE. This final analysis does suggest that there are potential differences in having a semester-long research experience as compared with a monthlong SRE, with an advantage for the CRE. However, the size of the sample and lack of optimal matching of covariates would suggest that further research is needed to be more confident in the comparison of the SRE and CRE student outcomes.

There are also limitations in the interpretation of the other methods used in this study. Interview data were collected at the end of the course following the SRE and a month after the end of the traditional lab component of the semester. The timing of the interview collection could potentially interact with the statements made about both of the sections of this course. Additionally, the fact that some of the students experienced the research in an honors section of the course may have biased their perceptions. Also, in accordance with the practice of the PITS survey administration (Hanauer et al., 2017), data were collected at the end of the traditional lab and at the end of the SRE, but no baseline data were collected at the beginning of the course. Accordingly, we do not know about the incoming rating levels of participants on the PITS survey and cannot compare later ratings at the end of the traditional lab component or the SRE with initial ratings. Finally, as the sample size was relatively small, no secondary psychometric validation was done on the PITS survey with the specific group of students who are the focus of this study.

Despite these limitations, the strongest evidence in this study consists of the PITS survey data in the repeated-measures comparison, which show that what the SRE contributes to the traditional lab course is an increased sense of project ownership. The development of an increased sense of project ownership has been shown in other studies to be an important component of increased engagement and persistence in the sciences (Hanauer et al., 2016, 2017; Corwin et al., 2018). Thus, the development of project ownership through the inclusion of an SRE has value.

More broadly, the data suggest an interesting differentiation between the two sections of the course, with value assigned to both. The traditional lab was seen to have value in terms of what it was designed for: developing foundational skills and procedures and providing a positive introduction to biology content. As seen in the qualitative data, the SRE at the end of the semester provides students with a sense of participating in authentic research involving enhanced levels of excitement and engagement. They expressed an increased feeling of personal responsibility and the sense of working as scientists as members of a real scientific project.

The SRE we describe here involved multiple institutions. It is important to note that three of the four schools were 2-year colleges. Community colleges account for nearly half of all U.S. undergraduates and more than half of Hispanic and Native American undergraduates (American Association of Community Colleges, 2018); thus, exploring interventions in this setting is vital (Henter et al., 2016). The instructors all found the collaboration among institutions valuable, but this study did not investigate whether the multi-institutional nature of the project was responsible for any of the student results. We hypothesize that an SRE conducted within a single institution would confer the same benefits that we document here.

In sum, the results suggest that the SRE does add value to the science education of the students in these courses and that the addition of this course component can provide an authentic research experience that increases a sense of project ownership. This is not an insignificant achievement for a course of this type and suggests that the SRE may have a role in the improvement of the traditional laboratory course so that it is capable of engaging students and thereby supporting their persistence. The fact that the SRE has increased practical and economic flexibility when compared with CRE models of authentic research education and could be less intrusive and easier to implement than other options may be especially true in the community college context.

The SRE, while having advantages over the traditional lab in terms of project ownership, has lower outcomes than the semester-long CRE, which still seems to offer a richer research experience. But even if improvements over the traditional lab are limited to the enhancement of a student’s sense of project ownership, this would still be a valuable move forward. Importantly, the implementation of a new SRE would seem to be far simpler and face fewer hurdles than the implementation of a new CRE. As the SRE can be situated in an existing course, there is less need to overhaul the whole of the first- and second-year curriculum. Furthermore, because the SRE involves a hybrid model that includes components of the traditional lab, faculty concerns over student ability to acquire basic lab abilities and competencies can be mollified. This should allow the SRE to have increased levels of faculty support for the incorporation of early research experiences in the curriculum. Finally, because the SRE is of shorter duration, costs will presumably be lower than the full CRE, and it should be easier to attain appropriate levels of budgetary support. Overall, the recommendation here is that the SRE is a serious option for enhancing the traditional laboratory for students in a wider range of educational settings and offering the possibility that a broader set of students will have access to an authentic research experience.

ACKNOWLEDGMENTS

REFERENCES