A Course-Based Undergraduate Research Experience (CURE) in Biology: Developing Systems Thinking through Field Experiences in Restoration Ecology

INTRODUCTION

In this study, we posit that integrating design-based research (Scott et al., 2020) and nature-based learning (Jordan and Chawla, 2019) approaches in biology can lead to gains in holistic systemic thinking (Verhoeff et al., 2018), thinking like a scientist, and greater equity for historically unrepresented student groups. While such experiences can be achieved through participation in research apprenticeships and internships, course-based undergraduate research experiences (CUREs) have a greater potential to impact a broader range of students. This paper invites not only the students to broaden their thinking to expand their horizons, but also faculty to broaden their design and implementation of science education. Similar to Wilson et al.’s (2020) approach of using modeling to help students build schemas for understanding and using core biology knowledge, we present a multi-week CURE in introductory biology for majors to develop systems-thinking and foundational research skills that lead to a significant learning experience for students.

Through the successful implementation of such a CURE, faculty are better able to integrate the personal, social, and cognitive domains and to have lasting impact on promoting a more equitable cohort of future scientists (Fink, 2013). Authentic inquiry-based learning experiences offer undergraduate students a preview of the skills, knowledge, and intellectual stimulation of being a scientist and engaging in scientific inquiry. Limited access to apprenticeships and internships often can make these experiences restricted to a small subset of undergraduate students in large-enrollment courses at public universities. Meaningful research experience and exposure to scientific inquiry can lead to higher likelihood of persistence in the major and continuation in science careers after graduation (Armbruster et al., 2009; Laursen, 2019). Integrating CUREs into traditional high-enrollment courses with labs provides an alternative, more inclusive model for research exposure and training for students (Handelsman et al., 2004; Auchincloss et al., 2014).

CURE participation is also helpful for increasing students’ identity and sense of belonging, with associated benefits for students historically underrepresented in science (Marx et al., 2019; O’Brien et al., 2020). Students of color, first-generation students, and those experiencing poverty are particularly vulnerable to high failure and dropout rates (Slater et al., 2006; Freeman et al., 2007). Equity gaps start before college and can persist beyond college into careers unless remedied (Harris et al., 2019). CUREs are an effective mechanism to address equity gaps (Bangera and Brownell, 2014). The increased likelihood of students pursuing postgraduate science, technology, engineering, and mathematics (STEM) degrees and careers and persevering in STEM courses as a result of participating in early research experiences benefits all students (Corwin et al., 2015a). Research suggests that CUREs can overcome inequities inherent in traditional research experiences (Estrada et al., 2016).

MacDonald and colleagues (2021) examined the impact of students’ self-efficacy and future goals through introductory CUREs, finding that student-reported self-efficacy varied by demographics.

In addition to research experience, a key element in preparing successful biologists is training in holistic systems thinking (Verhoeff et al., 2018). Complex biological processes and natural phenomena may be explained, understood, and interpreted through complex and dynamic (biological) systems; this is called “systems thinking” (Evagorou et al., 2009; National Research Council, 2011). According to Verhoeff and et al. (2018), a holistic perspective to systems thinking is that our comprehension of biological phenomena emerges from studying dynamic components of systems. Understanding such components can help to explain and predict natural phenomena. This is especially observed in ecology (Hmelo-Silver et al., 2017). Applying a systems approach and the ability to reason about dynamic changes in ecosystems is critical in this time of climate change as ecological literacy is becoming more and more fundamental (Hmelo-Silver et al., 2007). Building students’ knowledge and ability to reason about the ecosystems where they live will also contribute to their ecological literacy.

Developing systems thinking in students is a core component of the Next Generation Science Standards (NGSS; National Research Council [NRC], 2012) and necessary to prepare biologists to engage with the complex systems-level problems of our time (Woodin et al., 2010). In response to NGSS, pedagogies for teaching systems thinking have been primarily focused on pre-service teachers and K–12 students (Yoon and Hmelo-Silver, 2017). Modeling is another effective tool for helping students make connections within biological systems (Wilson et al., 2020). Systems thinking has often been taught from a mathematical (Kappler et al., 2017) or mechanistic perspective to help facilitate understanding of cellular and organismal biological systems (Liu and Hmelo-Silver, 2009). In ecology, helping students make connections across temporal and spatial scales in complex, macro-level ecosystems can be a challenging pedagogical task (Sommer and Lucken, 2010). In the same vein, our holistic systems-thinking perspective goes beyond reductionist approaches to help students recognize interconnections from individuals to ecosystems and apply core concepts to novel situations (Grace, 2015).

Nature-based education has been demonstrated as an effective pedagogical tool relative to traditional classroom experiences. Nature-based learning programs (also called field-based programs in ecology and the geosciences) promote learning, foster critical-thinking and problem-solving skills, and increase leadership and communication skills (Kuo et al., 2019). In addition, intensive field courses in nature develop self-efficacy in science while increasing educational attainment for historically marginalized students (Lopatto, 2007; Jordt et al., 2017; Beltran et al., 2020). Through immersive experiences, students gain strong personal connections to nature and ecosystems through which they build their ecological systems-thinking abilities (Beltran et al., 2020). However, traditional intensive field-based programs are often restricted to college students who are sufficiently privileged to not work during college or summer breaks, restricting their broader impact. Inclusion of a nature-based research component through a CURE model opens this opportunity to the majority of students.

Nature-based projects on or near campus in introductory biology courses have the potential to combine the benefits of CUREs, design-based research, and nature-based experiences, while also developing holistic systems thinking. Available to all students, CUREs are a means to decrease equity gaps that otherwise exist for the limited number of students who can participate in traditional undergraduate research experiences (Auchincloss et al., 2014). In this paper, we present a 6-week, inquiry- and nature-based experience—the Dune CURE—that tracks the progress of the long-term restoration of coastal dune plant communities near the California State University Monterey Bay (CSUMB) campus at Fort Ord Dunes State Park (the field site). Through this open-ended experience, students gain first-hand proficiency with field research, including adhering to and practicing an established sampling protocol, following field guides to identify plant specimens, collecting and organizing data, formulating and revising an independent research question, analyzing and reporting results, searching and citing the relevant scientific literature, scientific writing, and peer review. Major course themes, including ecological community dynamics, restoration ecology, species interactions, and plant adaptations to environmental stressors, are reinforced and conceptualized through firsthand engagement in nature doing research. The combination of active inquiry-based learning in nature using CURE and design-based principles has the potential for ripple effects that extend far beyond just the immediate impact of any particular course (Figure 1).

In this study, we probe the effectiveness of this nature-based CURE model for student learning, including: 1) Did students increase holistic systemic thinking about ecology and the ability to transfer knowledge to novel situations? 2) Did students achieve concrete skills demonstrating learning the elements of the research process? 3) Did students make affective gains in their attitudes toward themselves as scientists? 4) Did equity for Latinx students increase in the course after the CURE was implemented?

COURSE ORGANIZATION, STUDENT POPULATION, AND INSTRUCTIONAL TEAM

Ecology, Evolution, Biodiversity and Plants is the laboratory component that supports the second semester of the Introductory Biology lecture course for biology and marine science majors. At CSUMB, the course is typically taken at the end of the sophomore year after students have completed the general chemistry series and the cellular, molecular, and animal physiology course in the core biology sequence. Ecology, Evolution, Biodiversity and Plants interweaves a series of learning experiences to develop students as scientists, beginning with information literacy, critical thinking, and evaluating literature early in the term. During the last third of the term, the Dune CURE experience builds on these skills and adds data collection and analysis; formulating, testing, and revising hypotheses; and scientific writing supported by primary literature. Students develop research skills while making connections between biodiversity, evolution, plant biology, and ecology. The course is coordinated by a faculty member and taught by a team of instructors from a pool of tenure-track faculty, lecturers, and graduate student teaching assistants. For most students, this lab constitutes their first nature-based fieldwork and intensive data analysis experience. CSUMB is a Hispanic-serving institution (HSI), and Latinx students, a constituency that is increasing rapidly at our institution, represent 49% of the student population (CSUMB Office of Institutional Assessment and Research [CSUMB IAR], 2020; Figure 2). In addition, the proportion of Latinx students at CSUMB significantly exceeds the proportion of Latinx people within the U.S. population at large (18.2%; U.S. Census Bureau, 2020), and 55% of CSUMB graduates are first-generation college students (CSUMB IAR, 2020). Figure 2 indicates the percentage of Latinx students at CSUMB, the percentage of Latinx students within the California State System, and the percentage of Latinx students in the United States.

Project Structure

The 6-week Dune CURE was originally developed by S.E.W. to engage students in an authentic ecological field research experience by monitoring a long-term dune plant restoration project near CSUMB campus using standardized field data collection procedures. Through this experience, students have collected 20 years of data tracking the effectiveness of plant restoration following lead remediation from U.S. Army activities and comparing long-term ecological change of restored and unrestored field sites in the coastal dune habitat of the Monterey Bay. The 6-week experience included 2 weeks when students worked in the field learning sampling techniques and how to identify major plant species. In Fall 2018, E.S. and C.D.S. elevated this field and lab-based experience to more explicitly communicate and emphasize to students that they were conducting research following a CURE model. In addition, E.S. developed and implemented a transparent assignment design approach to more clearly break down and describe the specific learning outcomes and assessments for both students and instructors. Besides inherent benefits from increased clarity and cohesions for both faculty and students, transparent assignment design has been found to be particularly helpful for reducing equity gaps for minoritized students (Winkelmes et al., 2016). These changes were improved iteratively with informal feedback on their effectiveness. E.S. worked in close collaboration with the team of instructors to support consistent facilitation of the CURE.

The 6-week lab experience consists of: 1) week 1: small groups of three to five students work to investigate project history and resources to develop and present a summary to their classmates of key contextual elements of the entire Dune CURE experience; 2) week 2: field trip to learn plant species using a jigsaw approach (Aronson, 1978) at the site where students will collect field data; and 3) week 3: field trip for students to collect data in pairs using a standardized technique and to begin formulating potential questions individually based on their direct experience in the ecosystem. 4) Between lab weeks 3 and 4, students enter collected data into a standardized spreadsheet available to all students and search, select, and summarize data to answer their own questions as figures or tables, 5) which culminates in students submitting their preliminary research questions and figures. 6) In week 4, faculty and peer-support review of preliminary research question and figure, data analysis and graphing, standards of scientific writing, and literature search for relevant primary research articles related to understanding scientific questions and supporting the discussion section. 7) In week 5, students use an in-class rubric-driven peer review of draft papers in small groups with instructor-led role modeling; and 8) in week 6, students submit their final research papers (Table 1). Findings from student papers are regularly shared with the Fort Ord Dunes State Park resource managers.

This open-ended project presents many possible avenues for student analyses and reporting. Due to a range of possible research questions and approaches as well as varying levels of training and experience with statistics, some of the final report points are allocated by the level of difficulty of analysis or breadth attempted. Because students have collected data at multiple coastal dune sites near campus for nearly two decades, a large data set is available for analysis, leading to a wide variety of possible ecological- or restoration-focused research questions. While students are only responsible for collecting and entering one transect of data (in pairs), each student is able to analyze any combination of data from the site visited as well as other labs sections’ site data and previous years’ data to answer one or more questions.

In addition to collecting and analyzing field data and writing up results, another important element of the Dune CURE is providing support for students as they develop their research questions. Following their submission of field data consisting of accurate plant identification and corresponding percent cover estimates to a collated and shared data set at the end of week 3, each student prepares and submits a preliminary testable research question based on any of the current or previous semesters’ field data and a relevant and original preliminary figure. The research question must include some change over time, comparison of multiple sites, comparisons between restored and unrestored areas, single or multiple taxa, and/or relevant meteorological variables. Students are reminded that “perfection” of the question and figure is not expected, but that they must submit both for credit for the assignment electronically through our learning management system.

In the week 4 lab meeting, students are led through the first step in refining and revising their research questions and figures. Students write their previously submitted research questions and sketch their figures on dry-erase boards (Figure 3). Next, working with the whole group, the instructor moves through each of the questions and figures, inviting the students to present and reveal their thinking processes in support of their research questions and figures. Through a series of questions and in consultation with classmates, the instructor guides each student to consider important revisions, such as ensuring a testable research question, changing the scope of the question, and/or other elements that need more work. By the end of the session, all students have a refined research question and some idea of how to revise their figures to represent the required elements for scientific publication. Class time is available for students to work, supported by the instructor and fellow students, toward improving graphical analyses, reviewing how the data should or can be analyzed, and finding primary sources to cite in the discussion. Additional instruction and support is provided on the structure and form of scientific journals and writing sections of a paper using published journal articles as models. The students are further supported in the revision process through cycles of in-class peer review and consultation with the instructor and science writing tutors before submitting their final research papers for grading.

Theoretical Perspectives

Design-based research involves “learning ecologies” exploring complex systems of interactions through learning theories (Brown, 1992; Pfeffer and Renken, 2016) and includes iterative feedback and revision (Scott et al., 2020). Instructors in Ecology, Evolution, Biodiversity and Plants examined emerging promising practices with evidence of their effectiveness and progressively iterated or revised the strategies and/or tools in real time as necessary (Collins et al., 2004; Coley and Tanner, 2015; Dolan, 2015; Lo et al., 2019). As members of the research team include instructors, and the research questions being investigated produce generalizable results that have the potential to impact teaching broadly, then this constitutes a design-based research approach (Cobb et al., 2003). This research also reflects “action research,” which is another methodology used in education research, when the research questions are self-reflective about how a researcher/instructor can improve his or her own classroom practices (Stringer, 2013).

In each subsequent semester of the CURE module, instructors reflected on the outcomes of the educational interventions from the previous semester. By identifying the features of the instructional strategies that addressed retention and student success, researchers could then revise those aspects that were not helpful to learning and teaching informal STEM (Zagallo et al., 2016). Each iteration led to another research cycle designing, testing, evaluating, and reflecting on refining the instructional strategies in support of student learning, success, and retention (Sandoval, 2014). By using theories of learning, the researchers investigate the complex systems of interactions within the environment, and between instructors and students as “learning ecologies” to support the process of student learning (Brown, 1992; Cobb et al., 2003; Collins et al., 2004; Peffer and Renken, 2016). Critically, the emphasis of the research focuses on characterizing the learning environment rather than controlling the research conditions (Barab and Squire, 2004). Flexibility to modify interventions as identified by Hoadley (2004) assists in collecting the evidence to evaluate and characterize the underlying learning process or mechanism involved in the different learning outcomes, in this student self-efficacy.

Pajares argues that student perceptions of their abilities and confidence in conducting research are a critical link between their acquisition of research skills and knowledge and what they do with these acquired skills, including their aspirations for research careers (2003). This study includes measures of participants’ research skills and attitudes as potential predictors of student research self-efficacy (Wilson et al., 2020) and desire to persist in science as well as an empirical examination of the mediating effects of discovery, iteration, and collaboration. We consider STEM research career aspirations as medium- to long-term outcomes; identity as a scientist as a developing outcome; and research skills as short-term outcomes. Bandura articulated self-efficacy as a construct that strongly determines behavior regarding a specific domain, frequently increases with successful experiences related to that domain, and decreases with failures in related experiences (Bandura, 1986, 1997). Sources for self-efficacy beliefs include personal performance accomplishments/mastery of skills, vicarious learning, social persuasion, and psychological/affective states and reactions (Bandura, 1997).

Social cognitive career theory as described by Lent and colleagues examines the relationship among person, cognitive, environmental, experiential, and behavioral factors that influence career interests, choices, and performance (Lent et al., 1994; Brown & Lent 1996). Social cognitive career theory links student career interests, aspirations, and choices to students’ self-efficacy, outcome expectations, interests, and environmental supports/barriers. In the context of a CURE, social cognitive career theory serves as a lens to explore the acquisition of research skills, student identity as a scientist, and the impact on student aspirations for careers in STEM (Adedokun et al., 2013).

METHODS

We hypothesize that students, as scientists, increase their skills in research, analysis, and scientific writing skills by conducting a nature-based research project in a local ecosystem. To address the research questions, we employed a mixed-methods approach including both qualitative and quantitative methods (Walker and Warfa, 2017). To determine whether students increased in holistic systemic thinking about ecology and the ability to transfer knowledge to novel situations (RQ1), we evaluated pre and post student work from the outset and the end of the course using a rubric. To address whether students acquired concrete skills demonstrating learning the elements of the research process, we administered specific competency-based assessments (RQ2). To determine whether students made affective gains in their attitudes toward themselves as scientists, we implemented a survey composed of constructs from reliable and validated instruments (RQ3). As the data violated assumptions of normality, nonparametric tests were used in these analyses. Finally, to determine whether equity for Latinx students increased in the course after the CURE was implemented, an evaluation of equity gaps based on student grade point average (GPA) in the course was conducted (RQ4).

We also inquired into whether the research experience would increase the students’ ability to apply quantitative reasoning. Additionally, we evaluated whether students could recognize and apply evolutionary and ecological principles presented in other contexts. This CURE included field exposure to a specific local ecological community. By developing their own testable research question, contributing data to a larger effort, and summarizing relevant student-collected temporal and/or spatial data sets, students connected their work to the applied regional context and/or ecological literature in a research paper.

This study is based on 167 undergraduate students enrolled across the 2019 Spring and Fall semesters. The study was conducted with approval from the campus Institutional Review Board. The demographics of our student sample show higher representation of Hispanic/Latinx students (42%) and lower representation of Asian and Pacific Islander (8%), Black (6%), and Native American students (1%). CSUMB is a public midsized university minority-serving institution with average enrollment of 7600 (with approximately 2000 in the College of Science). The demographics of the course closely mirror those of CSUMB as an institution (Figure 2).

The Dune CURE project revisions align with Sandoval’s four “epistemic commitments” of a design-based research project, which informed the goals and implementation of the Dune CURE. The first epistemic commitment requires that design-based research is grounded in theories of learning, informing the design of instruction and improved by iteration (Barab and Squire, 2004). The second epistemic commitment of design-based research aims to produce measurable changes in student learning in classrooms around a particular learning problem (Anderson and Shattuck, 2012; McKenney and Reeves, 2013). The third epistemic commitment of design-based research generates design principles to guide the development and implementation of future instructional tools (Edelson, 2002). The fourth epistemic commitment observes student learning over time to ascertain the effects of how the intervention impacts student learning.

According to Sandoval’s first epistemic commitment (2014), a learning problem advances a theory of learning (McKenney and Reeves, 2013). This research investigated how conceptual frameworks based on fundamental scientific concepts (i.e., systems thinking in biology, evolutionary and ecological concepts, etc.) could help students reason productively about phenomena from a regional ecosystem (National Academies of Sciences, Engineering, and Medicine, 2018). The specific theoretical question was 1) Did students increase systemic thinking about local ecosystems and demonstrate transfer of evolutionary and ecological knowledge to novel situations?

In accord with Sandoval’s second epistemic commitment (2014), researchers investigated whether the student learning from the Dune CURE is directly applicable and impactful to students by asking three additional research questions: (2) Did students achieve concrete skills demonstrating learning the elements of the research process? (3) Did students make affective gains in their attitudes toward themselves as scientists? (4) Did equity for Latinx students increase in the course after the Dune CURE approach was implemented?

In the Discussion, the authors proffer principles that guide the development and implementation of interventions for field-based ecology components of biology courses, Sandoval’s third epistemic commitment (2014). The fourth epistemic commitment from Sandoval is the iteration across two semesters of the course, Spring 2019 and Fall 2019. While the course continued to be offered in Spring 2020, Fall 2020, and Spring 2021, due to COVID-19 and distance learning, significant portions of the Dune CURE were altered, and research to evaluate these changes is ongoing.

The Dune CURE curriculum supported students’ data-interpretation skills, requiring argumentation during which students constructed evidence-based claims from the data provided. Students also worked collaboratively to create figures, identify data patterns, and improve data visualization (Figure 3). These strategies were identified from the researchers’ data analysis of students’ initial research question and drafts of figures followed by data analysis of the final research papers. Using design-based research to characterize how learning occurs in the classroom then addresses mechanistic questions central to advancing biology education research (NRC, 2012; Dolan, 2015; Lo et al., 2019).

Assessment of Direct and Indirect Evidence

The use of multiple assessment methods provides converging evidence of student learning in the Dune CURE module using both direct and indirect evidence. Direct evidence was used to assess question 1, transfer of systems biology knowledge, and 2, skills in scientific analysis and writing. Indirect evidence was used to assess question 3, affective attitudes toward science. Comparisons of student course GPAs were used to determine 4, equitable outcomes. Details of methods for each question are described in the following sections.

RQ1: Transfer of Systems Biology Knowledge

The direct evidence of transfer of systems biology knowledge included two multiple-choice questions assessed pre/post Dune CURE and a final exam constructed-response question focused on plant community interactions in a novel local ecosystem. The pre/post Dune CURE multiple-choice questions assessed interpretation of a graph in an ecological context and evaluating the ecological and evolutionary context of plant restoration. Learning gains for the pre/post Dune CURE multiple-choice questions were compared between the two semesters before implementing the specific CURE elements (n = 187) to after implementation (n = 167). The change in performance on these items (delta) was evaluated with a t test assuming unequal variances but meeting distribution assumptions. Assumptions for the t test include: 1) independence: the observations in one sample are independent of the observations in the other sample; 2) normality: both samples are approximately normally distributed; 3) homogeneity of variances: the data do not meet this assumption; and 4) random sampling: both samples were obtained using a random sampling method (Agresti, 2013).

For the final exam constructed response, students were asked to apply their experience working with the plant community from the Dune CURE to inform their evaluation of the evolutionary context of native plant adaptations when presented with descriptions of a different regional plant community. The final exam constructed responses were scored by three faculty members using a rubric by three faculty members, following the same norming and calibration protocol as specified by Suskie (2018). To compare interrater reliability, we calculated Cohen’s kappa statistic for the final research papers (Landis and Koch, 1977).

RQ2: Skills Doing Scientific Research and Writing

The direct evidence of student learning evaluated the extent to which students performed various research skills based on the outcomes during the 6-week Dune CURE (Table 1). To better understand the progress that students made during the Dune CURE, we evaluated their ability to articulate a specific and testable research question and present a complete, accurate, and relevant figure (including estimates of variation) by comparing their preliminary submissions in week 4 with their submissions in the final paper in a pre/post analysis. Because these items were initially submitted after data collection, but before receiving feedback, peer review, or revision, they represent the students’ native abilities. These pre/post assessments were evaluated using a two-sample t test assuming unequal variances and meeting distribution assumptions.

We further evaluated the final research papers for writing in the discipline based on adherence to the norms of the scientific community, including presenting evidence to support claims, appropriately citing relevant scientific literature, and tying their results to the overall project’s restoration goals and/or the larger ecological context. Student work samples were scored using criteria from the Association of American Colleges and Universities Written Communication VALUE Rubrics (Rhodes, 2010). The constructed responses were evaluated blindly according to a rubric that was first normed and calibrated by three evaluators (Suskie, 2018). Rubric norming aligns rubric raters before the formal rating process of student work, whereas interrater reliability statistics are a check on the ratings after the fact. Interrater reliability has been determined through a count of ratings receiving the same scores divided by the total number of ratings completed. This measure of interrater reliability has been shown to be the most commonly applied when calculated to exact or adjacent agreement (Jonsson and Svingby, 2007). The target for agreement is 100%, but Stemler’s (2004) guidance that agreement between raters should reach at least 70% has been adopted. Interrater reliability was 82%. Of the 167 enrolled students, faculty assessed a randomly selected sample of 90 pre-CURE and 113 post-CURE student work samples.

RQ3: Affective Attitudes toward Science

We queried students in the preclass assessment about their career and postundergraduate goals and whether they considered themselves a “scientist” (n = 167). To better understand the attitudes that students developed during the Dune CURE, we compared their attitudes from the prerequisite course with their attitudes in the Dune CURE. The survey instrument contained constructs from the Laboratory Course Assessment Survey (LCAS; Corwin et al., 2015b), the Undergraduate Research Student Self-Assessment (URSSA; Weston and Laursen, 2015), and Persistence in the Sciences (PITS; Hanauer et al., 2016). Responses to the instrument were collected in the 16th week of each semester. The constructs include Thinking like a Scientist, Personal Gains, Attitudes and Behaviors, Ownership, Self-Efficacy, Networking, Collaboration, and Discovery and Relevance (Table 2).

The third construct, Iteration, from the LCAS survey was added to the instrument later and thus responses do not exist for Fall and Spring semesters in 2019; the omission originally was due to faculty perceptions that for CURE modules lasting only 4 to 6 weeks, there would be insufficient time for iteration given content requirements for courses. Given the findings from Goodwin et al., (2021) of the significance of failure and iteration to identity formation as scientists, this construct is now currently being assessed, and faculty include iteration intentionally in their course design. The survey instrument also solicited information about student aspirations for graduate education and research careers (n = 148 of 167 total). Student response to the survey instrument was voluntary and free from coercion; thus, only 148 out 167 students completed the survey (89%). Instructors received de-identified, aggregated data by semester. The variables included in the model were all constructs from reliable and validated instruments (Table 2).

Outcomes from the Dune CURE were compared with responses from the first semester of Introductory Biology using the Wilcoxon-Mann-Whitney two-sample rank-sum test (Wilcoxon, 1945; Mann and Whitney, 1947). The effect was quantified using the Hodges-Lehmann (HL) estimator, which is consistent with the Wilcoxon test (Hollander and Wolfe, 1999). This estimator (HLΔ) is the median of all possible differences in outcomes between the second semester of Introductory Biology, Ecology, Evolution, Biodiversity and Plants, and the first semester of Introductory Biology, Molecular and Cell Biology and Animal Physiology. A nonparametric 0.95 confidence interval for HLΔ accompanies these estimates, as does ρ, an estimate of the probability.

The results from the Fall 2019 administration of the instrument were also evaluated using the chi-square test (Agresti, 2007; McHugh, 2013), another nonparametric statistical test. The chi-square test for independence or the chi-square test of association is used to discover whether there is a relationship between two variables. For the chi-square test for independence, the data must meet two assumptions. These two assumptions are: the two variables should be measured as ordinal; and the two variables should consist of two or more categorical, independent groups. Example-independent variables that meet this criterion include gender (three groups: males, females, and nonbinary), ethnicity (four groups: Caucasian, African American, Asian, and Native American/Pacific Islander). In this case, we evaluated Latinx versus non-Latinx.

RQ4: Assessing Equitable Outcomes

To evaluate the impact of the CURE on equity in the classroom, we assessed academic equity indices (AEI) as described by Bensimon et al. (2006) and exemplified by Perna et al. (2010) and Hatch et al. (2015). The AEI are numeric measures of proportionality of how a specific subset of students achieve a particular outcome in comparison to their numerical proportion of the larger population. We evaluated AEI for Latinx students achieving a passing score and for Latinx students achieving “A’s” or “B’s” (thereby assessing whether the CURE intervention lead to increased competency overall or elevated performance by Latinx students). AEI for Latinx students passing the course was evaluated using equation (1):

We evaluated AEI for Latinx students achieving “A’s” and “B’s” using equation (2):

The resulting AEI scores were evaluated relative to a score of 1.0: an AEI score of 1.0 represents equity, whereas a score greater or less than 1.0 indicates an equity gap. Scores great than 1.0 are evidence of overrepresentation of Latinx students, and those less than 1.0 underrepresentation of Latinx students. To better understand the impact of the Dune CURE on equity, we averaged the calculated AEI scores for the three semesters before and following implementation of the CURE. The resulting before and after average AEI scores were evaluated statistically with a two-sample t test (Helmert, 1876a,b; Lüroth, 1876; Walpole, 2006).

LIMITATIONS

While instruments with published reliability and validity were used to evaluate R3 and R4, instructor-generated exam questions and test items were used to evaluate R1 and R2. The instructor-generated exam questions were authored by faculty with expertise in systems biology and 20 years and 10 years teaching expertise in this specific course. We include the statistical measures used to analyze these questions and constructs (AERA, 2014). For measuring the transfer of systems biology knowledge, a pre/post exam was offered to 169 undergraduates for two semesters before the CURE and to 167 undergraduates for two semesters, while the CURE was implemented, before changes in instruction due to the COVID-19 pandemic. The assessment was administered as a pretest during the first week of class and as an end-of-semester posttest. The pre data collected for the Dune CURE were from Fall 2017 and Spring 2018. The post data were collected in Fall 2019 and Spring 2019.

To maximize efficiency to administer and score this component of an exam, we elected to use a 10-item multiple-choice format. Items were generated based on prior research, the teaching experience of the authors, and the course curriculum, with two questions directed toward the constraints of the Dune CURE. Statistical properties of item scores were examined to determine which items, if any, were not functioning as intended. The extent to which a test is repeatable and yields consistent scores constitutes reliability (Crocker and Algina, 1986). When students perform consistently across items within a test, the test is said to have item homogeneity (Bardar et al., 2007). Two item characteristics, item difficulty and item discrimination, were considered. Item difficulty, p, is defined as the proportion of students answering that item correctly (Crocker and Algina, 1986). For the two items associated with the Dune CURE, the item difficulties for the pre exam were 0.52 and 0.31, respectively, and for the post exam were 0.79 and 0.85, respectively.

Item discrimination measures how well an item differentiates between students who score relatively high or low on the entire inventory. Estimating how students’ performance on the test can be generalized begins by determining how consistently students performed across items or subsets of items on the given test (Crocker and Algina, 1986). Students averaged a 4.01 on the pre assessment and 7.01 on the post assessment before the Dune CURE. During the semesters with the Dune CURE intervention, students averaged 5.3 on the pre assessment and 7.4 on the post assessment.

Reliability can be estimated using internal consistency methods, such as Cronbach’s alpha for Likert data or Kuder-Richardson KR20 for dichotomous data. Alpha represents the smallest fraction of total score variance that is due to true score variance instead of errors in measurement. The minimum acceptable value for alpha is typically 0.6–0.7 (Litwin, 1995). The KR20 for the entire concept inventory is 0.53. However, we note that the KR20 for the two questions specific to the Dune CURE is 0.11. The pre/post multiple-choice questions form a concept inventory, not a construct. The questions relating to the Dune CURE are independent questions covering an array of topics. One question inquires about restoration ecology and evolution: “What would be the challenges for restoring plant communities based on a smaller gene pool?” The other question is an exponential growth question that assesses quantitative reasoning, but in the context of a restored organism. As there are only two questions pertinent to the Dune CURE, and they are not intended to form a construct, the assessments of internal consistency expectedly produce low results, accurate for items that do not have true internal consistency. The questions in the concept inventory are aligned to and intended to assess university, major, and course learning outcomes.

RESULTS

RQ1: Transfer of Systems Biology Knowledge

Students showed a large increase in knowledge on post-Dune CURE relative to pre-Dune CURE multiple-choice questions involving quantitative reasoning (t₍₁₆₆₎ = −3.144, p = 0.0011) and marginal, but not statistically significant, gains (t₍₁₆₆₎ = −1.126, p = 0.130) in evaluating the ecological and evolutionary context of plant restoration (Figure 4). Based on the Dune CURE final research papers scored by multiple reviewers, we found a 95% agreement and Cohen’s kappa of 0.876, evidence for stronger interrater reliability (Landis and Koch, 1977).

When tasked with interpreting a constructed-response story problem about evolution of plant traits in a local ecosystem similar but not identical to that experienced in the Dune CURE, students excelled at recognizing evolution as the product of a selective process (84%) and explaining evolution as the product of adaptive plant traits or selective environmental factors (Figure 5). Fewer students had a complete understanding including the ability to provide supportive evidence for both adaptive traits and selective environmental factors. The story problem also demonstrated that a large majority of the students were able to generate unique responses that supported their arguments using evidence in the language of the discipline.

Furthermore, when students from the lower-division course, Ecology, Evolution, Diversity, and Plants, participated in the subsequent upper-division research methods course, Microbiology, the entirety of the latter was a CURE. Of the students enrolled, 74% were from Ecology, Evolution, Diversity, and Plants with prior research experience, all of whom passed the research methods course. The remainder, 26%, were upper-division transfer students who did not have the opportunity to take the Ecology, Evolution, Diversity, and Plants course and conduct research through the Dune CURE.

RQ2: Skills for Scientific Analysis and Writing

A significantly higher percentage of students were able to demonstrate or exceed proficiency in creating a testable question in their final research paper after collaborative review and revision relative to their initial submitted research question (t₍₁₄₅₎ = −7.4005, p < 0.0001; Figure 6). Similarly, significantly more students were able to demonstrate or exceed proficiency in presenting an accurate analytical figure analysis that addressed their question after collaborative review and revision with peers and the instructor (t₍₁₉₄₎ = −7.947, p < 0.0001; Figure 6).

Likewise, students showed major overall proficiency in their ability to write like scientists in their final research papers. Students also excelled at presenting their results according to disciplinary standards. They overwhelmingly succeeded (89%) in properly stating the results before presenting figures and labeling and were also largely successful (75%) at referencing the figures in the text as evidence of their analyses. We observed in the Spring 2019 semester that a common error (23%) was the inclusion of interpretation of the results in the results section, which was explicitly discouraged in the rubric. In Fall 2019, this error was observed much less frequently (4%).

Students were mostly successful at relating their results to a broader context and explaining their findings in reference to relevant cited scientific literature. Overall, by the final paper, students achieved or exceeded the level of understanding appropriate for introductory students, achieving the tone and style of a journal paper (Figure 7).

Specifically, 70% wrote using a professional tone intended for fellow scientists; clarity with well-developed ideas; and attention to detail, concise writing, and correct formatting of scientific names (Figure 7). Students were successful (90%) at relating their results to either a specific ecological context, a specific restoration goal at the site, or both (Figure 8).

RQ3: Affective Attitudes toward Science

After completing the Dune CURE, students reported gains in nearly all affective domains measured and interest in pursuing research after completing their undergraduate degrees. There is a marginal trend, although not statistically significant, over one semester (Supplemental Table S1). There are also no statistically significant results to report; see Supplemental Table S2 for results from the chi-square test. Additional results include 50% of students involved in the Dune CURE reporting an increase in their interest or likelihood of pursuing research or graduate school in science. More than 80% of students reported pre-existing interest in graduate school or medical school before the Dune CURE, whereas only 65% of students in other groups reported such pre-existing interests (Figure 9).

RQ4: Assessing Equitable Outcomes

We found a much greater fraction of Latinx students received “A’s” and “B’s” following implementation of the CURE model relative to the three semesters before implementation (t₍₄₎ = −2.43, p = 0.036), while the implementation of the CURE marginally improved passing rates for Latinx students (t₍₄₎ = −1.90, p = 0.064; Figure 10).

DISCUSSION

In this study, we presented and evaluated a novel curriculum for an introductory biology course that combines a nature-based, Dune CURE model that introduces ecology and restoration principles in a local context, while developing scientific research skills (field data collection techniques; data summary, analysis, and interpretation; peer review; and scientific writing, in an outdoor active-learning environment with a real-world context. Student involvement in such high-impact practices has been shown to improve multiple outcomes (Kuh, 2008). Our findings demonstrate that a multi-week field CURE experience is a successful model of student engagement and research in introductory life science education.

This course aims to develop nature literacy and holistic systems-thinking skills through open inquiry. One reason we think the Dune CURE experience is effective is that we have group work and assessments every week. Students cannot hang back, as there are multiple points that require engagement each session as well as outside class. For example, students know they will be informally and publicly quizzed on their plant identification skills at the end of the first field trip. They are then quizzed again in a similar manner near the beginning of the second field trip, before they begin collecting data. Even though we give them several tries to reduce the stakes of the assessments, the peer pressure leads to students taking the learning very seriously from the beginning. Given the academically diverse population at CSUMB, the short time frame for comparing growth, and the lack of paired data that would be preferred for demonstrating significant gains on a per-student basis, it is not too surprising that affective gains cannot be demonstrated at a traditional significant level. While these gains did not prove to be statistically significant at the 0.05 level, these self-reported gains trended in the positive direction (see Supplemental Data, Tables S1 and S2). Even so, given these limitations, multiple affective gains were observed at the p ≤ 0.10 level, suggesting the trends are real. We plan to conduct factor analysis of the survey constructs as part of future assessment work.

One of our key results was the increased engagement of Latinx students in their learning based on higher proportion of “A’s” and “B’s” (Figure 10). The low participation of Black and Brown students in ecology and evolution is a significant challenge for these disciplines (O’Brien et al., 2020). Activities such as the Dune CURE may be one solution to engage a broader diversity of students in ecology, evolution, and environmental science. Beltran et al. (2020) found that intensive field-based learning experiences increased retention, self-efficacy, and graduation rates for underrepresented minority students in ecology and evolutionary biology. However, access to intensive courses has traditionally been limited primarily to middle- to upper-income populations who are predominantly white, have had relatively easy access to open spaces, and can afford the time to take extended field courses. Lower-income, urban, and/or students of color often have had less access and/or opportunities to safely be in open spaces (Kappler et al., 2017). Further study would illuminate the benefits of making nature-based experiences part of introductory courses, such that all students are exposed to learning outside in open spaces.

For many students, the Dune CURE is not only their first experience doing ecology research, but also their first experience in direct contact with nature off a paved path. During the Dune CURE, students are prompted to notice patterns and ask questions, incorporating their new knowledge into their existing frameworks or schemas. These new observation skills, as well as learning about the many species endemic to this area, provide a whole new context for seeing the world. It is as if their eyes were opened to the diversity of life on the dunes, when before it was a wasteland along the highway or on the way to the beach. Through this nature-based Dune CURE, participants are afforded the opportunity to move from an outsider to insider status relative to their local natural environment. For instance, students regularly describe in class discussions how they now notice plant communities as part of their everyday lives and observe patterns and changes in local ecosystems in new ways. There are some types of feedback from students in the form of student personal communications about their experience during their undergraduate years that surface after they graduate. Since the inception of the Dune lab experience, students have routinely described their experiences as transformative for connecting with the place. Such communications are informal and anecdotal, but shed light on how these experiences can be impactful but might take more life experience and maturity to become clear. Future studies could survey alumni to determine whether the Dune CURE did indeed increased place-based awareness and greater holistic thinking. As the COVID-19 pandemic and our changing climate has illustrated (Dobson et al., 2020), it has become increasingly important for all students to be educated about human connections to ecosystems, and our anecdotal evidence based on students’ direct place-based experiences may be one component of that education through further study.

Beyond the increased connection to nature, the observation skills gained and connections that students make during the Dune CURE also improve their holistic systems thinking. The direct experience working within an ecosystem likely increases their ability to see the interconnected biological and physical systems acting in connection with conservation, public policy, and other human interactions. For instance, some students become interested in restoration, the connections with past uses of the land, and habitat loss. These connect directly back to evolutionary and ecological content from the course and to news events in the region. The direct evidence from several measures confirmed that the Dune CURE promotes an increasingly sophisticated understanding of biological systems—as seen when students were asked to explain the evolutionary basis of plant adaptations like those they had experienced while conducting the Dune CURE. In addition, students were able to synthesize their field experience, their prior knowledge, and new information to infer understanding of new systems as seen on summative evaluations. Finally, success at tying together their results to the broader ecological context and the scientific literature also highlights the students’ ability to make meaningful connections between their Dune CURE experience and the broader context of the project.

Another pervasive observation by instructors is how the process of open inquiry provided the opportunity for students to notice patterns while collecting field data and then use their firsthand experience and inspiration to further explore and develop their own research questions. Because students were free to choose their own research questions, their inquiries were not restricted to specifically addressing the overall restoration goals, as long as the questions could be answered with available data. This reflects the underlying nature of the CURE—an avenue for discovery, iteration, and collaboration (Auchincloss et al., 2014). The freedom of choice of research project direction and generation of questions aligns with the goal of engaging diverse students through authentic experiences (Wood and Harris, 2015). One caveat is that this freedom also leads to a broader range of final projects that can be more challenging to grade and document as learning within a narrow educational research context.

Given the academically diverse population at CSUMB, the short time frame for comparing growth, and the lack of paired data that would be preferred for demonstrating significant gains on a per-student basis, it is not too surprising that affective gains cannot be demonstrated at a traditional significant level. Even so, given these limitations, multiple affective gains were observed at the p ≤ 0.10 level, suggesting the trends are real. In consideration of the CSUMB student population and study constraints, we recognize some confounding variables and limitations of our study, but also seek to convey the potential broader benefits of implementing field-based CUREs in providing transformative learning experiences.

The process of thinking like a scientist enables students to capture the results from their inquiry (Coil et al., 2017). During data analysis, most students summarized patterns across spatial and/or temporal scales and demonstrated a relatively sophisticated understanding of variation in their data. Articulating explicitly that the process of science can involve different roles and responsibilities, such as an individual analyzing publicly available data sets versus a single individual collecting data, analyzing those data, and disseminating results and conclusions from their data emerges as an important component for class discussions. As science often answers specific questions that can result in furthering understanding when aggregated, each contribution, regardless of size and significance, uncovers the unknown. Thus, the sharing through networks and long-term collaboration at regional, national, and global scales is essential. Some students employed their applied statistics course knowledge to do more extensive analyses, practicing their higher-level statistical and modeling skills, typically in R (R Core Team, 2020). This opportunity for a deep dive into data analysis on a project to which they contributed is a significant additional learning experience and motivator for some students.

Preparation for the writing portion of the Dune CURE is scaffolded on previous course experiences such as library research, evaluating sources, and identifying and modeling the steps in the peer-review process. Students were quite successful at finding scientific literature that connected their questions to broader academic knowledge (Table 1, Figure 6). Cirino et al. (2017) emphasized the importance of building foundational skills in research to be able effectively communicate science. Writing captures the cognition associated with the learning that goes with the research process, including revising and iterating their understanding. Students’ abilities to think and write like biologists was evident in the Dune CURE research papers (Figure 7). Students showed very high gains in analytical skills and self-efficacy (85%; Figure 8), likely due to being given the freedom to analyze any subset of a large data set, which they both contributed to and drew upon in their personal inquiry process. Students gauge the effects of their actions in research, and their interpretations of these effects help create their efficacy beliefs. Outcomes interpreted as successful serve to raise self-efficacy; those interpreted as failures lower it. The culminating research paper generated by students provides an opportunity to synthesize their experience, knowledge, and skills. This academic achievement impacts their sense of competence. Importantly, the analytical skills developed in this CURE are further developed in subsequent courses, building future professional competence.

While CUREs are noted for their potential to increase experience and exposure to the mental processes, behaviors, and skill set associated with scientific research (Corwin et al., 2014; Corwin et al., 2015a; Brownell et al., 2015), it is likely that this impact may depend on the initial belief systems and self-perceptions of the students. Given that most students came into the Dune CURE experience with an existing goal of pursuing graduate or professional school (Figure 9), our focus in this process is to increase persistence such that students realize their personal and professional goals. Anecdotally, we have heard from multiple students who attend graduate school (in diverse fields from marine science to molecular biology) that they credit the Dune CURE as a formative experience that sparked their interest in scientific research.

CONCLUSION, IMPLICATIONS, AND BROADER IMPACTS

By framing this design-based research approach for the Dune CURE as an iterative, progressive refinement rather than a test of a particular intervention when all other variables are controlled, we recognize that: 1) the field-based Dune CURE experiences are unique at any given time, making it difficult to truly “control” the environment in which an intervention occurs or establish a “control group” that differs only in the features of an intervention; and 2) many aspects of the classroom and lab experience may influence the effectiveness of the CURE intervention. Because the Dune CURE explicitly emphasized ecological phenomena in real-world contexts (Windschitl et al., 2012; Scott et al., 2018), the Dune CURE students practiced using holistic systems thinking. The researchers then studied evidence of student learning using learning progression frameworks to describe empirically derived patterns in student thinking that represent cognitive shifts in the ways students conceive a topic (Gunkel et al., 2012; Scott et al., 2019).

Providing exposure to research experiences early in undergraduate academic pathways often requires creating time and space in the curriculum for field experiences. Faculty perpetually negotiate the trade-offs between depth and breadth in foundational science courses. There are practical limitations, including time, resources, and infrastructure. Coordinating accessibility of the field site(s) and cultivating instructor knowledge improves the experience in the Dune CURE. Even with these additional time and resource demands, the higher rate of receiving “A’s” and “B’s” for Latinx students with the Dune CURE supports the benefits of increased depth in introductory biology experiences. The concepts and goals of this CURE can be applied to a wide variety of introductory experiences, not just those campuses with local access to large open space areas. For example, other field-based CUREs use resources readily available on most campuses, such as squirrels and trees (Dizney et al., 2020). Coordinating accessibility of the field site(s) and cultivating instructor knowledge improves the student experience in the Dune CURE.

Implementing a field-based, open-inquiry CURE in large courses with a diverse teaching team involves inherent challenges that are surmountable. As described by Brownell et al. (2015), there is potential for a “teacher effect” on student performance—a marked effect (positive or negative) emanating from difference in instructors rather than from the CURE curriculum. In the Dune CURE, this effect is a possibility, given the large-enrollment format with several instructors ranging from graduate students to full-time lecturers and tenure-track professors. However, when we evaluated our assessments, we found no significant difference between sections or instructors. It is possible that some of the variation observed in several outcomes could be associated with differences in the instructors and their seemingly minute variances of emphasis in the research project. We strive for continuity of instruction through regular instructor lab meetings, coaching newer instructors, and the manual of the Dune CURE, which covers the module-relevant background information, outcomes, timeline, rubrics, and deliverables. Thus, the structure of the course supports student learning in multi-section, multi-instructor environments.

Competency and skills for data collection and analysis are just a few of the attributes sought by STEM employers. Internal surveys for program improvement of scientific and technical employers in the region have found written communication to be a priority for hires, alongside practical skills. We found substantial improvement in the ability of students to write like scientists through this CURE. In particular, students’ ability to develop and analyze their own questions and to have personal experience with the system and data they collected and clear expectations and repeated in-class activities and reminders over a few weeks appeared to be important factors in their success. Another top attribute employers want to see on résumés is problem-solving skills and the ability to work as part of a team. Ninety-one percent of employer respondents are seeking signs of a candidate’s problem-solving skills, and 86% want proof of a candidate’s ability to work as part of a team (National Association of Colleges and Employers, 2020). Our indirect evidence suggests emerging gains for Dune CURE experience, although the results are not statistically significant. Overall, we found the Dune CURE supports the development of holistic systems-thinking, analytical skills, and writing skills, all of which are essential for the emerging STEM workforce. The recent increase in Latinx students is consistent with the larger demographic patterns observed at our institution, more broadly across California, and at institutions of higher education across the United States. However, the proportion of bachelor’s degrees in STEM remains disproportionately low for Latinx students (National Center for Education Statistics, 2019). The reduction of the performance gap between Latinx and other students is a documented outcome of CUREs, including the Dune CURE (Figure 9). CUREs are important interventions for introductory biology courses to increase exposure and experience in the process of science and research and to promote equity and preparation for graduate school and further research experiences (Kloser et al., 2013; Bangera and Brownell, 2014). Decreasing this gap also resonates with students’ expressed academic and professional goals and achieving these goals.

In conclusion, instituting a nature-based CURE for all biology and allied majors creates a transformative experience wherein students view themselves as active members in the scientific community who are connected to the place they live, increasing equity that can have far-reaching implications (Figure 1). While this CURE model requires several weeks of introductory biology lab time, we have found that it is worth it for the knowledge and skills gained and the equitable learning environment for all students. Consider the following two scenarios: one in which we continue to teach biology as a window, with students passively looking out on the research experiences of others, or as an open door, with all students invited into the field to synthesize their own prior knowledge and new experiences into an understanding of the interconnected systems of biology with the broader world.

ACKNOWLEDGMENTS

REFERENCES