ASCB logo LSE Logo

General Essays and ArticlesFree Access

Relevance of Science, Conceptualization of Scientists, and Contextualized “Failure” as Mediators in the Development of Student Science Identity

    Published Online:https://doi.org/10.1187/cbe.22-04-0074

    Abstract

    The Research Experiences in Marine Science (REMS) Program is a Hawai’i place-based CURE (course-based undergraduate research experience) for late high school and early undergraduate students wherein students conduct independent research that draws upon the history, culture, and ecosystem of their local communities. In addition to providing meaningful access to marine science education and training, REMS addresses a fear of failure expressed by students who view their culture and personal identity as incompatible with undergraduate science pathways. Data about student attitudes toward and conceptualizations of science and scientists were collected through pre- and postprogram open-ended survey items, Draw-a-Scientist Tests, and postprogram interviews. Results suggest the combination of place-based elements and an authentic research experience shifted students’ conceptualization of scientists to a “humanized” construct. The emergence of this theme coincided with students recognizing themselves as scientists, gaining confidence in content understanding and research skills, increasing interest in science as a career pathway, and recognizing how science affects their communities. This study demonstrates how a CURE that emphasizes the cultural relevance of science, an inclusive conceptualization of a “scientist”, and contextualized role of “failure” in science, may contribute to historically marginalized students recognizing themselves as scientists and ultimately persisting in science careers.

    INTRODUCTION

    Environmental Impacts on Student Experiences

    Student experiences in a Science, Technology, Engineering, and Math (STEM) learning environment play a large role in influencing student interest and participation in science (Cheryan et al., 2009; Ramsey et al., 2013). Several models based on social cognitive and identity theories have been proposed that describe factors that strengthen or weaken specific psychological constructs as a student experiences feedback cues in social learning environments, such as the science classroom. For example, Estrada et al. (2011) propose an applied Tripartite Integration Model of Social Influence (TIMSI) that links mentorship and research experience with integration into the STEM community via three mediating factors: science self-efficacy (i.e., one’s belief in their ability to perform tasks to reach an outcome in a science context), science identity (i.e., self-recognition as a person of science), and integration of science values (i.e., how important are scientific values to the student?). Another example is the Social Cognitive Career Theory (SCCT) model (Lent et al., 1994; Byars-Winston et al., 2016) that describes how sources of self-efficacy within a science learning environment influence science self-efficacy, science identity, and outcome expectations (i.e., belief in the likelihood to reach a specific outcome) which in turn contribute to choosing a career in STEM. Each of these models inform how negative experiences resulting from either the student’s action (e.g., failing a test) or presence (e.g., being the only student of a specific race or ethnicity in the classroom) may negatively impact student confidence, sense of belonging, and motivation, and ultimately persistence in academic and professional pathways.

    Theoretical Framework for Examining Environmental Impacts

    SCCT (Lent et al., 1994) provides a suitable framework for exploring which elements of a learning environment affect the student experience, whether positively or negatively. SCCT posits that a learning experience (e.g., a research program), influenced by inputs personal to the student (e.g., personality, knowledge, ethnicity, etc.) or encompassing the broader contextual background (e.g., Hawai‘i marine science classroom), provides elements that contribute to career outcomes (e.g., persisting in STEM career pathway). Within the learning experience, the constructs of (a) performance accomplishments, (b) vicarious learning, (c) social persuasion, and (d) affective arousal affect self-efficacy and outcome expectations. In the context of fear of failure, if the sources of self-efficacy from the learning environment are negative (i.e., the student has a negative learning experience), then students will be less confident in their abilities to attain a specific outcome and thus less motivated to move toward that goal. However, students with high self-efficacy (e.g., “I can test a hypothesis”) and positive outcome expectations (e.g., “I can support a family with a STEM career”) would be more interested in or behave in a way to reach specific career goals (e.g., earning a STEM degree). In addition to self-efficacy, studies also suggest that additional factors, such as perceived value of science and science identity, are important copredictors of persistence in STEM for students from historically excluded groups (Carlone and Johnson, 2007; Chemers et al., 2011; Trujillo and Tanner, 2014; Byars-Winston et al., 2016; Ballen et al., 2017; ). To partially account for the role that science identity plays in student career choices, Byars-Winston et al. (2016) developed a modified SCCT model that incorporates science identity as a predictor of STEM persistence.

    From an Identity-Theory viewpoint, science self-efficacy and science identity are overlapping constructs. Carlone and Johnson (2007) defined three constructs in science identity: performance, competence, and recognition. Subsequent studies examining student science identity have combined “performance” (ability to complete science tasks) and “competence” (belief in ability to complete science tasks) into a single construct (Hazari et al., 2010) that is analogous to “science self-efficacy” (Flowers and Banda, 2016). Other constructs within the previously presented social identity models could also be compared with the additional constructs of student science identity. The “recognition” construct of the identity model (i.e., recognition as a person of science) can be compared with a general “science identity” factor in the TIMSI and SCCT models, and the “interest” construct as proposed by Hazari et al. (2010) (i.e., interest in science,) could be likened to the interest generated from “integration of science values” (TIMSI) or “motivation to pursue science” (SCCT). Thus, models that explore factors contributing to student persistence in STEM, also help explain how student science identity develops within specific learning environments (Chemers et al., 2011; Hosbein and Barbera 2020). Understanding which factors in a science learning experience are most influential to the development and maintenance of student science identity is critical to promoting positive outcome expectations and resilience.

    A Hawai‘i, Place-Based Science Classroom

    Previous work from our lab, which was primarily quantitative in nature, has demonstrated that Hawai‘i students, particularly Native Hawaiian and Pacific Islander students, experience positive shifts in science identity constructs after participation in a place-based, summer research program (Ambrosino and Rivera, 2022). The current study uses several qualitative instruments to identify the elements of the Research Experiences in Marine Science (REMS) Summer Program valued by student participants in the development of their science identity and to explore student narratives for explaining shifts in identity metrics.

    The REMS program is a course-based undergraduate research experience (CURE) for Hawai‘i students transitioning from high school to college STEM programs (Rivera et al., 2022). In modeling ways to evaluate the effects of CUREs on undergraduate participants, Corwin et al. (2015) described student identity formation as potentially correlated with increased tolerance for research obstacles and persistence in STEM. As a place-based CURE that targets Native Hawaiian and Pacific Islander students and contributes to the University of Hawai‘i at Mānoa’s 2025 strategic priority to become a Native Hawaiian place of learning (Mānoa 2025: Our Kuleana to Hawai‘i and the World, 2015), the REMS program may provide unique insights to the development of resilience to failure and science identity for Hawai‘i students in STEM. This knowledge is especially necessitated as Native Hawaiian students are rarely included in studies of undergraduate experiences of students from groups historically excluded in STEM (Allaire, 2018).

    Thus, this exploratory study sought to examine the self-reported themes on relationships between the science identity of Hawai‘i students transitioning to undergraduate STEM programs and their experiences within a place-based CURE. The analysis was guided by four research questions: a) How do Hawai‘i students evaluate a place-based, marine science CURE? b) How do students conceptualize a “person of science”? c) What factors within a culturally relevant CURE influence a student’s conceptualization of a “person of science”?, and d) In what ways does a place-based research experience contribute to Native Hawaiian student resilience in STEM pathways?

    METHODS

    Study Context

    For this study, the instruments utilized were primarily qualitative in nature. Although some quantitative analyses were used to explore differences within and across student groups and to apply statistical tests, the primary goal of the study is to identify and describe thematic constructs important to the students in the development of their science identities. The analysis was influenced by the previously described SCCT and student science identity models as well as the results of a previous study (Ambrosino and Rivera, 2022) that demonstrated significant longitudinal shifts in science identity construct metrics for REMS participants. However, as the current study was an exploratory endeavor, the analysis was flexible and driven by themes elicited from the students.

    This study took place at the Marine Science Research Learning Center of the Hawai‘i Institute of Marine Biology (HIMB) during the 2013–2018 REMS Summer Programs, and in 2019–2020 during the pilot of REMS Excel (REMS XL)–a longer, more project-focused version of REMS. The authors were instructors for each year of the programs and developed the curriculum for each REMS iteration. 2018 REMS and 2019–2020 REMS XL students were directly recruited into this study after University of Hawai‘i at Mānoa Institutional Review Board Approval (Protocol # 2019-00605). Anonymous data from previous years (2013–2017) were also included to provide data from 109 participants.

    Participants

    Participants were primarily female (61%), ranged in age from 15–24 years old, and all were either attending or recent graduates of Hawai‘i high schools or in the early stages of undergraduate programs. More than half of the participants self-reported their ethnicity as Native Hawaiian or Part-Hawaiian (51%; as defined by the University of Hawai‘i System race and ethnicity reporting protocols). The other students identified as Mixed Race (20%), Asian (17%), Caucasian (7%), Pacific Islander (4%), or Hispanic (1%). Admittedly, the participants represent a broad range of ages that cover significant stages in the development of personal and professional identity. However, this study attempts to highlight the voices of students in a place-based, marine science CURE that integrates a tiered mentoring framework across participant groups who are also members of an ethnic demographic often excluded from studies examining student experiences in STEM pathways. By including student responses from all of the REMS student groups, the results of this study, which explore potential shifts in student conceptualizations and influential programmatic elements, can elucidate experiences from a larger portion of a unique research learning community and may contribute to the development of future projects examining Indigenous student persistence in STEM.

    Mirroring the design of a previous study (Ambrosino and Rivera, 2022), participant responses were classed into one of three groups: 1) New Students – data from students who were experiencing their first REMS program; 2) Mentors – data from REMS alumni who returned to the program to participate as near-peer mentors for New Students; and 3) REMS XL – data from REMS XL program participants (all of whom were REMS alumni and in or entering undergraduate programs). These designations were utilized during the collection and initial analysis of participant data because significant between- and among-group differences had been observed in science identity metrics for participants in REMS programming. In the current study, the defining themes that emerged from each instrument were consistently echoed across the student groups. Thus, in reporting the results, individual group trends are noted where applicable, while overarching themes are described with illustrative excerpts from all groups.

    Data for this study were gathered and analyzed by the researchers who were also instructors for the REMS program. Participation in the research portion of the REMS program did not have any bearing on student participation in the normal curriculum, student educational stipends, or course credits when applicable. The data from the focus-group discussion were not associated with any individual participant. Pseudonyms are used where participant comments are reported. For the sake of this study, personal identifying information was not collected and/or stored. Participants signed releases for use of anonymous classroom data and photographs for reporting purposes.

    As the purpose of REMS is to use place-based pedagogy as a platform for students to launch into their own independent research endeavors, the researchers frequently emphasized to the participants the collaborative nature of this project (e.g., students guided the Draw-a-Scientist analysis by suggesting new themes during the focus-group interviews). The participants were also encouraged to continue using the course instructors as resources as they conducted their own research, even after completion of the course.

    Instruments

    The instruments for this study include materials produced through the normal REMS curriculum (pre- and postcourse surveys) as well as instruments that were created to elucidate additional insights to student attitudes and perceptions (student illustrations and focus-group interviews). Brief descriptions of these instruments, as well as logistical information about the administration of the instruments, are listed in Table 1. Completion of the instruments was voluntary, and students could opt out either partially or wholly without consequence to program participation or standing. All students from the 2013–2019 courses participated in the surveys (New Student N = 106; Mentor N = 27; REMS XL Student N = 12). The Draw-a Scientist test (DAST) protocol was administered during the 2017 and 2018 REMS programs, and all students from those cohorts participated (New Students N = 35; Mentors N = 12). The focus-group interviews included a sample of New Students (N = 6) and all near-peer Mentors (N = 5) from the 2018 REMS program, and all the participants of the 2019 REMS XL program (N = 12).

    TABLE 1. Summary of study instruments

    InstrumentYearsParticipants (N)AdministeredGeneral Description
    REMS program survey2013–2019New Students (106)Mentors (34)REMS XL (12)Total (152)Preprogram (First day)Postprogram (Last day)Anonymous, online surveyCombination of Likert-score items and Open-ended items with written responsesIndividually completed
    DAST tool2017–2018New Students (35)Mentors (13)Total (48)Preprogram (First day)Postprogram (Last day)Students given blank paper and 10 minutes to respond to prompt: “Draw a scientist.”Individually completed
    Focus-group interviews2018–2019New Students (6)Mentors (5)REMS XL (12)Total (23)Postprogram (Last day)Semiguided interview, recorded through digital video, and manually transcribedStudents discussed responses to verbal prompts and DAST imagesCompleted socially within participant group types (i.e., New Student group, Mentor group, and REMS XL Student group)

    Note: This table includes a list of each of the instruments used to collect data for this study. The columns include the name of the instrument, the years in which it was administered, the number of students within each participant category who completed the instrument, when the instrument was administered (pre- and postprogram, or evaluative (i.e., postprogram only), and a brief description of the instrument.

    REMS survey.

    On the first and last day of the program, an anonymous online survey was administered to program participants. The REMS survey was modeled after the Student Assessment of Learning Gains (SALG) instrument (Seymour et al., 2000) and utilized the SALG platform. As a SALG-based instrument, the REMS survey is a student-driven assessment of learning experiences and provides students an opportunity to evaluate personal growth and identify factors within a program that influence their development as young scientists. A total of 145 preprogram REMS surveys and 145 postprogram REMS surveys were administered over the 2013–2019 REMS/REMS XL programs (New Student N = 106; Mentor N = 27; REMS XL Student N = 12). The full New Student and Mentor 2018 pre- and postprogram survey instruments are available in Supplemental Material, sections A–D. Results from the quantitative Likert-type questions have been explored to assess shifts in student science identity constructs after participation in the REMS programs (Ambrosino and Rivera, 2022). The survey items used in the current analyses were a subset of open-response questions from the REMS program evaluation survey, were identified as relating to the identity constructs (e.g., Performance/Competence: “What did you learn about marine science in this program that you did not know previously?”; Interest: “How has your interest in science changed as a result of this program?”; Recognition: “Who or what is a scientist?”), and had consistent response rates and question wording through course iterations. The survey items were reviewed by experts in natural science and education research to confirm alignment with relevant constructs and content knowledge under investigation. Item responses were also triangulated with responses to analogous questions from the focus-group interviews to provide contextual validity evidence for the use and interpretation of the survey instrument.

    The open-ended survey item responses were coded via a constant comparative analysis (Glaser and Strauss, 1967; Strauss, 1987; Glaser, 1992) influenced by SCCT and the identity constructs used to frame this study’s design. The researcher utilized focused pattern coding (Saldaña, 2015) to highlight salient codes (e.g., those that appeared most frequently) and map relationships between similar codes (e.g., correlations between the appearance of one code and another). Longitudinal coding (i.e., comparing codes elicited from multiple timepoints) was used to examine differences in code frequency or use between pre- and postprogram responses, as well as to flag differences between student groups. The results of these analyses were then reviewed and discussed with the director of the REMS program (Rivera) until there was a complete agreement on the emergent codes and relationships.

    DAST Protocol.

    On the first and last day of the 2017 and 2018 REMS programs, students participated in an activity based on Chambers’ (1983) “draw-a-scientist” protocol to assess student conceptualizations of scientists (New Students N = 35; Mentors N = 12). The protocol for this activity was simple: while sitting in the classroom, students are given a pencil, a piece of white, 8.5 × 11″ printer paper, and the verbal prompt “Draw a scientist.” Student-drawn images of scientists illustrate the indicators students use to recognize others or themselves as scientists and allow students to express their abstract perceptions in a nonverbal, graphical way.

    The general composition of the images were quantitatively analyzed with a framework adapted from the definitions of stereotypic science indicators as described by Mead and Métraux (1957), Chambers (1983), and Christidou et al. (2016). Images were scored using three groups of coded scientist indicators – appearance, surrounding, and activity indicators (Table 2). Differences between average total and stereotypical indicators drawn per image on the first versus the final day of the REMS program were analyzed with a paired Student’s t test. McNemar’s test of marginal homogeneity was used to analyze differences in frequency of specific indicators, such as the gender of the scientist drawn.

    TABLE 2. DAST indicator groups, codes, and subcodes

    Indicator GroupCodeSubcode
    AppearanceGenderMale*/Female/Not Indicated
    Number of scientistsClothingPPE (except lab coat and glasses)One/multipleLab coat*/everyday clothesLab gear*/field gear
    Eyeglasses*N/A
    Facial hair*N/A
    Mythic (e.g., Einstein hair)*N/A
    SurroundingsLocationResearch Instruments*Indoors* / outdoors / bothN/A
    Knowledge symbolsN/A
    TechnologyN/A
    ActivityManual analytical tasksN/A

    Note: Asterisks denote indicators stereotypical of the Western, Euro-centric scientist, as first described by Mead and Métraux (1957). Stereotypical indicator list adapted from the lists utilized by Mead and Métraux (1957), Chambers (1983), Schinske et al. (2015), and Christidou et al. (2016).

    Focus-Group Interviews.

    The focus group interviews were conducted on the final day of the 2018 REMS program (July 2018), after completion of the research symposium, and the final day of the 2019–2020 REMS XL program (February 2020). A total of 11 participants from 2018 REMS program (six New Students and five Mentors) and 10 participants from REMS XL took part in the focus group interviews (see Table 3). The REMS New-Student group included students entering their sophomore through senior year of high school and one high-school graduate, whereas the REMS XL group was comprised of nine undergraduate students and one high-school graduate who was entering an undergraduate program.

    TABLE 3. Focus-group participant demographics

    GroupPseudonymEthnicityGrade level
    2018 REMS New StudentsLewa (F)Shanea (M)Daniela (F)Nalu (M)Mai (F)Makana (F)Mixed RaceCaucasianHispanicNative HawaiianNative HawaiianNative HawaiianSophomoreSeniorSeniorHigh-school graduateSeniorJunior
    2018 REMSMentorsJusten (M)Tiare (F)Rosie (F)Liam (M)Misa (F)CaucasianNative HawaiianNative HawaiianMixed RaceMixed RaceUndergraduateUndergraduateHigh-school graduateSeniorJunior
    2019-2020 REMS XLTalia (F)Shanea (M)Sarina (F)Chris (M)Mason (M)Cora (F)Celia (F)Yoko (F)Kaylee (F)Bryce (M)AsianCaucasianNative HawaiianMixed RaceNative HawaiianMixed AsianAsianMixed AsianNative HawaiianAsianUndergraduateHigh-school graduateUndergraduateUndergraduateUndergraduateUndergraduateUndergraduateUndergraduateUndergraduateUndergraduate

    Note: Gender (F – Female, M – Male) indicated in parentheses beside each student pseudonym is self-reported answer to open-ended prompt. Racial categories reflect categories utilized as per University of Hawai‘i race and ethnicity reporting protocols: http://manoa.hawaii.edu/miro/wp-content/uploads/2014/07/Race-Ethnicity-Student-and-Faculty.pdf.

    aShane participated as a New Student in 2018, and as a REMS XL student in 2020.

    The group interviews lasted approximately 30–45 min and were moderated by C.M.A. using a guided-interview format (Krueger, 2002; Lichtman, 2013) with a mixture of verbal questions (which focused on the student experience in REMS and student attitudes toward and conceptualizations of science and scientists) and select student artifacts (pre- and postprogram Draw-a-Scientist illustrations) to prompt discussion. The guiding questions included:

    • Are you satisfied with the completion of your project?

    • What difficulties did you face during your project? In execution? In analysis? In dissemination?

    • Was there a lesson module you found most helpful? Least helpful?

    • Has this course helped in your confidence as researchers? Do you feel more experienced?

    • Who or what is a scientist?

    • How did your previous knowledge integrate with what you learned?

    • Is your experience or new knowledge applicable to your life outside of the program or in your community?

    • Has this course changed your perceptions of science in general? Of science as a career path?

    • Do you think environmental stewardship is important? Why or why not?

    • What might make this experience more appealing to other students?

    • Is there anything we haven’t covered that you’d like to mention or discuss?

    During the interview, C.M.A. took notes on a laptop regarding any statements made by the participants in response to the planned questions or spontaneous questions posed by the focus group itself. New Students and Mentors in the 2018 REMS program were also asked to review images produced during the DAST that had been administered on the first and last day of the program. The participants were handed five preprogram images, then five postprogram images and asked which details or patterns in the images described the concept of a scientist to them.

    Interviews were recorded with digital video cameras and were all conducted with the researcher face-to-face in the classroom facility with an exception of three REMS XL students who participated via Zoom video conferencing as they were attending universities out of state. Raw video files were encrypted and saved on a password-protected computer stored in a locked room and deleted upon completion of the analysis. Verbal responses from the interview were transcribed, assigned pseudonyms (see Table 4), and coded using constant comparative analysis to elicit themes from the participants framed within the student science identity model. This analysis was used to illustrate how students view their own science identity and to highlight the program factors that students value most.

    TABLE 4. Excerpt from 2018 REMS New Student interview transcript

    The excerpt below followed the prompt: Can you apply your new knowledge and skills beyond the classroom?
    Line #SpeakerTranscript
    081LewaMm. I definitely learned stuff. Like, just like, I dunno, managing my time. ‘Cause it’s like… another school…kinda, but it’s like better than school. [General giggles]
    082DanielaIt's fun! I don’t, like, hate coming here. I don’t get like super stressed where I wanna like just crawl in a ball and like [gestures]
    083Researcher (C.M.A.)Aww!
    084ShaneIt's like, If – I feel like what we learned here is basically like school but everything mushed into like one thing that you actually enjoy. Like reading, mathematics, sciences, all into one and it revolves around the ocean. And it's like not much more better than that.
    085DanielaYeah, even history too-
    086ShaneYeah, history too.
    087Daniela‘Cause I kinda learned a lot about Hawaiian history which I, like, don’t know a lot about it. Heh.
    088LewaYeah, um, I think also because all of the people here are kinda interested in the same thing, it helps us all to, like, really click and stuff.
    089NaluMmhmm. You’re not by yourself.
    090LewaYeah. It's really nice, like, to be around other people who really like science, like how I do. It's like, yeah, there’s some people who are like “yeah, science is cool, whatever”. Like, the labs are fun, and we dissected a pig. But like actual people who are really interested-
    091DanielaReally passionate.
    092LewaYeah. Like, I never thought that I'd be, like, into marine science. Like, I was always, like, well personally it was, like, into space. And, like, oh, space is so cool. And now I'm like [giggles] I’m starting to find, like…
    093ResearcherUh oh, we’re bringing you back down to earth, here?
    094LewaYeah. I-I dunno. I find, like, there's similarities.
    095ResearcherMmhmm.
    096LewaThere's a lot of unknown in both areas, like, discovering new things.

    Note. This excerpt is an example of the transcript produced by the researcher before coding. Speech was transcribed verbatim, with jargon, slang, and filler words included.

    Limitations

    As this was primarily a qualitative study and the student sample size was relatively small, the results might not be broadly generalizable to all classroom settings. Although place-based curricula may be adapted to reflect any population or locale, this study represents a specific instructional and cultural context that may not reflect analogous experiences in a continental US or international classroom. However, we emphasize that the purpose is to highlight the voices and experiences of students from groups historically marginalized in science, and thus our conclusions contribute to enrich our understanding of the challenges faced by students navigating undergraduate STEM pathways.

    Other potential limitations to be considered in the interpretation of this study’s conclusions are the variations in instructional staff and development of program content (which in turn impacts the content of the research instruments administered) within each iteration of the REMS program. Although a core team of five instructional and administrative staff remained consistent throughout the course offerings, each summer included an additional set of three to eight staff and a broad network of collaborators including graduate students, postdocs, professors, scientists, cultural practitioners, and resource managers to help instruct and support student research projects. While focusing on marine science, specifically tropical coral-reef organisms and systems, the educational content of the REMS programs has evolved over the years as new modules are developed and our understanding of curriculum theory is enhanced. We have not quantitatively analyzed pedagogical and subject matter shifts between program years, but anecdotally we have attempted to incorporate more place-based elements in the program each year. For example, the 2018 REMS program included more references to Hawaiian lore, more Hawaiian place and animal names, and more immersion in community field site exchanges than the pilot program in 2013. The pooling of data for each student group (i.e., New Student, Mentor, and REMS XL) as used in this study may help to mitigate any potential between year differences between course experiences.

    RESULTS

    The analysis identified 86 codes from the written survey responses, student images, and focus group interviews that were broadly categorized into three themes that reflect the research questions:

    • Research learning environment enhanced with relevant science. (RQ 1: How do Hawai‘i students evaluate a place-based, marine science CURE? This theme included comments around programmatic sources of science self-efficacy valued by the students.)

    • Student science identity and the conceptualization of scientists. (RQ 2: How do students conceptualize a “person of science”? and RQ 3: What factors within a culturally-relevant CURE influence a student’s conceptualization of a “person of science”? Students described how the conceptualization of a “humanized” scientist developed and influenced how they viewed themselves as part of the science community.)

    • Outcome expectations and overcoming the fear of failure. (RQ 4: In what ways does a place-based research experience contribute to Native Hawaiian student resilience in STEM pathways? Challenges faced by the students, and professional scientist responses to those challenges, helped them develop resilience for overcoming difficulties in their academic and personal pathways.)

    The following sections present the results of these analyses, broadly organized into these three thematic domains.

    Research Learning Environment Enhanced with Relevant Science

    To address the research question on student evaluations of a place-based CURE, student responses describing the effects and perceived value of the sources of science self-efficacy (i.e., mastery experiences, vicarious learning, verbal persuasion, and affective arousal) and situational interest provided by the REMS program are examined in the following sections.

    Marine Science Self-Efficacy.

    Two open-ended survey items asked students about knowledge and skills they hoped to gain or had gained through participation in the program. Postprogram responses (see Table 5) indicated students from all groups reported an increase in knowledge (e.g., “I learned a great deal a lot of new things and more things that I thought I knew but now I know a lot more.”). New Students most frequently reported an increase in their understanding of the relevance of scientific knowledge to their lives (32.1%; “Marine Science is really important to this world and mostly to this island because we’re in the middle of the ocean.”).

    TABLE 5. Reported Marine Science knowledge self-efficacy

    Preprogram survey prompt: “What will you learn about marine science and research through this program that you do not know now?”
    Percent of total student group comments
    CodeSubcodeNew Student (N)Mentor (N)REMS XL (N)
    General Marine Science content70.8% (75)37.0% (10)33.3% (4)
    Module contentaOcean acidification2.8% (3)3.7% (1)N/A
    Bioacoustics1.9% (2)0.0% (0)N/A
    Coral reef ecology0.9% (1)0.0% (0)N/A
    Biodiversity0.0% (0)0.0% (0)N/A
    Animal behavior0.0% (0)11.1% (3)N/A
    Project contentN/AN/AN/A
    Relevance of knowledge27.4% (29)25.9% (7)25.0% (3)
    Research as knowledge8.5% (9)14.8% (4)91.7% (11)
    Postprogram survey prompt: “What did you learn about marine science and research through this program that you did not know previously?”
    Percent of total student group comments
    CodeSubcodeNew Student (N)Mentor (N)REMS XL (N)
    General Marine Science content
    0.0% (0)3.7% (1)0.0% (0)
    Module contentaOcean acidification15.1% (16)3.7% (1)N/A
    Bioacoustics8.5% (9)0.0% (0)N/A
    Coral reef ecology7.5% (8)3.7% (1)N/A
    Biodiversity1.9% (2)7.4% (2)N/A
    Animal behavior0.9% (1)0.0% (0)N/A
    Project content9.4% (10)29.6% (8)22.2% (2)
    Relevance of knowledge32.1% (34)22.2% (6)0.0% (0)
    Research as knowledge7.5% (8)3.7% (1)66.7% (6)

    Note. Matrix of code and subcode frequency in student (New Student [N = 106], Mentor [N = 27], REMS XL [N = 12]) comments. Codes are not exclusive.

    aThese modules were part of the REMS curriculum.

    Several students from the focus group interviews described how an increase in knowledge nurtured their confidence. Shane, a new student in the 2018 REMS cohort, began the discussion by describing a feeling of accomplishment in knowing more at the end of the program:

    I mean, the program definitely did give me a lot of information that I didn’t really have beforehand. And so I’m kinda happy that I have that now cause it’s, like, add that to your book and, like, look – look what I can do.

    In response to the question item about new skills (see Table 6), before the program all student groups frequently anticipated greater mastery in general research skills (New Students: 32.1%; Mentors: 25.9%; REMS XL: 50%) and many hoped they would increase public speaking or presentation skills (New Students: 20.8%; Mentors: 22.2%; REMS XL: 50%). After the program, students reported increased confidence in many science skills such as: public speaking, technical writing, critical thinking, and adaptability and open-mindedness (e.g., “You have to be flexible and work with all types of people. Also, you need to prepare to be wrong about what you think you know.”). Two codes that emerged from the postprogram data that were not present in the preprogram data were “Resilience” and “Relevance of Skills.” In postprogram responses, 7.5% of New Students discussed increased resilience to difficulties in science, and 6.7% reported a better understanding of the relevance of science skills to their everyday lives.

    TABLE 6. Reported Marine Science skills self-efficacy

    Preprogram survey prompt: “What skills besides the ones already mentioned will you gain through this program?”
    Percent of total student group comments
    CodeSubcodeNew StudentMentorREMS XL
    Research as skill32.1% (34)25.9% (7)50% (6)
    Presentation Skills20.8% (22)22.2% (6)50% (6)
    Technical writing8.5% (9)11.1% (3)0.0% (0)
    Interpersonal skills2.8% (3)0.0% (0)0.0% (0)
    Teamwork6.6% (7)11.1% (3)16.7% (2)
    Communication6.6% (7)14.8% (4)0.0% (0)
    Leadership0.0% (0)7.4% (2)0.0% (0)
    Resilience0.0% (0)0.0% (0)0.0% (0)
    Relevance of skills0.0% (0)0.0% (0)0.0% (0)
    Postprogram survey prompt: “What skills besides the ones already mentioned have you gained through this program?”
    Percent of total student group comments
    CodeSubcodeNew StudentMentorREMS XL
    Research as skill17.0% (18)18.5% (5)33.3% (3)
    Presentation skills20.8% (22)11.1% (3)33.3% (3)
    Technical writing7.5% (8)3.7% (1)11.1% (1)
    Interpersonal skills10.4% (11)22.2% (6)0.0% (0)
    Teamwork7.5% (8)14.8% (4)0.0% (0)
    Communication9.4% (10)0.0% (0)22.2% (2)
    Leadership0.0% (0)18.5% (5)0.0% (0)
    Resilience7.5% (8)3.7% (1)11.1% (1)
    Relevance of skills6.7% (7)0.0% (0)11.1% (1)

    Note. Matrix of code and subcode frequency in student (New Student [N = 106], Mentor [N = 27], REMS XL [N = 12]) comments. Codes are not exclusive.

    The students from all groups emphasized the importance of being able to apply the knowledge and skills that they gained throughout the course. By practicing their skills, utilizing their knowledge, and handling instruments in the laboratory and the field, the students felt they were better able to retain their newly acquired knowledge, as demonstrated by this exchange between Daniela and Shane in response to the researcher asking about how the REMS program differed from their typical science class:

    Daniela: [Science is] More than just read a passage and annotate it.

    Shane: It’s a, it’s a lot more hands-on, and I feel like that’s a way more better learning than books, because you read it and it goes in one ear and out the other. But when you actually do it, it teaches you a lesson, because you actually get the feel for what it happens and how it works.

    Participating in science as a practice resonated with the students, and “just be[ing] able to do stuff [they] wouldn’t have had the opportunity to do otherwise” contributed to fostering self-efficacy through the authentic research experience.

    Interest in Marine Science Pathway.

    Responses to the open-ended questions highlighted an increased interest in marine science (“From a scale from one to 10 I’d say a 12.”). Students also reported their interest in marine science was influenced by an increased awareness of their relationship with science beyond the classroom (“As a result of this program, I’ve become more interested in science as I have learned how interconnected everything is.”). All student groups reported high interest in marine science at the start of the program (New Student: 55.0%; Mentor: 79.2%; REMS XL: 75.0%; Figure 1). At the end of the program, an increase in interest was reported by 68.3% of New Students and 64.0% of Mentors. REMS XL responses indicated a continued interest. Although REMS XL students did not report an increased interest in Marine Science after the program, they did report an increase in the intention to pursue STEM pathways (66.7% postprogram compared with 16.7% preprogram; Figure 2).

    FIGURE 1.

    FIGURE 1. Student interest in Marine Science and change in interest. Note. Student responses (New Students [N = 106], Mentors [N = 27], REMS XL [N = 12]) indicating (a) “highly interested” in Marine Science preprogram as a percentage of total student responses (white bars) and (b) percentage of students who mentioned an increase in interest after the program (black bars). No students reported a decline in interest.

    FIGURE 2.

    FIGURE 2. Student intent to pursue STEM pathway. Note. Student responses (New Students [N = 106], Mentors [N = 27], REMS XL [N = 12]) to REMS Survey prompt “Please comment on your current level of interest in science” mentioning intent to pursue STEM pathway before and after participation in the program.

    Lewa, a New Student in REMS, discussed how little discoveries in familiar environments affected her interest in science:

    I never thought that, like, marine science was, like, a thing for me. It was always just, like, space stuff. I like space because I like learning about how the world works. […] Just the idea of studying the unknown is very interesting. And you can find the same [i.e., unknown ocean depths] thing right in your backyard. Or at least in my case.

    As with the students’ confidence in their perceived science competence, their interest in science was increased as they were exposed to practical applications of their knowledge. This experience of science as a practice left a deep impression on Lewa:

    We’ll read, like, the lab or whatever, and then the next day we’ll go out and apply it, which is really nice. Like, I really liked that. Just, like, reading, ok yeah whatever … [Rolls eyes and waves hand dismissively]. And then you start to, like, actually understand “Oh! That’s what I read about. So that’s what we’re doing! Oh, cool!”

    Another concept that emerged from the discussions regarding interest in marine science included students feeling encouraged as competent researchers. The students were not simply interested in participating; they also felt pride in participating because of what they could accomplish. Misa, a mentor in the 2018 REMS cohort, described the gratification that resulted from putting effort into her research:

    I think it’s hard, but it is, like, it’s, like – I guess, satisfying in some ways to, like, know that you’re, I mean not really, but the way you’re, like, are actually doing something. That you’re changing things, I guess.

    Liam, another mentor, agreed with Misa and added a comment referring to his surprise about the relationship he developed with the new student members of his research group: “And, like, talking with your group members and, like, they talk to you and refer to you with, like, a certain level of respect, and it’s, like, kinda, wow!” In the case of these two students, the encouragement and confidence they perceived in response to their interest and effort in science contributed to their continuation as members of the REMS XL cohort.

    Valuing the Purpose of Research.

    Two of the open-ended survey questions probed the students to describe perceived relationships between humans and the marine environment. Student responses revolved around three elements: 1) the impact of humans on the marine environment, 2) an awareness of the value and importance of the ocean, and 3) a sense of obligation for responsible, positive stewardship of marine resources. The students also expressed an increase in awareness of or desire for integrating science knowledge and skills with other aspects of their lives. This included developing relationships with other members of their home and science communities (“I was able to connect our results and research to personal and family stories and experiences.”), as well as applying their knowledge and skills in other contexts (“I will forever be grateful for learning how to speak in front of other people, confidently, about a study I’ve done. It also gave me skills that I could use in everyday life like cooperation and communication.”). Students also reported a new recognition of how science affected their everyday lives (“I learned how science ties into our world and just how important it is to us”).

    During the focus-group interviews, the students described their desire to benefit their communities, and to share their knowledge and experiences with friends and family. The students recognized their communities (both their home and science communities) are built upon the relationships, knowledge, and experiences of the members. Toward the end of the REMS XL program, students participated in a community workday at a traditional Hawaiian fishpond, where several students were also conducting their independent research. When asked about the experience, the students noted that it nurtured a sense of purpose for their research and developed a cultural and environmental ethical context. Most also noted an appreciation for “giving back” to the community, as noted in this written comment: “Being out and observing in the actual field connects to scientific research because aside from giving back, you can actually see what is going on and why your research work would be valuable.”

    Student Science Identity and Conceptualization of Scientists

    To explore student conceptualizations of a “person of science”, this section describes student responses to open-ended survey prompts and guided interviews. An analysis of student illustrations of “a scientist” is also included to enrich the verbal and written data.

    In response to the survey item “In your own words, who or what is a scientist?” (Table 7), at the start of the program students defined scientists with predominantly Process-type descriptors (e.g., research, experiments, tests hypotheses, etc.; New Student: 18 comments; Mentor: 6 comments; REMS XL: 9 comments). After the program, there was a shift toward Character-type descriptors (e.g., curious, desire to learn, dedicated, etc.). Also emerging postprogram were comments that scientists advocate for change and have a responsibility to be honest. In their written responses at the end of the program, 76.5% of New Student, 80.0% of Mentor, and 42.9% of REMS XL student responses described a scientist as Anyone (e.g., “Anyone who is curious about the natural world.”).

    TABLE 7. Student codes describing scientists

    Preprogram survey prompt: “Who or what is a scientist”
    Frequency student group comments
    CodeSubcodeNew StudentMentorREMS XL
    Process1869
    Researches524
    Experiments510
    Tests hypotheses400
    Uses sci method111
    Educates110
    Discovers100
    Builds Knowledge100
    Asks questions/seeks answers014
    Characteristic924
    Curious212
    Knowledgeable100
    Passionate100
    Dedicated101
    Desire to learn201
    Desire to make positive Change200
    Like me010
    Postprogram survey prompt: “Who or what is a scientist?”
    Frequency student group comments
    CodeSubcodeNew StudentMentorREMS XL
    Process1436
    Researches413
    Experiments210
    Tests hypotheses100
    Uses sci method000
    Educates000
    Discovers000
    Builds knowledge102
    Asks questions/seeks answers611
    Characteristic1056
    Curious622
    Knowledgeable000
    Passionate100
    Dedicated011
    Desire to learn302
    Desire to make positive change000
    Like me010

    Note. Matrix of code and subcode frequency in student (New Student [N = 17], Mentor [N = 5], REMS XL [N = 12]) comments. Codes are not exclusive.

    Analysis of student-drawn images of scientists (see Figure 3 for examples) provided additional insight to student conceptualizations of a typical scientist. For New Student and Mentor scientist images, there were significant decreases in the averages of both the total number of indicators used (New Student pre: 4.7 ± 2.1, post: 3.7 ± 2.1, p = 0.02; Mentor pre: 4.8 ± 1.4, post: 3.4 ± 1.7, p = 0.007) and the indicators stereotypical of Western scientists (New Student pre: 3.5 ± 1.8, post: 2.2 ± 1.5, p = 0.0003; Mentor pre: 4.1 ± 1.4, post: 2.5 ± 1.8, p = 0.006; Figure 4). This decrease in use of stereotypic indicators is also reflected in the stereotype subcategories, as illustrated in Figure 5. In particular, the frequency of glasses (New Student: 65.7–22.9%; Mentor: 61.5–28.5%) and lab coats (New Student: 51.4–20.0%; Mentor: 69.2–21.4%) decreased significantly.

    FIGURE 3.

    FIGURE 3. Sample pre–postpaired DAST images. Note. Examples of student depictions of scientists (a) before and (b) after participation in the REMS program. Images are paired (top image is preprogram and bottom image is postprogram image from same student).

    FIGURE 4.

    FIGURE 4. Frequency of “scientist” indicators per student DAST image. Note. Average number of (a) total indicators and (b) indicators stereotypical of a Western scientist to denote “scientist” in student images (New Student [N = 34]; Mentor [N = 13]) before and after participation in the REMS program. Pre–post differences were compared with a two-tailed, paired Student's t test. *p < 0.05. **p < 0.01. ***p < 0.001.

    FIGURE 5.

    FIGURE 5. Western scientist stereotypes in new student and mentor DAST images. Note. Average number of indicators stereotypical of a Western scientist in student images ((a) New Student [N = 34]; (b) Mentor [N = 13]) before and after participation in the REMS program. Pre–postshifts were analyzed with McNemar's test of marginal homogeneity. Male scientists were initially considered a Western stereotype because the proportion of male images was larger than the proportion of males in the classroom (see Figure 6 for analysis of DAST image gender data). *p < 0.05, **p < 0.01, ***p < 0.001.

    A significant shift also occurred in the indicated gender of the drawn figures (Figure 6). In the beginning of the program, New Students drew 44% of their scientists as male and 24% as female. Mentors drew 62% of their scientists as male and 31% as female. By the end of the program, New Student figures were 35% male and 59% female, and Mentor figures were 38% male and 54% female. The postprogram proportion of scientist genders indicated by New Students shifted significantly from preprogram proportions (χ2[3, N = 32] = 8.83, *p < 0.05). Post-hoc McNemar tests showed significant change between the proportion of male and female figures. It was also noted that 30% of the students also unambiguously drew themselves as the scientist on the final day of the program.

    FIGURE 6.

    FIGURE 6. Gender of DAST figures. Note. Percent of male, female, or unknown gender figures in student images (New Student N = 34; Mentor N = 12) drawn before and after participation in the program. The postprogram proportion of male and female figures drawn by New Students shifted significantly, as indicated by the results of a McNemar-Bower test. * p < 0.05.

    As demonstrated by the students’ DAST images, it was clear that their conceptualization of a person of science shifted after participation in the REMS or REMS XL program. At the completion of the REMS XL program, Sarina viewed scientists through a broader lens that incorporated “alternative” researchers, such as traditional Native Hawaiian fishpond caretakers:

    [T]here’s not really, like, a “real” scientist in a way. Like, people are scientists in many different aspects and in different fields. Like, some of those scientists that I look up to now are the fishpond managers. Like, they know some crazy amount of, like, research that they’ve applied just by working in their field. […] You can be a researcher in many different aspects and not just in the field or in the lab bench and in lab gear and all that fun stuff.

    Mason, another participant in REMS XL, agreed with Sarina’s comment and described a scientist simply as someone with curiosity willing to put forth effort:

    I used to think that scientists were people who knew a lot of this stuff. But in reality, it’s people who are, like, they’re actually just people who don’t know everything, but they’re willing to learn and go through the effort of actually figuring out “What makes this tick?”, “How does it work?”, “Why is this the way it is?” And, yeah. So, it’s just like us, since if you’re curious and you wanna learn, you have a question, and you’re willing to go through the efforts and the lengths to figure an answer out, figure it out and answer that question, type of thing.

    As students worked alongside professional scientists, their image of a scientist was humanized. Mason discussed his relief at realizing that scientists were relatable:

    Meeting people from the science community and seeing how things actually are […]. And so, like, it also builds, like, connection. You can relate more, and it kinda reduces more, like, the stress. Like “Oooh, how am I going to amount to these people?” And they’re, like, doing the same thing. And so, it’s interesting to learn that.

    The idea of becoming a scientist emerged as an attainable goal. During the New Student group discussion, Nalu described the sentiment as feeling like “you’re not by yourself.”

    Participating in authentic research experiences also enabled the participants of REMS and REMS XL to become more aware of the structure of the science community. One of the mentors, Tiare, describes a sense of belonging that developed with an awareness of the many positions that scientists fill within their community:

    I feel like it kinda, like, opens up the idea that you’re actually not alone, when you think about it. Everyone’s like, “Oh, scientists. You’re normally in a lab, you’re doing your own thing, you have to do everything by yourself.” And then when you do this program, it’s like there’s so many different roles that come into play when it comes to doing science. Where there is people behind the scenes, but they also work with the people who go out in the field, and they actually work together to collaborate to make one big thing.

    The importance of different knowledge systems became especially apparent during discussions with the participants of REMS XL. Cora described her perception of scientists in this way:

    Yeah, to me it’s anyone who can ask a question of like “Why does this work?” and “How does it work?” That’s, like, what a scientist is to me. Um, even, like, the people who don’t have an education, but you know they’re out there in, like, the field and, like, they understand more than, like, the “real” scientists. Like they don’t know the basic, like, the biology or chemistry of it, but they know, like they take notice of the patterns in the natural world. That to me is kinda considered as a scientist.

    The other students agreed that for a person to be considered a scientist, they did not have to limit themselves to one way of thinking, one way of knowing.

    Outcome Expectations and Contextualizing Failure

    To identify some of the ways in which a place-based program could influence student persistence in STEM, this section describes student-identified factors that influenced their ability to overcome challenges related to conducting research and participating in the science community.

    In the program evaluation portion of the REMS survey, New Students and Mentors were asked to comment on their group project experience. Responses to this item were coded as either “Highlight” or “Challenge” (Table 8). “Working with Others” was the most frequent Highlight code occurring in New Student (24.0%) and Mentor (22.2%) responses. The most frequently mentioned Challenge for both student groups was “Interpersonal Differences” (New Students: 11.5%; Mentor: 33.3%). Although listed as a Challenge, Interpersonal Differences were often framed as a learning experience (e.g., “I honestly loved being able to be in a group that I can connect with others, there was the occasional problems but in the end everything was worth it in the end and we grew passed that.”).

    TABLE 8. Group-project evaluation response assessment

    Survey prompt: “Please comment on what you liked and did not like about the group research project.”
    CodeNew Student comments NMentor comments N
    Highlight of group projects
     Working with others25 (24.0%)4 (22.2%)
     Shared experience5 (4.8%)0 (0%)
     Field experience9 (8.7%)1 (5.6%)
     Hands-on5 (4.8%)0 (0%)
     Ownership2 (1.9%)1 (5.6%)
     Accomplishment4 (3.8%)1 (5.6%)
    Challenge of group projects
     Interpersonal differences12 (11.5%)6 (33.3%)
     Uneven contributions5 (4.8%)2 (11.1%)
     More time needed8 (7.7%)2 (11.1%)

    Note. Matrix of codes elicited from student (New Student [N = 106], Mentor [N = 27]) comments. Codes are not exclusive.

    In addition to fostering relationships on an individual level, students described increased interest and ability to navigate social dynamics on a community level. Sarina described how she became aware of the respect and understanding necessary for working with or participating in a community:

    I think for me it kinda, like, grew a deeper, like, appreciation. […] Like, I was always careful to be able to have that access to go to He‘eia fishpond and do my research there, and I was super thankful for that. […I]t really puts a different perspective and appreciation for communities opening up to let you do research in their areas. Like, if you don’t identify, and stuff like that.

    The students in each of the interview groups discussed the importance of diversity of thought and cultural grounding in science. The Mentors viewed the sometimes-contentious interpersonal experiences as helpful exercises in patience, thoughtfulness, and creative problem solving. Students also recognized the significant contributions to research a diverse group can provide because “there will always be different perspectives”, “many different people have many different kinds of ideas”, and “more minds means more ways of thinking.”

    During the focus-group interviews, students noted that working alongside scientists allowed them to experience how researchers often rely on each other to gather information or to overcome obstacles and make sure their projects are completed satisfactorily1. Sarina, a participant of REMS XL who had experience as both a REMS mentor and undergraduate intern, described her surprise at realizing that it was a desire for more knowledge that drove researchers in their endeavors:

    Like, after REMS I thought, like, a lot of the researchers, like, would kinda just know off the back of your, like, I dunno you, like, learn it somewhere, you know. […]For me, like, working with [instructor], I didn’t realize, like, [Instructor] also doesn’t know some things and we just had to ask for help from other people. And, like, learning that it’s ok to ask for help, like, in any aspect of your life, really.

    Scientists do not know everything, which is why they devote themselves to seeking knowledge. This pursuit of information often requires seeking help from other more knowledgeable researchers, experts, or community members to guide the scientist’s research endeavors. This thought was expanded upon by Cora, another REMS XL participant, who responded to Sarina’s comment on the importance of reciprocal collaboration:

    Learning that collaboration with others was—is, will help you grow faster. Um, not only in your research, but as a person too. And, like, just your personal goals. Like being able to reach out to people, make friends. Not just because you want something, but like a give and take, like, you can provide something for someone.

    Thus, the students recognized the importance of scientists actively contributing to the systems they studied, instead of only reaping information for private, personal enrichment.

    The final question on the REMS survey asked students what advice they would give to future cohorts participating in the program. Many responses urged future students to enjoy the program as much as possible. Many student comments also sought to soothe anxiety which they may have also felt entering an intensive research program (“You will honestly love it. Remember to have fun and don’t stress”) or preparing for the final symposium day (“I would tell future students that do not be scared or nervous on presentation day, have fun throughout the entire program”).

    Responses to this final survey item also indicated students had developed resilience to some of the challenges faced by scientists in navigating the research process. Students explained to their future counterparts that experiments with unexpected results were not failures, but pathways to new research questions (“Just because your data is off, doesn’t mean your project was a flop. There’s always an explanation or a reason you got that certain data and It will always lead you to keep asking more questions”). Some responses explicitly described how experimental obstacles were to be expected, but that the value of persevering was worth the temporary frustrations (“I would give future members advice to never give up. At times, things may get challenging because not everything will go the right way but never give up because once the challenge is over come, it will all be worth it.”).

    DISCUSSION

    This study sought to examine student evaluations of a Hawai‘i, place-based, experiential marine science research program, identify and describe the factors mediating student science identity development and conceptualizations of a “person of science”, and explore how a culturally-responsive CURE might contribute to a unique student population’s persistence in STEM. The analysis of codes elicited from surveys, student images, and focus-group interviews indicated the existence of three thematic elements potentially mediating the pathways between marine science self-efficacy, student science identity, and choosing a marine science or general science career pathway. These elements included: a) relevance of marine science, b) conceptualization of “person of science”, and c) contextual framing of “failure.” These results thus support previous suggestions (Robnett et al., 2015) that models using self-efficacy as a mediator between research experience and science identity should include additional mediators for marginalized students.

    Relevance of Marine Science

    Science self-efficacy is highly predictive of persistence in science pathways, but the internalization of science values (Estrada et al., 2011) or the recognition of culturally relevant research goals may be more predictive for the career choices of historically excluded students, particularly Native Hawaiian and Pacific Islander students (Allaire, 2018). Place-based curricula, such as those delivered during the REMS program, nurture situational interest (Hosbein and Barbera, 2020) through experiential and culturally grounded learning environments.

    In science identity models, interest in science has been suggested as a necessary domain for the development of a student’s sense as a “person of science”, as well as to predict persistence or success in science fields (Hazari et al., 2010). Students in this study valued the authentic, hands-on research experience provided by the REMS program as it increased their science self-efficacy by allowing them to practice the skills they learned. The students also reported increased enthusiasm for these fields after completion of the course and highlighted the importance of engaging instructors and relevant content that connected personally with those in the classroom and fostered interest in marine science.

    In place-based pedagogies, an increased sense of place also provides the foundation for an increased sense of responsibility or stewardship toward the contextual place (Gruenewald, 2008; Semken and Brandt, 2010; Membiela et al., 2011). The students in REMS and REMS XL, through their newfound appreciation for their local and science communities, realized that belonging to a community requires a reciprocal relationship that depends on a balanced give-and-take approach (“Instead of, like, taking everything that they’re giving you, instead of, just, pushing all that you have on to them.”). This type of relationship is also emphasized in the Hawaiian Kūlana Noi‘i research framework (Kūlana Noi‘i Working Group, 2021) in the concept of a‘o aku, a‘o mai/aloha aku, aloha mai2 that encourages mutually beneficial relationships through community engagement, understanding knowledge ownership and access, and responsible accountability.

    Of the sources of self-efficacy defined in SCCT, vicarious experiences (through observing a mentor or professional researcher) may be one of the more influential factors in the development of science identities for students from historically excluded groups (Kricorian et al., 2020). However, when students participate in research programs with a peer group or instructor who are not of the same race or ethnicity, the students are much less likely to experience positive shifts in their science identity (Flowers and Banda, 2016). The demographics of REMS program staff reflects the diverse racial demographics of Hawai‘i, and thus these instructors provide greater representation from groups historically excluded from STEM fields. REMS instructors share their experiences as scientists and encourage students to connect with community leaders and cultural practitioners to execute their research projects. REMS students reported an appreciation for this exposure to professionals and communities they did not previously associate with scientific research.

    Conceptualization of “Person of Science”

    Western stereotypic views of scientists may discourage marginalized students from pursuing science career and academic pathways (DeWitt et al., 2013), while having counter-stereotypical views may increase success in science (Schinske et al., 2015). Results from this study indicated student conceptualizations of the science community broadened, and scientists were humanized as students experienced the methodologies and worldviews of a diverse (in terms of ethnicity, age, professional position, etc.) range of researchers and cultural practitioners. The students also recognized that all communities, including the science community, are built and influenced by the participants.

    The concept of community emerged again and again throughout the written survey responses and interview discussions. As this study takes place in a Hawai‘i classroom, it is important to acknowledge that community relationships are central to the education (e.g., Na¯ Hopena A‘o) and research (e.g., Kūlana Noi‘i) frameworks of this place (Kūlana Noi‘i Working Group, 2021; Hawai‘i State Board of Education Ends Policy 3 or E-3, Nā Hopena A‘o, 2015). The identity model described by Carlone and Johnson (2007) suggests a student’s role identity (i.e., student science identity) is developed and maintained through connections with personal and social identities, as well as the student’s place, which includes the science learning environment. A science learning environment that encompasses values and concepts from the scientific and local communities can then strengthen a student’s science identity without diminishing the other aspects of the student’s identity (Kim and Sinatra, 2018). Then, as students become scientists in their own right, they may transform the science community itself.

    The DAST images highlighted several interesting shifts in students’ perceptions of what a scientist is in addition to the codes that emerged from the written survey responses. The images drawn after the program utilized fewer stereotypical indicators to denote a “scientist.” Fewer indicators may represent a more generalized conceptualization of who a “person of science” could be. These results mirrored the codes from the survey responses that demonstrated more students noted that “anyone could be a scientist” (as indicated with explicit text in their pictures), or that a scientific career is not limited to a narrow demographic field. The decrease in indicators utilized by the students in this program to draw their figures may signify that participation in REMS breaks down traditional stereotypes about scientists, and reverses students’ previously held conceptualizations (Chambers, 1983).

    Interestingly, both male and female students drew predominantly male (or neutral) scientists at the start of the program, but after the program the female students almost exclusively drew female scientists (while the male students continued to draw male scientists). Women, and particularly women of color, still experience many challenges to their participation in STEM fields, but these challenges can be mitigated by developing more inclusive learning environments, for example, utilizing instructors who are diverse in race/ethnicity, gender, and age (Kim et al., 2018). For each iteration of the REMS program, the staff has been composed of at least 50% female scientists. Perhaps the shift in the gender of drawn scientists is due to students vicariously learning from or developing mentoring relationships with female professionals, which may ultimately motivate female students to recognize participation in STEM fields as a possible career goal. Also of note, this instructor roster which contributed to the positive shift in female student conceptualizations of scientists did not affect male student conceptualizations. The DAST images also demonstrated a shift toward student self-recognition as a scientist after participation in the program. This is important because integration of a student’s conceptualization of a scientist with their own self-image is demonstrated to predict persistence in science (McDonald et al., 2019).

    After participation in the program, students noted that scientists were more relatable. The students viewed conducting research as an attainable career goal, and they felt pride at being contributing members of the scientific community. A sense of belonging to the science community is especially beneficial for historically excluded students as it has been demonstrated to mitigate anxiety in navigating Western research environments (Fisher et al., 2019). During the REMS program, the students saw themselves as scientists and felt a comradery amongst their peers and the program instructors. Models such as identity-based motivation framework suggest this positive sense of self within the science community may contribute to persistence in science pathways (Oyserman and Destin, 2010).

    Contextual Framing of “Failure”

    Fear of failure is a complex construct that is expressed through an interplay between affective responses, personality, and cognitive factors (Henry et al., 2021). Conroy et al. (2003) applied the cognitive-motivational-relational theory of emotion (Lazarus, 1991) to describe fear of failure as a multidimensional construct composed of two processes: 1) a student anticipates failure is possible or recognizes they are presently failing, and 2) a student predicts that failing in this situation will have negative consequences. In STEM academic settings, fear of failure may have specific contextual drivers (such as fear of uncertain future or fear of upsetting important others) and has been linked with procrastination, reduced motivation, and attrition from STEM pathways (Onwuegbuzie et al., 2009; Zhang et al., 2018; Nelson et al., 2019; Ceyhan and Tillotson, 2020; Henry et al., 2021). Ironically, although the natural science research process itself requires the ability to navigate obstacles and utilize an iterative process in response to failure (e.g., data does not match predicted patterns), research and instructor-driven curricula are only beginning to address ways in which to nurture this perspective and skill in students (Simpson and Maltese, 2017; Henry et al., 2019).

    The independent research projects conducted by REMS students are inspired by professional research at HIMB, but the student projects are required to explore novel research questions. Many of the projects involve using under-researched organisms or developing novel observational or experimental techniques. This experience exposes students and instructors to experimental obstacles that they must troubleshoot effectively to reach a desirable project outcome together. Guided by their instructors through this experience, students began to recontextualize what they considered to be failure in science. They saw that researchers must anticipate and address challenges as part of the iterative process of science. Students also learned that unexpected results should encourage the development of new research questions to be explored, instead of denoting that an experiment failed outright.

    “Failure” framed in this context addresses two of the cognitive processes involved with the fear of failure: a) anticipating failure and b) expectation of adverse consequences to that failure (Conroy et al., 2003). The students were able to observe how the instructors (researchers themselves) responded to frustration, uncertainty, and anxiety in dealing with novel research projects constrained by tight deadlines. Students who were alumni of the program and already familiar with the course content continued to benefit and find value in the experience as they continued to learn through their shifting roles in the program (i.e., as Mentors to novice participants and as undergraduate researchers in REMS XL). The inherent iterative nature of conducting novel research provided a fitting context and safe environment for students to experience and learn from “failure.”

    Beyond a sense of failure due to perceived lack of achievement, the REMS program also aims to address a sense of failure from not matching or integrating with a preconceived stereotype of a Western scientist (Schinske et al., 2015; Starr, 2018). While attempting to navigate undergraduate STEM programs, students from groups historically marginalized in science not only experience disproportionately fewer racial/ethnic or same-gendered peers and instructors, they may also experience direct racial prejudices which impair a sense of belonging to the scientific community (Fisher et al., 2019; Kricorian et al., 2020). In addition to this unwelcoming academic environment, some students may also be discouraged from pursuing STEM from members of their home communities who see science as a job done by “others” (DeWitt et al., 2013; Allaire, 2018). The student groups at each stage of participation (New Students, Mentors, and REMS XL students) appreciated the place-based aspects of the REMS curriculum and working alongside researchers who shared characteristics with themselves. The REMS XL students in particular noted the value of doing authentic, culturally grounded research with professional scientists who demonstrated how someone like them could succeed and thrive in the science community.

    CONCLUSION

    Recognition of oneself as a “person of science” is an important component of developing and maintaining a science identity. But as this study focuses on the experiences of marginalized students, it is important to be mindful of the ideological context in which a perceived “person of science” develops. Our goal is not to ask students to fit a certain stereotype that may clash with their social or personal identities in order to become a scientist, but instead to examine how a science learning environment can prepare Hawai‘i students to share their unique experiences and world views to enrich the science community.

    The results of this study, which uncover shifts in students’ conceptualizations of what it means to be a scientist, demonstrate that the development of a student’s identity is not independent from the science learning experience (which includes the instructor, the classroom, the other students, the curriculum, etc.) and the relative place in which they are both situated. This aligns with previous research that indicates science identity salience, and ultimately a decision to pursue STEM pathways, in Native Hawaiian students and professionals is the result of a mixture of internal cognitive processing and external motivational factors grounded in cultural contexts (i.e., shifts in conceptualizations as well as relevance/value of science; Allaire, 2018). Thus, the results of this study suggest that the conceptualization of a “person of science”, and a recontextualized understanding of “failure” may act as influential mediators between the science learning experience and Native Hawaiian student resilience in STEM pathways.

    ACCESSING MATERIALS

    The 2018 pre- and postprogram New Student and Mentor REMS survey instruments are available online as supplemental materials.

    FOOTNOTES

    1We emphasize to our students that a satisfactory conclusion does not require the data overwhelmingly support a specific hypothesis. Unexpected results are often more exciting for professional scientists than predicted results.

    2In broad terms, this phrase expresses the notion of a bi-directional flow of knowledge (a‘o) and compassion (aloha).

    ACKNOWLEDGMENTS

    The authors would like to thank all the REMS instructors and staff for their efforts to empower Hawai‘i students, especially Mackenzie Manning, Yoshimi Rii, Sherril Leon Soon, Jeremy Soriano, Kelvin Gorospe, Hoaka Thomas, and Alexi Meltel. We also sincerely thank the students of REMS for allowing us to share in their experiences. We are grateful for the many discussions on culturally based education research with Margie Maaka, Liana Wong, and Summer Maunakea, and science-based education research with members of CC Bio INSITES community of National Science Foundation. Finally, we thank the Hau‘oli Mau Loa Foundation for their generous support of the REMS program, Lisa Corwin and Jeff Schinske of the CC Bio INSITES initiative for their generous support of this manuscript publication, and the STAR Achievement Scholarship and University of Hawai‘i at Ma¯noa COE General Scholarship which supported exploration into our students’ experiences.

    REFERENCES

  • Allaire, F. S. (2018). Themes from a narrative analysis of Native Hawaiian experiences in science, technology, engineering, and mathematics. Journal of Ethnographics and Qualitative Research, 12, 173–192. Google Scholar
  • Ambrosino, C. M., & Rivera, M. A. J. (2022). A longitudinal analysis of developing marine science identity in a place-based, undergraduate research experience. International Journal of STEM Education, 9(1), 1–24. https://doi.org/10.1186/s40594-022-00386-4 Google Scholar
  • Ballen, C. J., Wieman, C., Salehi, S., Searle, J. B., & Zamudio, K. R. (2017). Enhancing diversity in undergraduate science: Self-efficacy drives performance gains with active learning. CBE—Life Sciences Education, 16(4), 1–6.  https://doi.org/10.1187/cbe.16-12-0344 Google Scholar
  • Byars-Winston, A., Rogers, J., Branchaw, J., Pribbenow, C., Hanke, R., & Pfund, C. (2016). New measures assessing predictors of academic persistence for historically underrepresented racial/ethnic undergraduates in science. CBE—Life Sciences Education, 15(3), ar32. https://doi.org/10.1187/cbe.16-01-0030 LinkGoogle Scholar
  • Carlone, H. B., & Johnson, A. (2007). Understanding the science experiences of successful women of color: Science identity as an analytic lens. Journal of Research in Science Teaching, 44(8), 1187–1218.  https://doi.org/10.1002/tea.20237 Google Scholar
  • Ceyhan, G. D., & Tillotson, J. W. (2020). Early year undergraduate researchers’ reflections on the values and perceived costs of their research experience. International Journal of STEM Education, 7(1), 1–9. https://doi.org/10.1186/s40594-020-00248-x Google Scholar
  • Chambers, D. W. (1983). Stereotypic images of the scientist: The draw-a-scientist test. Science Education, 67(2), 255–265.  https://doi.org/10.1002/sce.3730670213 Google Scholar
  • Chemers, M. M., Zurbriggen, E. L., Syed, M., Goza, B. K., & Bearman, S. (2011). The role of efficacy and identity in science career commitment among underrepresented minority students. Journal of Social Issues, 67(3), 469–491.  https://doi.org/10.1111/j.1540-4560.2011.01710.x Google Scholar
  • Cheryan, S., Plaut, V. C., Davies, P. G., & Steele, C. M. (2009). Ambient belonging: How stereotypical cues impact gender participation in computer science. Journal of Personality and Social Psychology, 97(6), 1045–1060.  https://doi.org/10.1037/a0016239 MedlineGoogle Scholar
  • Christidou, V., Bonoti, F., & Kontopoulou, A. (2016). American and greek children’s visual images of scientists: Enduring or fading stereotypes? Science and Education, 25(5–6), 497–522.  https://doi.org/10.1007/s11191-016-9832-8 Google Scholar
  • Conroy, D. E., Metzler, J. N., & Hofer, S. M. (2003). Factorial invariance and latent mean stability of performance failure appraisals. Structural Equation Modeling, 10(3), 401–422.  https://doi.org/10.1207/S15328007SEM1003_4 Google Scholar
  • Corwin, L. A., Graham, M. J., & Dolan, E. L. (2015). Modeling course-based undergraduate research experiences: An agenda for future research and evaluation. CBE—Life Sciences Education, 14(1), 1–13.  https://doi.org/10.1187/cbe.14-10-0167 Google Scholar
  • DeWitt, J., Archer, L., & Osborne, J. (2013). Nerdy, Brainy and Normal: Children’s and Parents’ Constructions of Those Who Are Highly Engaged with Science. Research in Science Education, 43(4), 1455–1476.  https://doi.org/10.1007/s11165-012-9315-0 Google Scholar
  • Estrada, M., Woodcock, A., Hernandez, P. R., & Schultz, P. W. (2011). Toward a model of social influence that explains minority student integration into the scientific community. Journal of Educational Psychology, 103(1), 206–222.  https://doi.org/10.1037/a0020743 MedlineGoogle Scholar
  • Fisher, A. J., Mendoza-Denton, R., Patt, C., Young, I., Eppig, A., Garrell, R. L., ... & Richards, M. A. (2019). Structure and belonging: Pathways to success for underrepresented minority and women PhD students in STEM fields. PLoS ONE, 14(1), 1–14.  https://doi.org/10.1371/journal.pone.0209279 Google Scholar
  • Flowers, A. M., & Banda, R. (2016). Cultivating science identity through sources of self-efficacy. Journal for Multicultural Education, 10(3), 405–417. https://doi.org/10.1108/JME-01-2016-0014 Google Scholar
  • Glaser, B. G. (1992). Emergence vs. forcing. Basics of grounded theory analysis. Mill Valley, CA: Sociology Press. Google Scholar
  • Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory: Strategies for qualitative research. New York, NY: Aldine De Gruyter. Google Scholar
  • Gruenewald, D. A. (2008). The best of both worlds: A critical pedagogy of place. Environmental Education Research, 14(3), 308–324.  https://doi.org/10.1080/13504620802193572 Google Scholar
  • Hawai‘i State Board of Education Ends Policy 3 or E-3, Nā Hopena A‘o (Issue November). (2015). State of Hawai‘i Department of Education. Retrieved December 1, 2021, from https://www.hawaiipublicschools.org/DOEForms/NaHopenaAoE3.pdf Google Scholar
  • Hazari, Z., Sonnert, G., Sadler, P. M., & Shanahan, M.-C. (2010). Connecting high school physics experiences, outcome expectations, physics identity, and physics career choice: A gender study. Journal of Research in Science Teaching, 47(8), 978–1003. https://doi.org/10.1002/tea.20363 Google Scholar
  • Henry, M. A., Shorter, S., Charkoudian, L., Heemstra, J. M., & Corwin, L. A. (2019). FAIL is not a four-letter word: A theoretical framework for exploring undergraduate students’ approaches to academic challenge and responses to failure in STEM learning environments. CBE—Life Sciences Education, 18(1), ar11. https://doi.org/10.1187/cbe.18-06-0108 LinkGoogle Scholar
  • Henry, M. A., Shorter, S., Charkoudian, L. K., Heemstra, J. M., Le, B., & Corwin, L. A. (2021). Quantifying fear of failure in STEM: Modifying and evaluating the Performance Failure Appraisal Inventory (PFAI) for use with STEM undergraduates. International Journal of STEM Education, 8(1), 43.  https://doi.org/10.1186/s40594-021-00300-4 Google Scholar
  • Hosbein, K. N., & Barbera, J. (2020). Alignment of theoretically grounded constructs for the measurement of science and chemistry identity. Chemistry Education Research and Practice, 21(1), 371–386.  https://doi.org/10.1039/c9rp00193j Google Scholar
  • Kim, A. Y., & Sinatra, G. M. (2018). Science identity development: An interactionist approach. International Journal of STEM Education, 5(1), 1–6. https://doi.org/10.1186/s40594-018-0149-9 MedlineGoogle Scholar
  • Kim, A. Y., Sinatra, G. M., & Seyranian, V. (2018). Developing a STEM identity among young women: A social identity perspective. Review of Educational Research, 88(4), 589–625.  https://doi.org/10.3102/0034654318779957 Google Scholar
  • Kricorian, K., Seu, M., Lopez, D., Ureta, E., & Equils, O. (2020). Factors influencing participation of underrepresented students in STEM fields: Matched mentors and mindsets. International Journal of STEM Education, 7(1). https://doi.org/10.1186/s40594-020-00219-2 Google Scholar
  • Krueger, R. A. (2002). Designing and conducting focus group interviewshttps://doi.org/10.1136/bmj.311.7000.299 Google Scholar
  • Kūlana Noi‘i Working Group. (2021). Kūlana Noi‘i v. 2. University of Hawai‘i SOEST College Program. Retrieved December 1, 2021, from https://seagrant.soest.hawaii.edu/kulana-noii/ Google Scholar
  • Lazarus, R. S. (1991). Progress on a cognitive-motivational-relational theory of emotion. American Psychologist, 46(8), 819. MedlineGoogle Scholar
  • Lent, R., Brown, S. D., & Hackett, G. (1994). Toward a unifying scct and academic interest, choice and performance. Journal of Vocational Behavior, 45, 79–122. Google Scholar
  • Lichtman, M. (2013). Making meaning from your data. In Qualitative Research in Education: A User’s Guide 3rd ed., Los Angeles, CA: Sage Publications. Google Scholar
  • Mānoa 2025: Our kuleana to Hawai‘i and the world. (2015). The University of Hawai‘i at Mānoa. Google Scholar
  • McDonald, M. M., Zeigler-Hill, V., Vrabel, J. K., & Escobar, M. (2019). A single-item measure for assessing STEM identity. Frontiers in Education, 4(July), 1–15. https://doi.org/10.3389/feduc.2019.00078 Google Scholar
  • Mead, M., & Métraux, R. (1957). Image of the scientist among high-school students. A pilot study. Science, 126(3270), 384–390.  https://doi.org/10.1126/science.126.3270.384 MedlineGoogle Scholar
  • Membiela, P., DePalma, R., & Pazos, M. S. (2011). A sense of place in the science classroom. Educational Studies, 37(3), 361–364.  https://doi.org/10.1080/03055698.2010.506340 Google Scholar
  • Nelson, K. L., Nelson, K. K., McDaniel, J. R., & Tackett, S. (2019). Majoring in STEM: How the factors of fear of failure, impostor phenomenon, and self-efficacy impact decision-making. National Social Science Journal, 52(1), 76–87. http://nssa.us Google Scholar
  • Onwuegbuzie, A. J., Dickinson, W. B., Leech, N. L., & Zoran, A. G. (2009). A qualitative framework for collecting and analyzing data in focus group research. International Journal of Qualitative Methods, 8(3), 1–21.  https://doi.org/10.1177/160940690900800301 Google Scholar
  • Oyserman, D., & Destin, M. (2010). Identity-based motivation: Implications for intervention. The Counseling Psychologist, 38(7), 1001–1043.  https://doi.org/10.1177/0011000010374775 MedlineGoogle Scholar
  • Ramsey, L. R., Betz, D. E., & Sekaquaptewa, D. (2013). The effects of an academic environment intervention on science identification among women in STEM. Social Psychology of Education, 16(3), 377–397.  https://doi.org/10.1007/s11218-013-9218-6 Google Scholar
  • Rivera, M., Ambrosino, C., Manning, M., Soon, S., Rii, Y., & Gorospe, K. (2022). Broadening participation through Research Experiences in Marine Science: An early-admit immersive college course provides experiential, place-based scientific training for Hawai‘i high school students. Oceanography, 35(2), 60–71. https://doi.org/10.5670/oceanog.2022.204 Google Scholar
  • Robnett, R. D., Chemers, M. M., & Zurbriggen, E. L. (2015). Longitudinal associations among undergraduates’ research experience, self-efficacy, and identity. Journal of Research in Science Teaching, 52(6), 847–867.  https://doi.org/10.1002/tea.21221 Google Scholar
  • Saldaña, J. (2015). The coding manual for qualitative researchers (3rd edition). Qualitative Research in Organizations and Management An International Journal, 12(2), 169–170. https://doi.org/10.1108/QROM-08-2016-1408 Google Scholar
  • Schinske, J., Cardenas, M., & Kaliangara, J. (2015). Uncovering scientist stereotypes and their relationships with student race and student success in a diverse, community college setting. CBE—Life Sciences Education, 14(3), 1–16. https://doi.org/10.1187/cbe.14-12-0231 Google Scholar
  • Semken, S., & Brandt, E. (2010). Implications of sense of place and place-based education for ecological integrity and cultural sustainability in diverse places. In Tippens, D. J. Mueller, M. P.van Eijck, M. Adams, J. D. (eds.) Cultural studies and environmentalism: The confluence of ecojustice, place-based (science) education, and Indigenous knowledge systems (pp. 287–302). Dordrecht, Netherlands: Springer. https://doi.org/10.1007/978-90-481-3929-3 Google Scholar
  • Seymour, E., Wiese, D., Hunter, A., & Daffinrud, S. M. (2000). Creating a better mousetrap: On-line student assessment of their learning gains. Google Scholar
  • Simpson, A., & Maltese, A. (2017). “Failure is a major component of learning anything”: The role of failure in the development of STEM professionals. Journal of Science Education and Technology, 26(2), 223–237.  https://doi.org/10.1007/s10956-016-9674-9 Google Scholar
  • Starr, C. R. (2018). “I’m not a science nerd!”: STEM stereotypes, identity, and motivation among undergraduate women. Psychology of Women Quarterly, 42(4), 489–503.  https://doi.org/10.1177/0361684318793848 Google Scholar
  • Strauss, A. L. (1987). Qualitative analysis for social scientists. Cambridge, UK: Cambridge University Press. Google Scholar
  • Trujillo, G., & Tanner, K. D. (2014). Considering the role of affect in learning: Monitoring students’ self-efficacy, sense of belonging, and science identity. CBE—Life Sciences Education, 13(1), 6–15.  https://doi.org/10.1187/cbe.13-12-0241 LinkGoogle Scholar
  • Zhang, Y., Dong, S., Fang, W., Chai, X., Mei, J., & Fan, X. (2018). Self-efficacy for self-regulation and fear of failure as mediators between self-esteem and academic procrastination among undergraduates in health professions. Advances in Health Sciences Education, 23(4), 817–830.  https://doi.org/10.1007/s10459-018-9832-3 MedlineGoogle Scholar