A Student-Centered, Entrepreneurship Development (ASCEND) Undergraduate Summer Research Program: Foundational Training for Health Research

INTRODUCTION

Although recent years have seen modest gains in the number of underrepresented minority (URM) students who complete doctoral work and persist in science, technology, engineering, and mathematics (STEM) careers, African-American, Native American, Latinx, and other minority groups are still woefully underrepresented in the STEM and biomedical sciences fields (National Academy of Sciences, National Academy of Engineering, and Institute of Medicine, 2011; President’s Council of Advisors on Science and Technology [PCAST], 2012; Fiegener and Proudfoot, 2013; Valantine and Collins, 2015; Valantine et al., 2016; Gibbs and Marsteller, 2016; National Center for Science and Engineering Statistics (NCSES), 2020). As the U.S. population continues to diversify, disproportional underrepresentation in the STEM workforce not only presents a social justice issue but also poses a growing economic problem for the country (PCAST, 2012; Freeman and Huang, 2014). It has become well established that diverse teams are more creative, effective, efficient, and lucrative, resulting in higher-quality research, products, and health services (Herring, 2009; Hunt et al., 2015). Thus, we must continue to emphasize the development and assessment of effective interventions to increase participation of all aspects of the U.S. population in STEM and biomedical research (National Academy of Sciences, National Academy of Engineering, and Institute of Medicine, 2011; Meyers et al., 2016; Valantine et al., 2016).

Participation in authentic undergraduate research experiences enhances STEM retention and students’ interest in pursuing research and graduate degrees (Seymour et al., 2004; Lopatto, 2007; Russell et al., 2007). This has shown to be important, in particular, for students from URMs (Nagda et al., 1998; Schultz et al., 2011; Jeffe et al. 2012; Hernandez et al., 2018a).

The research literature suggests that, for URM, first-generation, and female students, sociopsychological factors are as important as academic preparedness in generating STEM persistence and self-efficacy (Estrada et al., 2011; Merolla and Serpe, 2013; Spitzer and Aronson, 2015; Byars-Winston et al., 2016; Tibbetts et al., 2016). Factors such as science self-efficacy, sense of belonging, and science identity have been characterized as essential and interrelated cognitive and emotional predictors in URM students’ decisions to persist in their science education. Studies suggest that these factors are leading mediators and moderators in effective undergraduate science training for URM students (Chemers et al., 2011; Merolla and Serpe, 2013; Trujillo and Tanner, 2014; Lujan and DiCarlo, 2017; Hernandez et al., 2018b). Students’ sense of autonomy (or ownership in their learning) further strengthens science identity and self-efficacy (Lujan and DiCarlo, 2017). For first-generation undergraduate students, in particular, the personal and social relevance of their research is of great importance (Thoman et al., 2013).

At Morgan State University (MSU), we have developed an innovative training model, A Student-Centered, EntrepreNeurship Development (ASCEND), geared at building student science identity, self-efficacy, and health science knowledge through interdisciplinary training that emphasizes team building and collective ownership of research ideas. This training model is based on the hypothesis that increasing students’ autonomy in the selection of research topics, along with community building, will enhance their science identity and self-efficacy, enhance retention to graduation, and ultimately result in committed young researchers with leadership ability (Kamangar et al. 2017, 2019).

The Summer Research Institute (SRI) at the center of this paper represents the first and foundational step in the ASCEND training model developed under the National Institutes of Health Building Infrastructure Leading to Diversity (NIH BUILD) Initiative at MSU. The ASCEND model differs from most other research training programs in that it abandons the apprenticeship model for a group-based, student-centered training model that promotes entrepreneurial attributes such as creativity, autonomy, proactivity, strategizing, risk taking, and personal responsibility (Brockhaus, 1982; Bird, 1988; Hisrich, 1990; Ghosh and Raharam, 2015; Robinson et al., 2016; Rosique-Blasco et al., 2016).

The SRI has been geared, in particular, toward retention of lower-division undergraduate URM students in biomedical sciences with the long-term objective of encouraging postgraduate training and health science careers, which is the focus of the subsequent years of training in the ASCEND model. In the first 2 years of college, students make crucial decisions to continue or withdraw from STEM majors (PCAST, 2012). For URM students, interest in STEM majors is initially equivalent to that of other groups, but disproportionately more URM and female students withdraw from STEM disciplines (National Academy of Sciences, National Academy of Engineering, and Institute of Medicine, 2011; PCAST, 2012; Meyers et al., 2018). Thus, strengthening research knowledge and self-efficacy along with science identity in lower-division URM students should increase the number of students who are successfully retained to graduate in their science majors as well as prepare them for subsequent research training in the ASCEND Program with a focus on graduate school preparedness.

This study focuses on assessing the effect of the ASCEND SRI in measures of identity, attitude, and aspirations, including items related to academic self-concept, scientific self-efficacy, science identity, adjustment to college, engagement in research, and satisfaction with faculty mentorship; participation in academic and professional student organizations; and intent to pursue a career in biomedical research. We hypothesized that our training approach would result in increased science self-efficacy and science identity in SRI participants and therewith improved retention in their majors and subsequent college completion.

The survey questions and constructs used here were developed after an extensive literature review and are also included in the Cooperative Institutional Research Program surveys administered by the Higher Education Research Institute (HERI; Eagan et al., 2017; McCreath et al., 2017). In particular, the questions used here were part of consensus instruments that were applied to all BUILD-supported (NIH General Fund U54) institutions via the Consortium Evaluation Center (CEC), which guided individual BUILD site assessment with Hallmarks of Success (McCreath et al., 2017; NIH, 2019)

MATERIALS AND METHODS

Participants

Participants for the SRI were recruited from among lower-division undergraduates across STEM and social behavioral sciences. Students were required to have a minimum grade point average (GPA) of 2.8 and to have completed a minimum of 20 but no more than about 60 credit hours (counting toward their major) at the start of the summer. Because the program was federally (NIH) funded, students also provided documentation of U.S. citizenship or permanent residency. An 11-item rubric was devised to assess student suitability for the SRI training program. Points were allocated for being first-generation college attendees, being eligible for a Pell grant, having disability status, and having participated in community or volunteer work. Recommendation letters were scored for evidence of applicants being self-motivated and proactive, having leadership potential, and showing the ability to work well in a team. Students were asked to provide essay responses to five questions that focused on their interest in health research and their intent to engage in graduate education. One question also inquired about the students’ prior experience with overcoming personal and academic obstacles to assess resilience. Applicants were rated and ranked based on the above mentioned 11-item rubric. A maximum of 30 students were admitted into the SRI each year, except for cohort 4, when the maximum was 20. Not all students accepted into the program enrolled, but all enrolled students completed their training in the SRI. Participant demographics for each cohort are listed in Table 1.

Training Strategy and Components

The ASCEND SRI used a course-based undergraduate research experience–like approach instead of individual student placement in research labs (Auchincloss et al., 2014; Shapiro et al., 2015; Elgin et al., 2016; Staub et al., 2016). Instructions were based on evidence-based, student-centered approaches and relied heavily on group work (National Academy of Sciences, National Academy of Engineering, and Institute of Medicine, 2011). Students were supported by near-peer mentors (graduate students or senior undergraduates) with substantial research experience. A group of five instructors with disciplinary grounding in biology, chemistry, public health, and psychology comprised the instructional team.

The curriculum combined cooperative learning with hands-on research modules focused on health topics (see list in Supplemental Table A4). Students were introduced to scientific thinking and the research process as they worked in interdisciplinary groups. Instructors modeled different disciplinary approaches to health science research using scientific literature as well as short, hands-on research projects, both lab and field based. Students were challenged to identify real-world health science problems and propose solutions. Implicit in the SRI entrepreneurial focus are achievement orientation, becoming a greater (intellectual) risk taker, and developing emotional resilience (Bird, 1988; Hisrich, 1990; Ghosh and Raharam, 2015; Robinson et al., 2016). We thus aimed to cultivate a growth mindset in students as a means to enhance their personal self-efficacy (Trujillo and Tanner, 2014; Spitzer and Aronson, 2015; Claro et al., 2016; Byars-Winston and Rogers, 2019).

The curriculum had four sections: “I. Introduction to Health and Health Research,” “II. Basic Research Methodology,” “III. How to Choose a Research Topic,” and “IV. How to Write a Research Proposal.” Each section comprised a set of learning objectives that were assessed using rubrics in analyzing students’ written or oral deliverables, many of which were team based, but others required individual submissions. These assessments were purely formative. Although students received extensive feedback from near-peer mentors and instructors on their work, no grades were given nor were scores shared with them.

The training culminated in research proposals, generated by interdisciplinary participant groups and based on research questions of the students’ choice (see Supplementary Material Table A5). At the program’s completion, the students presented their proposals orally and in writing to an audience including campus faculty and staff, among whom were academic deans, the university provost and president as well as NIH program officers. The written proposals were reviewed by an NIH-style “study section” composed of faculty, postdocs, and graduate students with relevant disciplinary knowledge. Members of these review sections were recruited from our research partner institutions, Johns Hopkins University and the University of Maryland School of Medicine, as well as from MSU. Proposals were scored and ranked. Proposals deemed meritorious based on their scores were further developed in the following semester and eventually “funded” internally by the BUILD grant, as students moved on in their training to become ASCEND research scholars (see Supplementary Material Table A5). As indicated previously, the SRI was designed as entry training for a subsequent 2-year interdisciplinary group-research training experience—the ASCEND Scholars Program (see Kamangar et al. 2017, 2019). Based on students’ performance in the SRI and self-declared interest, on average, 76% of SRI participants over the course of the 4 years were selected to continue as ASCEND Scholars.

Summative performance outcomes were assessed via the students’ final oral and written proposal presentations and comprised:

1. ability to review and synthesize research literature orally and in writing;

2. ability to communicate and present (orally) scientific information fluently and in an organized way; and

3. ability to define the purpose of a research proposal and demonstrate this understanding by generating a team-based proposal.

For a detailed list of learning objectives please see Supplemental Table A1. As shown in this table, many of the objectives in sessions I through IV were iterative. These learning objectives were deepened for each of the subsequent instructional sections, and criteria for performance outcomes were increased incrementally.)

Measures

Measures and assessment items used in this study were informed by the Consortium-wide Evaluation Plan (McCreath et al., 2017), which is guided by the Hallmarks of Success (NIH, 2019) developed by the Diversity Program Consortium Coordination and Evaluation Center (DPC CEC, n.d.), of the National Institute of General Medical Sciences at NIH. Survey questions addressed constructs including perceptions of culture and environment, identity, attitude, and aspirations; the survey included items related to academic self-concept, scientific self-efficacy, science identity, adjustment to college, engagement in research and satisfaction with faculty mentorship, participation in academic and professional student organizations, and intent to pursue a career in biomedical research. In addition to these self-reported outcomes, we also employed an assessment of critical thinking skills—the Critical Thinking Assessment Test (CAT), which has been widely used across U.S. college student populations responses (Stein et al., 2007, 2016). Analyses in this paper focus predominantly on the constructs: scientific self-efficacy, science identity, adjustment to college, university’s contribution to research ability, academic self-concept, and belongingness. Descriptive analyses were conducted for all constructs showing differences (see Supplemental Table A2). To indicate sample mean predictability to the population mean, the standard errors are presented. To estimate variability and reliability of the sample, the standard deviations and Cronbach’s alphas are reported.

Science self-efficacy was measured using an eight-item scale and a 10-point rating rubric, with “1” the lowest rating as “not at all confident” and “10” the highest as “absolutely confident.” The Science Self-Efficacy construct in HERI surveys is defined as “a measure of students’ confidence in their ability to conduct scientific research” and asks: “How would you rate your ability to: ‘analyze data,’ ‘choose a research topic,’ or ‘work with others in a group’?” In cohort 1, the scale was introduced in the posttest to replace an ill-fitting Academic and Scientific Self-Efficacy questionnaire. For the remaining cohorts, the scale was administered at the beginning and end of the program to measure change (pre- and posttests). Moreover, the posttest also included a retrospective or reflective design component in which students were asked to rate what they thought their skill level was at the start of the SRI, now that they had completed their training (reflective test; see Supplemental Table A2). Additional questions were added for cohort 4 (2018). These questions included: “How would you rate your ability to: ‘Use technical science skills (use of tools, instruments, and/or techniques),’ ‘Generate a research question,’ ‘Determine how to collect appropriate data,’ ‘Explain the results of a study,’ and ‘Use scientific literature to guide research’?” (see Supplemental Table A2).

Science identity was only analyzed for cohort 4 (2018). The science identity construct from the CEC (DPC CEC, n.d.) was added to the pre-reflective test, and post-test questionnaires. This construct, which was adopted by HERI in 2016 from previous work (Estrada et al., 2011), is defined as “the extent to which students conceive of themselves as scientists,” and the items include: “I have a strong sense of belonging to the community of scientists,” “I derive great personal satisfaction from working on a team that is doing important research,” “I think of myself as a scientist,” and “I feel like I belong in the field of science.” It asks students to what extent the statements are true of them, and responses are collected on a five-point Likert scale: strongly disagree = 1, disagree somewhat = 2, neutral = 3, agree somewhat = 4, and strongly agree = 5.

The academic adjustment construct included four items: “Develop close friendships with other students,” “Develop effective study skills,” “Adjust to the academic demands of college,” and “Manage your time effectively.” Each was measured with a four-point Likert scale: very difficult = 1, somewhat difficult, somewhat easy, and very easy = 4.

University’s contribution to research ability was represented with one item: “This university has contributed to my ability to conduct research.” The Likert scale ranged from strongly disagree = 1 to strongly agree = 5.

The academic self-concept construct measure asked participants to “compare themselves to an average person of the same age academic ability, drive to achieve, mathematical ability, and intellectual self-confidence.” Likert five-point scale response choices included the comparison choices of lowest 10% = 1, below average, average, above average, and highest 10% = 5.

The belongingness items included: “I see myself as part of the campus community,” “I feel a sense of belonging to this campus,” “I feel that I am a member of this University,” “I feel valued at this University,” and “If asked, I would recommend this University to others.” The Likert scale ranged from strongly disagree = 1 to strongly agree = 5.

The CAT proprietary instrument is managed by the Center for Assessment and Improvement of Learning at Tennessee Technological University (TTU, 2020). The 15-question paper assessment booklet requires handwritten, primarily short-answer responses (Stein et al., 2007). The responses are scored by instructor teams using their rubric to assign points. Higher points are associated with evidence of more critical thinking displayed in the written responses.

Completion

Retention and graduation data were retrieved from the Office of Institutional Research at the university. Program admission required a GPA of 2.8 and between 20 and 60 credit hours. Retention at the university was defined as graduating or remaining enrolled at the institution as juxtaposed with leaving or transferring. Graduation rates for SRI participants were based on participants per sophomore year cohort of the year they participated in the SRI. Graduation rates for the university were based on first-time, full-time, degree-seeking freshman students for the freshman year of each cohort. For example, the freshman year of cohort 1 (2015) was Fall 2013.

Procedures

Questionnaires were administered on the first day (pretest) and last days (reflective and posttest) of the program for all cohorts. Reflective items asking students what they thought about their performance/knowledge/attitude in the beginning of the program, now that they had completed it, were collected directly after the same posttest questions. The CAT test was administered at the end of the SRI for cohort 1 (2015) but at the beginning and end of the SRI for cohorts 2 to 4. The student answer booklets were scored by MSU faculty and graduate students (some of whom were SRI instructors and near-peer mentors) trained in a CAT scoring workshop. The booklets subsequently were sent to TTU, which provided a results report of group means including national comparisons and the raw data used for further analysis. The survey questionnaires were administered electronically, and the CAT through a paper booklet.

All statistical analyses were conducted using IBM SPSS statistics software v. 24. Specific statistical tests are discussed together with the assessment for which they were conducted. Figures were generated using GraphPad Prism v. 5 with analogous statistical analyses. Institutional review board approval was obtained under the overall BUILD–ASCEND grant.

Data Analysis

For each year, we assessed within-cohort changes as gain (positive change from pre- to posttest) and, as applies, realistic gain (reflective to posttest) and realistic beginning (change from reflective to pretest). Realistic gain measures the difference of students’ perception of themselves at the beginning, now that they have completed the program, compared with their perceptions of themselves at the end, for each construct. The measure is more “realistic,” as it allows the students’ knowledge of a construct in the context of the program to inform their self-measure, as compared with imagining the construct’s meaning in the pretest. Realistic beginning is the difference between the gain and realistic gain. It measures the difference of the student’s pretest score without knowledge of the program to the reflective score, allowing the students to measure themselves more accurately on the construct as they reflect on a knowledge-informed construct. It represents the students’ initial estimation error. The constructs of interest included: science self-efficacy, science identity, academic adjustment to college, belongingness, academic self-concept, university contribution to research ability, and critical thinking. In addition, we determined the potency of each construct with pooled data across all cohorts. Finally, we evaluated changes between cohorts of different individuals on each of the measures that informed gain and relative gain scores in order to learn about the benefits of the specific measures across each cohort and the impact of program improvements. It is important to highlight that the iterative yearly program refinements are based on distinct cohorts and as such any changes may be the result of either program improvements or those inherent differences.

Analysis Plan.

The Wilcoxon signed ranks (W) test was used for gain scores (posttest minus pretest) and realistic gains (posttest minus reflective) to compare within-cohort paired sample data for each measure and to assess construct efficacy with pooled data across all cohorts. Each of the four cohorts was an independent sample; for between-cohort analyses, the Kruskal-Wallis H one-way analysis by ranks (KWH) was used as an analogue to the independent groups one-way analysis of variance (Siegel and Castellan, 1988). Difference scores were computed and used in the analyses of between-cohort constructs (see Supplemental Table A3). For each pair and omnibus test in which group average rank differences were found, the sample size, medians, range, adjusted p values, standardized z-test scores, computed effect sizes, and r values were reported (see Tables 2 and 3; Field, 2013, pp. 236–257).

Data Cleaning.

Before the analysis was conducted, student’s pre/post evaluation data were matched, examined for data entry accuracy, missing values, univariate and multivariate outliers, normality of distribution and fit for the test assumptions of multivariate analysis. Data were reviewed for each cohort across each of the pre/post evaluations of attitude and skills. Three negatively skewed outliers were adjusted to nearest larger score. No multivariate outliers were found. Homogeneity of variance, sphericity, and homogeneity of variance–covariance matrices assumptions were all violated. With parametric assumption violations, the corresponding nonparametric statistics were used for the assessment of changes within the constructs.

For analyses of all cohorts for each measure, summed values were created into indices for the constructs measured with more than one item or question (science self-efficacy, science identity, academic self-concept, belongingness, and critical thinking). Difference scores were created from the indexed data to assess change over program duration. The science self-efficacy index scores include three test measurements: pre, post, and Reflective. These difference scores were created for post minus pre- (gains) and for post minus Reflective (realistic gains). For cohort 1 (2015), only realistic gain scores were available, as the pretest was not administered. Cohort 4 (2018) also included Reflective questions for science identity. Difference scores were assessed for normality. Two difference scores were found to be extremely skewed and were adjusted to meet minimum normality requirements (Tabachnick and Fidell, 2007, p. 77). Homogeneity of variance, sphericity, and homogeneity of variance–covariance matrices assumptions were all violated. Because parametric assumptions were violated by the data, nonparametric Kruskal-Wallis independent one-way analysis by ranks was employed. A matched data set without corrections for outliers or normality was created and used for the analysis.

RESULTS

The data analyses within each cohort showed significant gains and realistic gains in science self-efficacy across cohorts. In addition, cohorts 3 and 4 expressed a greater ease with academic adjustment to college. Added science self-efficacy and realistic science identity questions asked of cohort 4 (2018) showed increased values for each question in both constructs.

The data analyses between cohorts show significantly lower science self-efficacy for cohort 1 (2015) compared with the other three cohorts. Finally, students in cohort 2 (2016) on average had less belief that the university contributed to their ability to do science than cohort 3 (2017). Low to moderate correlations between all constructs across cohorts are shown (see Table 4), except for critical thinking. The CAT showed no relationship with either of the other constructs.

Science Self-Efficacy

Overall, the students’ science self-efficacy began with a lower average median (Mdn = 57.0) compared with what they showed at the end (Mdn = 64.0); however, their perception of their science self-efficacy at the beginning of the program was even lower (Mdn = 53.0). Pooling data across all the cohorts, the science self-efficacy post score was significantly higher than the pre score; the reflective score was significantly lower than both the post and the pre scores (see Table 2).

Within-Cohort Assessment.

As illustrated in Figures 1 and 2, significant gains and realistic gains were found with the science self-efficacy measure for each cohort, including the additional questions for cohort 4 (2018; see also Table 3). The scale reliabilities Cronbach’s alpha for eight items (three and five additional items for cohort 4 additional questions) ranged from 0.56 to 0.89, with all but one reliability greater than 0.74. Excluded from this finding is the first cohort (2015), which was not administered a pretest for the science self-efficacy measure. Nonetheless, the students’ reflections to the beginning of the program and their posttest impressions showed significant gains and realistic gains.

Between-Cohort Assessment.

Omnibus testing of each factor across all cohorts determined significant differences for science self-efficacy, KWH (3) = 35.680, ρ < 0.000. Post hoc Bonferroni-corrected pairwise comparisons were conducted (see Supplemental Table A3). Three bivariate cohort combinations were found to have significantly different mean ranks. Cohort 1 (2015) had on average lower mean ranks than cohorts 2, 3, and 4 (2016, 2017, and 2018): between cohort 1 (2015) and 2 (2016), adj. ρ < 0.001, z = −3.671, r = −0.51; cohort 1 (2015) and 3 (2017), adj. ρ < 0.000, z = −5.058, r = −0.68; and cohort 1 (2015) and 4 (2018), adj. ρ < 0.000, z = −5.100, r = −0.73. Moderate correlations between all constructs across all cohorts are shown (see Table 4).

Science Identity

Science identity was not measured in the first three cohorts (1–3, 2015–2017). However, cohort 4 (2018) included measurement of several items representing a science identity construct (DPC CEC, n.d.).

Within-Cohort Assessment.

Science identity for cohort 4 (2018) showed significant gains with post to reflective comparisons (see Figure 3). Combined questions also showed significant gain, realistic beginning, and realistic gain (see also Table 3). The scale reliabilities Cronbach’s alpha for the three (pre) and five (post and reflective) items for cohort 4 questions ranged from 0.74 to 0.81.

Between-Cohort Assessment.

With only one cohort measurement, omnibus testing of each factor across all cohorts was not computed for science identity. Low correlations between constructs in cohort 4 (2018) were found to range from −0.116 with science self-efficacy to −0.238 with academic self-concept.

Academic Adjustment

Overall, the students began the program with a lower average median (Mdn = 10.0) than they showed at the end (Mdn = 12.0). Pooling data across all the cohorts, the post score was significantly higher than the pre score (see Table 2).

Within-Cohort Assessment.

Academic adjustment scores showed significant gains for cohorts 3 (2017) and 4 (2018; see Table 3). The scale reliabilities Cronbach’s alpha for the four items ranged from 0.25 to 0.78, with pretests from cohorts 3 and 4 below 0.65. Figure 4 shows academic adjustment comparisons of pre to post pairs by questions representing this construct.

Between-Cohort Assessment.

Omnibus testing of each factor across all cohorts determined significant differences for academic adjustment, KWH (2) = 19.868, ρ < 0.000 (see Supplemental Table A3). Post hoc Bonferroni-corrected pairwise comparisons were conducted. Two bivariate cohort combinations were found to have significantly different mean ranks. Cohort 2 (2016) had on average lower mean ranks than cohorts 3 and 4 (2017 and 2018): between cohort 2 (2016) and 3 (2017), adj. ρ < 0.000, z = −4.237, r = −0.59; and cohort 2 (2016) and 4 (2018), adj. ρ = 0.001, z = −3.306, r = −0.49. Small correlations between constructs across are shown (see Table 4).

Critical Thinking (CAT)

Overall, the students began the program with a lower average median (Mdn = 13.0) than they showed at the end (Mdn = 15.0). Pooling data across all the cohorts, the post scores were not significantly higher than the pre scores (see Table 2).

Within-Cohort Assessment.

CAT scores, which presented with a broad range, only showed significant gains in pre/post assessments in one cohort (cohort 3, 2017; see Figure 5 and Table 3). For cohort 1, we only had post scores. The scale reliabilities Cronbach’s alpha for the 15 items ranged from 0.11 to 0.65, with the lowest and highest reliabilities in pre- and posttest, respectively, for cohort 4. Construct differences were not seen for between-cohort assessment (see Supplemental Table A3). Low correlations between all constructs across are shown (see Table 4).

University Contribution to Doing Science

Overall, the students began the program with a lower average median (Mdn = 3.0) than they showed at the end (Mdn = 5.0). Pooling data across all the cohorts, the post score was significantly higher than the pre score with a large effect size (see Table 2).

Within-Cohort Assessment.

University contribution showed significant gains on the one question available across all cohorts. Unsurprisingly, students felt more strongly that the university contributed to their ability to conduct research at the end of the program compared with the beginning (see Table 3).

Between-Cohort Assessment.

Omnibus testing of each factor across all cohorts determined significant differences for university contribution to doing science, KWH (3) = 12.206, ρ = 0.007 (see Supplemental Table A3). Post hoc Bonferroni-corrected pairwise comparisons were conducted. One bivariate cohort combination had significantly different mean ranks. Cohort 2 (2016) had on average lower mean ranks than cohort 3 (2017), adj. ρ = 0.006, z = 3.287, r = 0.46. Low correlations between constructs across cohorts are shown (see Table 4).

Academic Self-Concept

Overall, the students began the program with the same average median (Mdn = 16.0) than they showed at the end (Mdn = 16.0; see Table 2). Construct differences were not seen for within-cohort assessment and between-cohort assessment (see Supplemental Table A3). Low to moderate correlations between constructs across are shown (see Table 4).

Belongingness

Overall, the students began the program with a slightly lower average median (Mdn = 20.0) than they showed at the end (Mdn = 21.0). Combined across all the cohorts, the post score was significantly higher than the pre score (see Table 2). Construct differences were not seen for within-cohort assessment and between-cohort assessment (see Supplemental Table A3). Low correlations between constructs across are shown (see Table 4).

Completion

For all four cohorts, 99% completed the program, and 83% continued in ASCEND and became ASCEND Scholars—the next training level. Overall retention at the university for program participants was 87% compared with 63% university-wide averaged over the 4 program years. Several students transferred to other universities. The 4-year SRI graduation rate¹ was 38% compared with the university 4-year graduation rate² of 19.8%, and the 6-year SRI graduation rate was 75% compared with 45% for the university.³ Overall, 97% of the SRI participants graduated.⁴ Of those who completed the SRI and have graduated over the four cohorts, an average of 97% continued in their sciences or social sciences majors.

DISCUSSION

Our long-term objective is to increase the number of students at MSU who successfully transition into graduate school and enter the health science research workforce. The current study was designed to ascertain whether an entrepreneurship-based interdisciplinary summer research training experience could increase students’ science self-efficacy and science identity and therewith retention for lower-division URM undergraduates in biomedical sciences. We assessed these parameters by individual cohort, across all cohorts, and also by the constructs themselves, using a cross-sectional and longitudinal design. The data shown here support our hypothesis that the BUILD–ASCEND SRI program—through student-centered activities, interdisciplinary group experiences, supported hands-on research experiences, and promoting entrepreneurial attributes in pursuing self-generated research questions—enhanced participants’ science self-efficacy and science identity (cohort 4, 2018) to sustain interest in biomedical research careers. In addition, our data show that the SRI experience increased students’ academic adjustment and sense of belonging, both factors likely related to increased retention and graduation rates.

Our results show robust increases in science self-efficacy as well as science identity for cohort 4. Science self-efficacy grew across the cohorts, as we fine-tuned our curriculum. Although recruited from the same demographics, each cohort had a somewhat unique analysis profile, illustrating the diversity among students’ prior experience and mindset. Retention in STEM and social/behavioral sciences disciplines and graduation rates of SRI participants across all cohorts was substantially greater than at the MSU overall. Our outcomes are consistent with models proposed by Chemers et al. (2001) and Frantz et al. (2017), wherein simultaneous increases in self-efficacy and science identity in research-based programs predict increased persistence in STEM fields at least through the baccalaureate degree. Our data are also consistent with observations by Wei and Woodin (2011) that interventions other than the typical apprenticeship-type training model can be very effective research training tools, particularly if they provide students with a sense of “responsibility and ownership.”

Science self-efficacy is associated with stamina in the face of aversion or disappointment, a willingness to cope and extend effort due to confidence, and an expectation of success in STEM and the social sciences and is predictive of persistence and completion in those fields (Bandura, 1977; Lent et al., 2005; Chemers et al., 2011; Estrada et al., 2011). It is noteworthy within this context that we have observed in a prior qualitative study that students struggled emotionally and in their attitudes in earlier phases of the SRI but showed elevated positive affect and thinking toward the conclusion of the summer experience (Jackson et al., 2018). Thus, gains may require such a struggle.

The student’s science self-efficacy consistently improved from pretest to posttest. Importantly, science self-efficacy showed substantial realistic gains between post- and reflective tests. It has often been shown that students initially overestimate their abilities and performance (Serra and DeMarree, 2016), especially in STEM courses (Lindsey and Nagel, 2015), which may explain the more significant p values and meaningful effect sizes and explained variance (see Table 3). These results are related to findings that prediction and anticipation of adhering to high academic expectations and rules in a demanding STEM major were related to high confidence or self-efficacy (Chemers et al., 2001; Lent et al., 2003, 2005). Additionally, Robbins et al. (2004) showed in a meta-analysis that academic self-efficacy was related to and a strong predictor of retention. However, Aronson, and Inzlicht (2004) showed that stereotype-vulnerable African Americans indicate, in real time, lower and fluctuating academic self-efficacy in areas of interest compared with less vulnerable African Americans and whites. Thus, introducing an opportunity for students to rate their abilities “before” relative to “after” the SRI training enables students to assess their skills and abilities reflectively and more accurately, based on a full understanding of the concepts.

In our study, science self-efficacy across all cohorts, measuring self-ability to perform tasks or skills, was moderately associated with academic self-concept, academic adjustment, belongingness, and the perception that the university contributes to the student’s ability to conduct research (see Table 4). Academic self-concept, measuring self-belief in academic ability, and drive to achieve; academic adjustment measured with adjusting to the demands of college and developing close friendships with other students; belongingness measured as feeling a sense of belonging, being a part of the college and the campus community, and feeling valued; and a willingness to recommend the school, all align in this self-selected and ambitious student population. These findings resonate with findings by Estrada et al. (2011) on the importance of a social system and connections for URMs in their integration into science.

Science identity in our study was not tested as robustly as science self-efficacy with using a validated science identity construct. Nevertheless, in this one cohort 4 (2018), the construct uncovered significant gains and realistic gains with large effect sizes in science identity in regard to “feeling of belonging to the science field and community, working on teams doing research, and thinking of themselves as a scientist” (see Table 3 and Figure 3). We surmise that measurement in earlier cohorts would have resulted in similar results.

Interestingly, science self-efficacy was not significantly correlated with science identity in our study. This would support the research on the tripartite integration model of social influence for STEM (Kelman, 2006), wherein science identity, as a distinct construct, accounts for persistence in STEM above science self-efficacy (Estrada et al., 2011; Hernandez et al., 2018a). However, interpretation of our data should be approached cautiously, because science identity as a construct did not receive a robust evaluation across all cohorts. With our STEM continuance and graduation rates, our results are aligned with other findings that a positive relationship exists between self-efficacy and pursuit of STEM careers (Estrada et al., 2011).

The SRI’s highly mentored, group-focused experiential training modules, incorporating entrepreneurial attributes, likely functioned to strengthen science identity. A review based on 60 published studies on undergraduate research experiences indicated that the students’ interaction with professors helped with their self-perception as scientists and recognition of their building skills, while the greater time spent with graduate students assisted with technical skill building (Linn et al., 2015). The interaction with five professors and five or more near-peer mentors in our program continually encouraged these types of activities and may have been a prominent factor in increased science identity. The SRI’s unique experiential structure allowed the students to be researchers with self-derived research questions over multiple well-defined modules and to generate a student-derived research project. Others have made similar observations using an entrepreneurial training model (Robinson et al., 2016).

The science identity construct for cohort 4 (2018) was moderately associated with academic self-concept. This study occurred at an historically black college or university (HBCU), and our findings may also parallel the Chang et al. (2016) conclusion that, for students with higher levels of domain association, lower levels of negative racial experiences are associated with better persistence. Finally, attending an HBCU may have also positively influenced the lack of a science identity gender effect. In previous studies, females were reported to perform lower than males (Lent et al., 2005; Frantz et al., 2017); this was not seen here, as the cohort percentage of females was more equivalent for cohort 4 (2018; see Table 1). Attending an HBCU thus may have had an impact. Comparisons with other institutional types in the future would provide more information.

Observed gains in academic adjustment items such as developing close friendships with other students, adjusting to college demands, developing effective study skills, and managing time effectively may be related to the program’s emphasis on group discussions and working in groups. Scores were significantly improved with large effect sizes for cohorts 3 and 4, as seen in Figure 3 and Table 3, which is likely a consequence of continued improvements and fine-tuning of our curriculum over the course of the 4 cohorts. As mentioned, we found moderate correlations with science self-efficacy; however, correlations were also seen with a recognition that the university contributed to the students’ ability to conduct research (see Table 4). The focus of the SRI on group processing of problems allowed the students to recognize and improve their own strengths, such as time management, and to explore other ways of handling problems through seeing and working with one another. This in turn may have resulted in participants seeing the university as assisting them with conducting research while increasing their feelings of belonging to the campus and community.

Because the SRI curriculum emphasizes problem solving, creative thinking, and effective communication, it comes as somewhat of a surprise that only one cohort showed significant gains in the CAT. Skills emphasized by the SRI are those also assessed in the CAT (Holmes et al., 2015; Tennessee Tech University, 2020). Most likely, the duration of our SRI was too short. While some studies have found improvements using the CAT assessment over a semester-long course (Gasper and Gardner, 2013; Carson, 2015; Rowe et al., 2015), most either did not see gains or showed mixed overall results (Frisch et al., 2013; Cargas et al., 2017; Perry et al., 2018). Improvement in CAT critical-thinking skills even across a full semester is uncommon (Stein et al., 2007; Rowe et al., 2015) and is typically facilitated when interventions intentionally teach and provide practice in particular skill building (Niu et al., 2013; Bensley and Spero, 2014; Abrami et al., 2015; Tiruneh et al., 2018; Halpern and Butler, 2019). While our program employed encouragement of critical thinking through its intense focus on research question development and experiential research activities, we did not focus intentionally on specific skills required in the CAT assessment.

Academic self-concept and belongingness items on our assessment tools are employed to measure broad terms relating to the larger university community. Academic self-concept included items asking comparisons of self to same-age others on “academic and mathematical ability”, as well as “drive to achieve,” and “intellectual self-confidence.” These constructs would not be expected to change substantially during an 8-week research introduction program. Similarly, the items representing “belongingness” were related to the campus or college globally versus a specific program. Both sets of these questions were derived from the Diversity Program Consortium items, whose broader focus across vastly different campus types were not informed specifically by an HBCU campus population and institutional mission or the impact of a specific program or intervention.

Limitations

This study has several limitations. The ASCEND program invited applications from the student body and in subsequent years from surrounding community colleges for students attending or intending to attend the university. The requirements included a certain GPA (2.8), which is slightly higher than average in relevant majors at the institution. Other requirements included writing several prompted statements, along with obtaining two letters of recommendation. Recruiting freshmen with these characteristics and abilities meant the sample was subject to self-selection and skewed to higher-functioning and higher-performing students. Further, the recruitment pool and subsequent SRI participants included a preponderance of females, which was higher than the overall percentage of females at the university but roughly reflected the female to male ratio in the students’ majors. This made it difficult to ascertain whether there might have been a male/female performance gap, as previously reported in the literature. Increased male participation would provide increased statistical power to afford the circumstances to show differences.

While we did make substantial curricular changes based on student feedback between the different cohorts, our interpretation that these improvements contributed to the growth in science identity and self-efficacy across cohorts cannot be clearly assessed. Each of the four cohorts included different individuals, such that the between-cohort analyses may reflect differences of the students more than perceived program refinements.

Future replications of our SRI training model will need to show whether the impact seen here on science self-efficacy and science identity in URM STEM and social/behavioral sciences students is transferable to other institutions, particularly non–minority serving institutions. This study was conducted at an HBCU in the mid-Atlantic region with greater than 90% of the student population being of African or African-American descent, and the majority of the instructors and near-peer mentors were from the same demographics as the students. As alluded to earlier, it is possible that the particular environment of an HBCU campus was instrumental to our success.

CONCLUSION

This 4-year study has provided proof of concept that our entrepreneurial research training program can be highly successful in increasing science self-efficacy, science identity, and retention in science majors for URMs. The ASCEND SRI is taught using an approach more akin to course-based undergraduate research than the typical SRI program (Auchincloss et al., 2014; Shapiro et al., 2015; Elgin et al., 2016; Staub et al., 2016; Frantz et al., 2017). As detailed earlier, our data show robust increases in science self-efficacy, science identity, and academic self-adjustment, all of which are strong predictors of student success in college and subsequent retention into science careers and graduate training. Our observations concur with those Frantz et al. (2017) that an entrepreneurial, group-based training model can instill the same gains typically associated with an apprenticeship-type undergraduate research experience (Seymour et al., 2004; Linn et al., 2015; Shapiro et al., 2015; Elgin et al., 2016; Staub et al., 2016; Frantz et al., 2017). However, unlike the traditional apprenticeship model, our ASCEND training model maximizes student participation while decreasing impacts on mentor resources, particularly those in a mentor-to-mentee ratio. This approach makes the training model scalable for institutions with limited faculty-mentored research experiences.

Our specific curriculum is transferable to other institutions seeking to encourage greater access to research experiences and opportunities for URMs at colleges and universities, and we would welcome the communication of any institution with interest in doing so. Moreover, the interdisciplinary nature of our curriculum and focus on cross-discipline group interactions and communication builds essential skills for Team Science in research and entrepreneurial organizations alike (Hisrich, 1990; Heinzen et al., 2018).

FOOTNOTES

ACKNOWLEDGMENTS

REFERENCES