Outcomes and Processes in the Meyerhoff Scholars Program: STEM PhD Completion, Sense of Community, Perceived Program Benefit, Science Identity, and Research Self-Efficacy
Abstract
Previous research has shown that the Meyerhoff Scholars Program at the University of Maryland, Baltimore County, is an effective intervention for high-achieving underrepresented minority (URM) students; African-American Meyerhoff students are significantly more likely to enter science, technology, engineering, and mathematics (STEM) PhD programs than comparison students. The first of two studies in this report extends the prior research by examining levels of PhD completion for Meyerhoff (N = 479) versus comparison sample (N = 249) students among the first 16 cohorts. Entering African-American Meyerhoff students were 4.8 times more likely to complete STEM PhDs than comparison sample students. To enhance understanding of potential mechanisms of influence, the second study used data from the 22nd (Fall 2010) to 25th (Fall 2013) cohorts (N = 109) to test the hypothesis that perceived program benefit at the end of freshman year would mediate the relationship between sense of community at the end of Summer Bridge and science identity and research self-efficacy at the end of sophomore year. Study 2 results indicated that perceived program benefit fully mediated the relationship between sense of community and both criterion measures. The findings underscore the potential of comprehensive STEM intervention programs to enhance PhD completion, and suggest mechanisms of influence.
INTRODUCTION
In recent years, educators, researchers, and policy makers have dedicated concerted effort to increase the numbers of underrepresented minorities (URMs) in the fields of science, technology, engineering, and mathematics (STEM). Since the early 1990s, URMs’ (e.g., African-American, Latino/a, American Indian, or Alaska Native students) share of science and engineering bachelor’s and master’s degrees has increased, although their share of doctorates has remained flat (National Science Foundation [NSF], 2015). African Americans, the URM group of primary focus in the current article, make up 13.3% of the U.S. population (U.S. Census Bureau, 2015). However, only 5% of those who earned doctoral degrees in STEM in the United States are African American (NSF, 2015).
Given the national emphasis on innovation in STEM, the production of a diverse body of STEM researchers (i.e., PhDs) to contribute to the scientific workforce in the United States is of utmost importance (National Academy of Sciences, National Academy of Engineering, and National Academy of Mathematics, 2007). Increasing the numbers of African Americans with a sustained interest in STEM is one way to meet this goal (Maton et al., 2012). Indeed, STEM education is necessary for ensuring progress in national science, technology, and economic development (National Academy of Sciences, National Academy of Engineering, and National Academy of Mathematics, 2007; Pender et al., 2010). Yet, in their pursuit of a STEM PhD, URM STEM majors disproportionately face historical and institutional barriers to academic success, including social and cultural isolation, high financial need, a lack of academic preparation, stereotype threat, an overwhelming course load, and insufficient career information (Astin, 1990; Seymour and Hewitt, 1997; Steele, 1997; Wilson, 2000). Therefore, the provision of targeted supports for URMs in STEM is key to increasing their representation in STEM fields.
Inclusive Excellence in STEM: Intervention Programs to Increase African-American Student Success
Research on precollege students related to parity for URMs in STEM has contributed to the development of college-level STEM interventions on campuses across the United States. These STEM intervention programs attempt to counteract historical and institutional barriers faced by students of color, including African-American students, mostly at predominantly white institutions. The majority of current programs aim to increase access, retention, persistence, and completion of STEM degrees among URMs (Maton et al., 2009, 2015). Model STEM interventions provide a combination of financial support, academic services, research opportunities, mentorship, and support groups to students (Pender et al., 2010; Maton et al., 2015).
Research has demonstrated that STEM interventions are effective at increasing retention and graduation rates among African-American students (Reichert and Absher, 1998; American Society for Engineering Education, 2007; Maton et al., 2015). Specifically, these programs effectively increase minority student interest in STEM, academic performance, undergraduate degree completion, and graduate school entrance (Maton et al., 2000; Barlow and Villarejo, 2004; Maton and Hrabowski, 2004; Alfred et al., 2005; Merolla and Serpe, 2013). While it is clear that these programs contribute to the increasing numbers of URMs in STEM, less is known about how these undergraduate STEM interventions impact URM completion of the STEM PhD. Thus, in the current article, STEM PhD completion by African-American students and one specific mechanism of influence will be explored within a model STEM intervention, the Meyerhoff Scholars Program.
Meyerhoff Scholars Program
The Meyerhoff Scholars Program (MSP) at the University of Maryland, Baltimore County (UMBC), is a multicomponent program aimed at increasing the representation and academic achievement of talented URMs in STEM fields. Instituted in 1988, the MSP was developed with the purpose of increasing the number of African-American males who pursued and excelled in STEM fields. After the first year, the program was opened up to African-American females, and in 1996, students of all backgrounds who were committed to increasing the representation of underrepresented students were admitted (Bridglall et al., 2013; Hrabowski, 2015). The MSP aims to counteract barriers to academic success in STEM for high-achieving URMs through a number of specific supports, including the Summer Bridge (a 6-week intensive STEM training program), study groups, a community of current and former STEM scholars to provide social and academic support, staff counseling and academic advising, and research opportunities. MSP program elements address four domains that are important to minority student success in STEM: 1) academic and social integration, 2) knowledge and skill development, 3) support and motivation, and 4) monitoring and advising (cf. Maton and Hrabowski, 2004; Stolle-McAllister et al., 2011; Hrabowski, 2015). The primary goal of the MSP is to increase the number of high-achieving underrepresented students who earn PhDs or MD/PhDs in STEM fields, through building a sense of community and strengthening students’ STEM identity and STEM research self-efficacy (Carter et al., 2009). Comprehensive STEM interventions like the MSP are increasingly recognized as having an especially high impact on enhancing the number of URMs in STEM fields (Kuh et al., 1991; Cole, 1995; Thompson et al., 2001; Millspaugh and Millenbah, 2004; Randall et al., 2004; Maton et al., 2009).
STUDY 1. STEM PHD COMPLETION
Previous evaluation studies of the MSP have examined program outcomes, components of the program, and psychosocial predictors of outcomes. In terms of outcomes, since 2000, studies have focused on comparing the academic success of Meyerhoff students with “declined” students (students who applied to the MSP, but declined the offer in order to pursue an undergraduate degree at a different institution; Maton et al., 2015). Findings show that African-American MSP students are twice as likely to graduate with a STEM bachelor’s degree than the declined comparison sample of similarly prepared students (Maton et al., 2000). Furthermore, MSP has positive effects on entrance into STEM PhD programs (Maton et al., 2000, 2009, 2012, 2015). Specifically, African-American MSP students are five times more likely to pursue a STEM PhD compared with the declined sample (Maton et al., 2009, 2012).
Whereas previous research has demonstrated the effectiveness of the MSP on STEM bachelor’s degree completion and entrance into STEM PhD programs, the current paper extends this line of research by examining, for the first time, the effect of the MSP on STEM PhD completion. Specifically, study 1 seeks to answer the question: Are African-American Meyerhoff students more likely to complete a STEM PhD than the comparison sample, controlling for covariates?
Methods
Research Participants.
The initial study sample consists of African-American students in the first 16 cohorts entering the MSP (entering 1989–2004) and African-American “declined” students (students who were offered the Meyerhoff scholarship between 1989–2004 but declined and attended other institutions). There are a total of 793 students included in these cohorts. Of these 793, 29 students were still in a STEM PhD program (including students completing their PhDs as part of their MD/PhDs), and the STEM PhD status of 36 students was unknown. These 65 cases were excluded from the analysis, leaving 728 students as the primary sample. Of the 36 students whose STEM PhD status was unknown, 25 were Meyerhoff students and 11 “declined” sample students, accounting for 4.7% and 4.2% of the original Meyerhoff and “declined” samples, respectively.
Table 1 presents characteristics of the Meyerhoff (N = 479) and declined (N = 249) samples. Preliminary analyses indicated there were more male students in the Meyerhoff (50.5%) than the declined (39.7%) sample (χ2(1) = 8.180, p < 0.05). In addition, Meyerhoff students had lower SAT Verbal scores (mean = 623.39) than declined students (mean = 638.34; t(785) = 3.235, p < 0.01). There were no significant differences between the Meyerhoff and declined samples on SAT Math and high school grade point average (GPA).
| Original sample | Excluded from sample | Final sample | Male (%) | SAT Math (M) | SAT Math (SD) | SAT Verbal (M) | SAT Verbal (SD) | High School GPA (M) | High School GPA (SD) | |
|---|---|---|---|---|---|---|---|---|---|---|
| Meyerhoff students | 533 | Still enrolled: 29 | 479 | 50.5* | 648.59 | 45.60 | 623.39** | 58.76 | 3.70 | 0.34 |
| Unknown: 25 | ||||||||||
| Declined students | 260 | Unknown: 11 | 249 | 39.7 | 658.83 | 50.20 | 638.34 | 63.50 | 3.77 | 0.36 |
| Total | 793 | 65 | 728 |
Procedure.
STEM PhD completion was obtained using multiple sources: transcripts, Meyerhoff office records, and/or Internet searches (university, social media, and professional websites). Meyerhoff and declined students were asked to sign informed consent forms (or assent forms if under 18) when they applied to the program (for students under 18, parents or guardians also signed consent forms). Degree completion data found through Internet searches were verified by comparing at least two sources.
The first research question explored the effectiveness of the MSP on STEM PhD completion outcomes. Logistic regression analyses were conducted for the full study sample to determine whether the observed differences in STEM degree completion achieved statistical significance. Covariates included in the analyses were high school GPA, SAT Math scores, SAT Verbal scores, gender, and cohort. A secondary purpose was to examine trends in STEM PhD outcomes over time, particularly how more recent cohorts differ from earlier ones. To this latter end, the sample was divided evenly into two subgroups: cohorts that entered the MSP between 1989–1996 and 1997–2004, respectively. Although students were nested within the 16 cohorts, multilevel modeling was not used in this study, because 50 or fewer clusters lead to biased parameter estimates in multilevel analysis (Maas and Hox, 2005).
Results
As indicated in the final two columns of Table 2, African-American Meyerhoff students in the 1989–2004 entering cohorts overall were 4.8 times more likely to complete their STEM PhDs (28.6% of entering students) than declined sample students (6.0%). Logistic regression analysis revealed that the 1989–2004 entering African-American Meyerhoff students were significantly more likely to complete a STEM PhD compared with the declined student comparison group, after controlling for covariates (Table 3; odds ratio [OR] = 7.5, Wald [df = 1] = 45.3, p < 0.001). The OR indicates that African-American Meyerhoff students were 7.5 times more likely to earn a STEM PhD degree with covariates controlled; the Wald statistic tests its significance vis-à-vis the true value of the parameter based on the sample estimate.
| 1989–1996 | 1997–2004 | 1989–2004 | |||
|---|---|---|---|---|---|
| Meyerhoff | Declined | Meyerhoff | Declined | Meyerhoff | Declined |
| 21.9% | 5.6% | 36.5% | 6.4% | 28.6% | 6.0% |
| 57/260 | 6/108 | 80/219 | 9/141 | 137/479 | 15/249 |
| Parameter estimate (B) | SE | Wald | p Value | OR | |
|---|---|---|---|---|---|
| Intercept | 6.885 | 1.882 | 13.377 | 0.000 | |
| SAT Math | −0.005 | 0.002 | 4.474 | 0.034 | 0.99 |
| SAT Verbal | −0.001 | 0.002 | 0.133 | 0.716 | 0.99 |
| High school GPA | −0.586 | 0.295 | 3.947 | 0.047 | 0.56 |
| Cohort | −0.052 | 0.025 | 4.492 | 0.034 | 0.95 |
| Gender (women) | 0.136 | 0.204 | 0.445 | 0.505 | 1.15 |
| Meyerhoff | 2.018 | 0.3 | 45.265 | 0.000 | 7.52 |
The first four columns of Table 2 provide trend data for STEM PhD receipt. Visual inspection of these data indicate that African-American Meyerhoff students from the more recent cohorts (1997–2004) were more likely to complete a STEM PhD (36.5%) compared with the earlier Meyerhoff cohorts (1989–1996; 21.9%).
Study 1 provides evidence for the long-term impact of the MSP on participants’ academic success in STEM. Less is known, however, about the mechanisms of influence underlying this finding. Previous research has shown that psychosocial factors such as science identity and research self-efficacy are predictive of increased persistence and academic achievement in STEM (e.g., Lent et al., 1986, 2008; Marra et al., 2009; Estrada et al., 2011; Adedokun et al., 2013; Tate et al., 2014; Larson et al., 2015). Yet, more research is needed to explore how comprehensive STEM interventions programs, like the MSP, promote and strengthen these factors. Study 2 examines whether perceived program benefit mediates the relation between psychological sense of community (a central component to the MSP) and science identity and research self-efficacy.
STUDY 2. MECHANISMS OF INFLUENCE: PSYCHOLOGICAL SENSE OF COMMUNITY, PERCEIVED PROGRAM BENEFIT, SCIENCE IDENTITY, AND RESEARCH SELF-EFFICACY
There is a pressing need for theory-driven research on mechanisms underlying the effectiveness of STEM interventions (Olson and Fagen, 2007; DePass and Chubin, 2008; Chubin et al., 2009; Chubin and DePass, 2012; Bowman and St. John, 2011; Byars-Winston et al., 2011). Within the MSP, a psychological sense of community is among the earliest program elements introduced to students (i.e., during a 6-week intensive STEM training program before the first Fall semester of college: Summer Bridge), and it is fostered throughout students’ college tenure and beyond college completion. Psychological sense of community encompasses one’s sense of belonging to a community as well as one’s relationships with community members. Sense of belonging to STEM intervention programs is a key contributor to success in STEM for URMs (Mondisa and McComb, 2015). More broadly, McMillan and Chavis’ (1986) theory of psychological sense of community argues that a sense of belonging, mattering, and integration within a community is an important social need.
For URMs, negative perceptions of campus racial/ethnic climate have been linked to diminished sense of belonging and low integration to the campus (e.g., Hurtado and Carter, 1997; Johnson et al., 2007; Johnson, 2012; Strayhorn, 2015) and lack of persistence (e.g., Johnson et al., 2007; Museus et al., 2008). The MSP is designed to improve perceived racial/ethnic climate, sense of community, and campus integration more generally by providing URMs with a supportive learning environment through social support and science activities.
Central to the current study is the expectation that students with a stronger sense of program community can be expected to more fully adopt key MSP values (e.g., academic excellence; commitment to pursue a STEM PhD), engage in the full range of program components and activities, and fully benefit from the program as a whole. Sense of community developed during Summer Bridge is hypothesized to contribute to maximal program engagement and benefit, which, in turn, can be expected to lead to enhanced STEM outcomes, including science identity (seeing oneself as a scientist) and research self-efficacy (believing that one can do the work of a scientist). Science identity and research self-efficacy are important short-term, psychosocial criterion variables to consider given evidence, in turn, that they are predictive of increased persistence and success in STEM (e.g., Lent et al., 1986, 2008; Marra et al., 2009; Estrada et al., 2011; Adedokun et al., 2013; Tate et al., 2014; Larson et al., 2015).
Identity theory (Burke and Stets, 2009; Stryker, 1980; Stryker et al., 2005) postulates that social structures help shape the development of one’s identities for different social roles. The MSP is designed to serve as a social structure for participants to develop science identity by integrating students into a scientific community that values and promotes engagement in scientific scholarship and research. Cumulative evidence suggests that research experience helps students to build up their science identity (e.g., Junge et al., 2010; Grunert and Bodner, 2011; Schultz et al., 2011; Thiry et al., 2011; Strawn and Livelybrooks, 2012; Adedokun et al., 2013; Merolla and Serpe, 2013). In turn, science identity is repeatedly found to predict STEM persistence and achievement (e.g., Carlone and Johnson, 2007; Malone and Barabino, 2009; Espinosa, 2011; Estrada et al., 2011; Eagan et al., 2013; Hernandez et al., 2013; Krogh and Andersen, 2013; Chang et al., 2014). This study hypothesizes that stronger psychological sense of community will predict stronger science identity for URMs via the mediating role of perceived benefit from program involvement.
Self-efficacy refers to personal judgments of one’s capabilities to execute behavior to attain certain goals. Bandura’s (1986, 1997) social cognitive theory posits that individuals’ domain-specific self-efficacy beliefs are shaped by their experiences and strongly determine behavior regarding that domain. Social cognitive career theory (Lent et al., 1994, 1996) further posits that career interests and choices are determined to a great extent by ones’ self-efficacy as well as environmental supports. STEM intervention programs can provide a supportive environment to foster students’ self-efficacy and enhance academic achievement through efforts to enhance student integration within a community of scientists. Previous research has consistently shown that learning environments that focus on research and social support increase research/scientific self-efficacy (e.g., Hurtado et al., 2009; MacPhee et al., 2013), which in turn is predictive of STEM degree completion and graduate school entry (e.g., Lent et al., 1986, 2008; Marra et al., 2009; Estrada et al., 2011; Adedokun et al., 2013; Tate et al., 2014; Larson et al., 2015). As such, it is hypothesized that efforts to promote a stronger psychological sense of community within the MSP will predict stronger research self-efficacy among participants, via the mediating role of perceived benefit from program involvement.
In summary, study 2 examines the hypothesis that perceived program benefit (assessed at the end of freshman year) fully mediates the relation between psychological sense of community (assessed at the end of Summer Bridge) and science identity and research self-efficacy (assessed at the end of sophomore year). This mediational model will be examined using cohort, gender, high school GPA, and SAT scores as covariates.
Methods
Research Participants.
The study sample comprises 109 students who entered the MSP in Fall 2010, Fall 2011, Fall 2012, and Fall 2013. The sample contains more men (N = 63, 57.8%) than women (N = 46, 42.2%). More than two-fifths are African American/black (N = 48, 44%), one-quarter Asian/Pacific Islander (N = 26, 23.9%), slightly more than one-fifth white (N = 23 21.1%), and the remainder Hispanic (N = 12, 11%). The average SAT scores are: 696.2, Math; 628.4, Verbal, and 632.9, Writing. The mean weighted high school GPA is 4.1.
Procedure.
The psychological sense of community scale was administered along with a set of other scales at the end of the 6-week Summer Bridge. The perceived benefit of program elements items were administered as part of a larger online survey at the end of freshman year. The science identity and research self-efficacy scales were administered along with other scales at the end of sophomore year. All students (or their parents or guardians) completed consent forms when they applied to the MSP.
Measures
Sense of Community.
Sense of community in the MSP at the end of the 6-week Summer Bridge program was measured by 12 items adapted from the original Sense of Community Index (Chavis et al., 2008), which was developed based on McMillan and Chavis’ (1986) theory of psychological sense of belonging. Sample items include “I can trust people in the program” and “I expect to be a part of the program for a long time.” Scale scores were calculated by taking the mean of all items completed. Reliability of this measure has been found to be high (Chavis et al., 2008), including in the current sample (Cronbach’s alpha = 0.82).
Perceived Program Benefit.
Perceived benefit of MSP elements at the end of year 1 was measured by 17 items developed by the research team that ask students to assess “how useful” to them each of 17 program elements are. These include: financial aid, study groups, tutoring, academic advising, personal counseling from staff, mentoring or support, organized social activities, faculty involvement, interactions with UMBC administrators, Meyerhoff community, and professional development. Scale scores were calculated by taking the mean of all items completed. This measure was developed for the study and is intended to assess student perceptions of the level of benefit of program involvement; as such, it serves as a measure of subjective program impact. The scale is highly reliable (Cronbach’s alpha = 0.87).
Science Identity.
Science identity at year 2 was measured by five items from the Scientific Identity Scale (Chemers et al., 2010) that ask students to assess how much being a scientist is viewed as part of who they are. Sample items include “have a strong sense of belonging to the community of scientists,” “feel like I belong in the field of science,” and “have come to think of myself as a scientist.” Scale scores were calculated by taking the mean of all items completed. High reliability was reported in previous studies (Chemers et al., 2010; Estrada et al., 2011).
Research Self-Efficacy.
Research self-efficacy at year 2 was measured by six items from the Scientific Self-Efficacy Scale (Chemers et al., 2010) that assess students’ ability to function as scientists. Sample items include “use technical science skills,” “generate a research question,” and “use scientific literature and/or reports to guide research and develop theories.” Scale scores were calculated by taking the mean of all items completed. High reliability was reported in previous studies (Chemers et al., 2010; Estrada et al., 2011).
Covariates.
Covariates were all obtained from MSP records. These include SAT scores (Math, Verbal, and Writing), weighted high school GPA, gender, ethnicity (African American/black, Asian American, Hispanic, white), and cohort (year of college entry).
Stepwise multiple regression was used to test the mediational role of perceived program benefit on the relation between sense of community and science identity and research self-efficacy, respectively. Consistent with Baron and Kenny (1986), full mediation requires 1) a direct relation between the exogenous predictor (sense of community) and the theorized mediator (perceived program benefit); 2) a direct relation between the exogenous predictor (sense of community) and the two outcome measures (science identity and research self-efficacy); and 3) an effect of the proposed mediator (perceived program benefit) on the outcomes (science identity, research self-efficacy), controlling for the exogenous predictor (sense of community). The covariates were entered as a group in the first step of each regression analysis.
The design effect for the two outcomes of interest was smaller than 2 (science identity 1.77; research self-efficacy 1.57). Thus, multilevel modeling (individuals nested within cohorts) is not necessary (Satorra and Muthen, 1995; Maas and Hox, 2005).
Results
Table 4 provides descriptive information for continuous study variables for cohorts entering in 2010 through 2013. For the sample as a whole, mean scores for primary study variables were relatively high: 3.5 (max = 4) for sense of community at end of Summer Bridge; 3.4 (max = 4) for perceived program benefit at end of freshman year; 4.1 (max = 5) for science identity at end of sophomore year; and 3.8 (max = 5) for research self-efficacy at end of sophomore year.
| Minimum | Maximum | Mean | SD | Scale range | |
|---|---|---|---|---|---|
| SAT Math | 600 | 800 | 696.2 | 47.53 | 200–800 |
| SAT Verbal | 430 | 790 | 628.4 | 68.20 | 200–800 |
| SAT Writing | 440 | 800 | 632.9 | 79.41 | 200–800 |
| High school GPA (weighted) | 3.2 | 4.6 | 4.1 | 0.30 | 0–5 |
| Cohort (year of entry) | 2010 | 2013 | 2011.8 | 0.95 | n/a |
| Sense of community (Summer Bridge)a | 1.8 | 4.0 | 3.5 | 0.39 | 1–4 |
| Perceived benefit of program elements (freshman year)b | 1.7 | 4.0 | 3.4 | 0.49 | 1–4 |
| Science identity (sophomore year)c | 2.2 | 5.0 | 4.1 | 0.64 | 1–5 |
| Research self-efficacy (sophomore year)c | 2.0 | 5.0 | 3.8 | 0.68 | 1–5 |
Table 5 provides the intercorrelations among all continuous study variables. SAT Math was positively and significantly related to science identity (r = 0.25) and research self-efficacy (r = 0.22) at sophomore year. None of the other covariates were significantly related to any of the psychosocial variables. As expected, sense of community at the end of Summer Bridge was positively and significantly related to perceived program benefit at end of freshman year (r = 0.40), and to a lesser extent to science identity (r = 0.29) and research self-efficacy (r = 0.26) at the end of sophomore year. Also as expected, perceived program benefit was positively and significantly related to both science identity (r = 0.35) and research self-efficacy (r = 0.33) at the end of sophomore year.
| SAT Math | SAT Verbal | SAT Writing | High School GPA | Gender (women) | Cohort | Sense of community (Summer Bridge) | Perceived program benefit (freshman year) | Science identity (sophomore year) | Research self-efficacy (sophomore year) | |
|---|---|---|---|---|---|---|---|---|---|---|
| SAT Math | — | |||||||||
| SAT Verbal | 0.27* | — | ||||||||
| SAT Writing | 0.37** | 0.58* | — | |||||||
| High school GPA | 0.17 | 0.04 | 0.32** | — | ||||||
| Gender (women) | −0.09 | 0.19 | 0.39** | 0.23* | — | |||||
| Cohort | −0.08 | −0.12 | −0.14 | 0.02 | 0.00 | — | ||||
| Sense of community (Summer Bridge) | 0.00 | −0.03 | 0.09 | −0.02 | 0.07 | 0.03 | — | |||
| Perceived program benefit (freshman year) | −0.03 | −0.19 | 0.04 | 0.00 | 0.05 | −0.02 | 0.40** | — | ||
| Science identity (sophomore year) | 0.25** | −0.11 | 0.06 | 0.07 | −0.07 | −0.03 | 0.29** | 0.35** | — | |
| Research self-efficacy (sophomore year) | 0.22* | 0.09 | 0.13 | 0.15 | −0.15 | −0.14 | 0.26** | 0.33** | 0.52** | — |
Sense of community at the end of Summer Bridge was positively related to the mediator, perceived program benefit at the end of freshman year (beta = 0.38, p < 0.01). Sense of community explained a significant 13.9% of the variance in perceived program benefit beyond that explained by the covariates, F change (1, 98) = 17.82, p < 0.01 (Table 6).
| Predictor variables | Model 1: step R2 change | Model 2: final equation beta | Model 2: final step R2 change |
|---|---|---|---|
| Step 1. covariates | 0.097 | — | — |
| Step 2. sense of community (Summer Bridge) | — | 0.38** | 0.139** |
Sense of community at the end of Summer Bridge was positively related to the first outcome variable (beta = 0.24), explaining a significant 5.6% of the variance in science identity beyond that explained by the covariates (Table 7, second row, second and third columns), F change (1, 98) = 7.32, p < 0.01. Sense of community was also positively related to the second outcome variable (beta = 0.26), explaining a significant 6.4% of the variance in research self-efficacy beyond that explained by the covariates (Table 8, second row, second and third columns), F change (1, 98) = 8.05, p < 0.01.
| Predictor variables | Model 1: step R2 change | Model 2: final equation Beta | Model 2: final step R2 change | Model 3: final equation beta | Model 3: final step R2 change |
|---|---|---|---|---|---|
| Step 1: covariates | 0.198** | — | — | — | |
| Step 2: sense of community (Summer Bridge) | — | 0.24** | 0.056** | 0.12 | — |
| Step 3: perceived program benefit (freshman year) | — | 0.31** | 0.072** |
| Predictor variables | Model 1: step R2 change | Model 2: final equation beta | Model 2: final step R2 change | Model 3: final equation betas | Model 3: final step R2 change |
|---|---|---|---|---|---|
| Step 1: covariates | 0.157* | — | — | — | |
| Step 2: sense of community (Summer Bridge) | — | 0.26** | 0.064** | 0.13 | — |
| Step 3: perceived program benefit (freshman year) | — | — | — | 0.33** | 0.084** |
Perceived program benefit was positively related to the first outcome variable (beta = 0.31), explaining a significant 7.2% of the variance in science identity beyond that explained by the covariates and by sense of community (Table 7, third row, last two columns), F change (1, 97) = 10.40, p < 0.01. Furthermore, sense of community was no longer significantly related to science identity (beta = 0.12, ns) with perceived program benefit entered (Table 7, second row, fourth column), suggesting full mediation by perceived program benefit of the relation between sense of community and science identity.
Perceived program benefit was also positively related to the second outcome variable (beta = 0.33), explaining a significant 8.4% of the variance in research self-efficacy beyond that explained by the covariates and by sense of community (Table 8, third row, last two columns), F change (1, 97) = 11.76, p < 0.01. Furthermore, sense of community was no longer significantly related to research self-efficacy (beta = 0.13, ns) with perceived program benefit entered (Table 8, second row, fourth column), suggesting full mediation by perceived program benefit of the relation between sense of community and research self-efficacy.
Figures 1 and 2 depict the mediational models for science identity and research self-efficacy, respectively.
Figure 1. Mediational model for science identity (sophomore year). The model demonstrates that, consistent with the expectations of mediation, the exogenous predictor (sense of community) significantly predicts both the outcome (science identity; beta = 0.24, p < 0.01) and the mediator (perceived program benefit; beta = 0.38, p < 0.01). Additionally, the proposed mediator significantly predicts the outcome (beta = 0.31, p < 0.01). Furthermore, the relation between sense of community and science identity no longer exists in the presence of perceived program benefit (beta = 0.12, ns).
Figure 2. Mediational model for research self-efficacy (sophomore year). The model demonstrates that, consistent with the expectations of mediation, the exogenous predictor (sense of community) significantly predicts both the outcome (research self-efficacy; beta = 0.26, p < 0.01) and the proposed mediator (perceived program benefit; beta = 0.38, p < 0.01). Additionally, the proposed mediator significantly predicts the outcome (beta = 0.33, p < 0.01). Furthermore, the relation between sense of community and research self-efficacy no longer exists in the presence of perceived program benefit (beta = 0.12, ns).
DISCUSSION
The results of the two studies move forward our understanding of the longer-term impact and mechanisms of influence of comprehensive STEM intervention programs. African-American Meyerhoff students were significantly more likely to enter and complete STEM PhD programs than comparison students. Perceived program benefit at the end of freshman year fully mediated the relation between sense of community at the end of Summer Bridge and science identity and research self-efficacy, respectively, at the end of sophomore year. The findings from both studies are discussed below, along with limitations and directions for future research.
Program Outcomes
The current findings support and extend earlier published findings of positive MSP impact on STEM PhD entry (e.g., Maton et al., 2012) and, more generally, positive outcomes from related programs (Carmichael et al., 1993; Barlow and Villarejo, 2004). Meyerhoff students who entered the program in 1989–2004 were more likely to earn STEM PhD degrees than comparison sample students who were accepted into the program but declined the offer and initiated STEM course work elsewhere. This is the first study in which STEM PhD completion was examined. The Meyerhoff students did not have higher SAT scores or high school GPAs than comparison students, suggesting that their higher levels of STEM PhD pursuit and completion were not due to greater precollege preparation or capability. Also, both Meyerhoff students and comparison students pursued STEM course work as freshmen, suggesting that the Meyerhoff and declined students entered college with comparable interest in STEM. Nonetheless, in the absence of random assignment, it cannot be ruled out that the groups differed initially, at least to some extent, in other characteristics (e.g., research excitement, STEM PhD commitment), which may have contributed in part to observed outcomes.
Of special note, the percentage of Meyerhoff students earning STEM PhDs increased dramatically from the 1989–1996 to the 1997–2004 entering cohorts. This jump in the number of Meyerhoff students earning STEM PhDs may be due to a number of factors, including the program attracting more PhD-focused, better prepared students over time; improvements over the years in program scope and quality; and the program’s growing national reputation. The percentage of comparison sample students earning STEM PhDs remained essentially unchanged during the same time period. It is not clear what explains this lack of change in comparison versus Meyerhoff students over the years; to some extent this may reflect low success rates in STEM majors for comparison students, a retreat from affirmative action, enhanced competition (e.g., international students), and an attraction to pursue other career paths.
Program Mechanisms of Influence
Perceived program benefit at the end of freshman year was shown to fully mediate the relationship between sense of community at the end of Summer Bridge and science identity and research self-efficacy, respectively, at the end of sophomore year. Financial aid, study groups, tutoring, academic advising, personal counseling from staff, mentoring and support, organized social activities, faculty involvement, interactions with UMBC administrators, Meyerhoff community, and professional development were among the program elements rated by students in the perceived program benefit measure. Each addresses an important challenge facing minority students pursuing undergraduate STEM degrees. For example, financial support allows students to afford college and to focus exclusively on their studies rather than working during college (Georges, 1999). Study groups give students the opportunity to provide and receive academic help in difficult STEM courses and have been linked to academic gain in previous research (Gandara and Maxwell-Jolly, 1999; Moreno et al., 1999). Academic advising from staff, peers, and mentors is important to help students make informed, strategic decisions about the number and type of courses to take (and retake) and which possible research opportunities to pursue (Seymour and Hewitt, 1997; Gandara and Maxwell-Jolly, 1999). The other program elements assessed have also been highlighted as important in the research literature.
Consistent with program theory, students who benefit broadly from the wide array of program elements also develop greater levels of science identity and research self-efficacy, two critical variables that have been found to be linked to academic success in STEM. This finding supports the view that the multifaceted, comprehensive nature of the MSP, addressing multiple areas of minority student need and challenge, explains its high levels of success over time. Students who perceive greater benefit from the varied elements, as a whole, appear to be those who develop the psychological attributes and personal competencies inherent to success as a scientist.
Why do some Meyerhoff students perceive greater benefit from program elements than others? One contributing factor appears to be a sense of community. Sense of community has been proposed as central to the effectiveness of “cohort models” of STEM intervention programs, in which close relationships are developed among students and with staff in a program. Indeed, one of the primary purposes of Summer Bridge, according to MSP staff, is to develop a “family-like” community atmosphere among students, one in which students provide and receive high levels of support, both academic and psychological, from others. Group cohesion, group accountability, and academic and social integration into the university setting are some of the important benefits of sense of community developed during Summer Bridge that may enhance student success.
The current findings suggest that sense of community influences student outcomes by contributing to effective engagement in the varied MSP elements. Although all students likely benefit to some extent from the sense of community, those who experience higher levels of community appear to benefit more. The mechanisms through which sense of community contributes to perceived benefit of program elements require further empirical study. In particular, use of reliable and valid multi-item measures of student engagement and benefit from each program element is necessary to advance understanding; such measures were recently developed and are currently being used by our research team (Maton, 2015). In addition, to rule out alternative explanations of findings, it is a priority for future research to assess at college entry student levels of science identity and research self-efficacy and related variables (e.g., research excitement, STEM PhD commitment). Such variables may simultaneously contribute to sense of community and to perceived benefit of program elements, thus accounting for the observed empirical relationships, and ideally should be assessed and statistically controlled for in future research.
Limitations
The current research has a number of limitations. Possible self-selection differences between Meyerhoff and declined comparison students limit the strength of the conclusions that can be drawn about the outcomes obtained. That is, students who opted to attend the program may be more committed initially to obtaining a PhD than those who declined the admissions offer. It should be noted, though, that all students accepted into the program expressed a strong interest in pursuing a STEM PhD and began college with a STEM focus. Only a random assignment design would provide a definitive means to overcome this design weakness. Such a design is difficult to implement in this research area. In addition, now that the program has been shown to be effective, a random assignment design may raise ethical concerns, as it would deny worthy students access to likely program benefits (Freedman, 1987). Furthermore, students who are assigned to the control group may underperform due to a resentful awareness of not being included (Ong-Dean et al., 2011).
A second limitation relates to the unknown validity of the perceived program benefit scale. It cannot be known whether this measure assesses actual benefit from program involvement or, alternatively, just a positive feeling about the program in the absence of concrete benefit. The items do possess strong face validity. Nonetheless, as noted above, our research team recently developed multi-item measures of student engagement in each program element, and these scales have been added to our research protocol and will be included in future studies.
A third limitation is the self-report nature of the four psychosocial variables. It is possible that the findings, at least in part, are due to “solo-method” bias, in that students who are positively predisposed may report higher levels on all four measures. Thus, rather than reflecting causal relationships, the findings may, at least in part, be a function of measurement bias through sole reliance on self-report measures.
A fourth limitation is the unknown predictive validity of the science identity and research self-efficacy measures. To date, research has not examined whether Meyerhoff students with higher levels of science identity and research self-efficacy at the end of their sophomore year are more likely to earn STEM PhDs than students with lower levels. Although science identity and research self-efficacy are widely cited in the literature as important for persistence in STEM majors and for STEM graduate study, it cannot be known whether the levels of variation on these variables among the Meyerhoff students will turn out to be predictive of their STEM PhD pursuit and completion. The current findings will prove especially important if science identity and research self-efficacy are found in the upcoming years to be predictive of academic outcomes for MSP students.
A fifth limitation is the relatively small sample size in study 2. It is only recently that the measures of interest were added to our research protocol, which accounts for the sample size. In future years, along with an increased sample size overall, there will be sufficient numbers of African-American students to conduct analyses solely on this subgroup and also to systematically compare the nature of program mechanisms across students of different ethnicities.
Finally, the generalizability of findings to programs in different universities and with differing arrays of program elements is not known. The MSP is relatively unique in its focus, its comprehensiveness, its high level of resources, and the high levels of commitment of the university administration to its success. Whether other programs embedded in their own unique contexts will yield similar outcome and process findings is a question for future research.
Future Research and Conclusion
The limitations notwithstanding, the current study represents one of the few systematic examinations of a college-based intervention program designed to increase STEM PhD receipt among URM students. Future research should include systematic comparisons of different intervention approaches; use of established measures of known reliability and validity; assessment of level of program involvement and engagement in different program components; use of multiple methods of assessment, including reports from peers and mentors; in-depth examination of the student experience, including qualitative data; assessment of all psychosocial variables from the start through the end of college; inclusion of a comparison sample to directly compare changes over time on the psychosocial variables of focus; and longitudinal tracking of STEM career outcomes, including involvement in STEM research, teaching, and mentoring. Furthermore, at the institutional level, research is needed to examine how faculty attitudes and expectations of students have changed over the years, as professors have seen more URM students succeeding and have become increasingly involved in course redesign and other experiments in teaching and learning.
In conclusion, enhancing the academic success of URM students in the STEM fields is a pressing national priority. It represents both an economic necessity, so our nation can stay competitive in the global economy, and a critical part of our nation’s larger social justice agenda. Increased understanding of the effectiveness of STEM programs and of the program elements and processes that contribute to positive outcomes represents a critical priority for future work. The current study represents one contribution to this important research agenda.
