Preparing Postbaccalaureates for Entry and Success in Biomedical PhD Programs

INTRODUCTION

Diversity in the United States is no longer a future consideration; it is the reality of the current societal landscape. Projections indicate that by 2023 there will be no racial majority among the nation’s youth, and the Census Bureau reported that the majority of births in the United States in 2011 were nonwhite for the first time (U.S. Census Bureau, 2012). In stark contrast to the population’s changing demographics, the population of leaders in the biomedical enterprises of academia, government, and industry is relatively homogeneous. Recent data by the National Science Foundation (NSF) show that the percentage of individuals from groups that are underrepresented (UR) in the sciences (African Americans, Hispanics, American Indians and Alaska Natives, and Pacific Islanders, and individuals with disabilities and from low socioeconomic backgrounds) decreases at each rung of the science, technology, engineering, and mathematics (STEM) academic career path. While 18.9% of STEM bachelor’s degrees are awarded to UR students, only 7.3% of STEM doctorates and 4.3% of STEM professors at R1 doctoral institutions are from UR groups (NSF, 2015).

Several studies have shown that diversity of thought and background promotes scientific progress (Page, 2008; Ferrini-Mundy, 2013; Freeman and Huang, 2014). Diverse teams of problem solvers outperform groups composed solely of the top performers (Hong and Page, 2004). In addition, a diverse faculty benefits student learning by providing a broader range of pedagogical techniques and more frequent student engagement in effective educational practices (Umbach, 2006). Therefore, it is imperative that educational institutions adapt to provide opportunities and environments for all students to explore their interests, achieve their potential, and become key players in our nation’s workforce to solve the large-scale challenges facing our society.

To address the specific disparity between life sciences bachelor’s degrees awarded to UR students compared with the percentage of UR biomedical PhDs awarded, the National Institutes of Health (NIH) implemented the Postbaccalaureate Research Education Program (PREP) (www.nigms.nih.gov/training/PREP). These programs target recent graduates from UR groups with an interest in graduate school and biomedical research careers. The goal of PREP is to strengthen the preparedness of UR scholars for research-intensive biomedical PhD programs and build a foundation of skills, experiences, and scientific self-efficacy that maximizes their chance of success and persistence in science careers. In fact, addressing the extreme lack of diversity at the faculty and senior scientist level is the ultimate goal of federally funded initiatives like PREP. Currently, there are 32 PREPs across the United States (National Institute of General Medical Sciences, 2016). All PREPs are housed at research-intensive institutions, and common features include ∼75% effort for mentored research and 25% effort for “further skills development” during (typically) a 1-yr appointment (National Institutes of Health [NIH], 2016).

While a previous study examined factors involved in a scholar’s decision to pursue postbaccalaureate training and how these factors related to an individual’s feeling of readiness for graduate school (Gazley et al., 2014), the work presented here examines outcomes of a PREP intervention itself and how specific programmatic components contribute to building the skills and scientific self-efficacy that are important for persistence in the biomedical training path. A program that successfully promotes inclusion in the sciences must not only focus on skill building but must also consider the unique “experiences and career development patterns” of UR individuals compared with their majority counterparts (Gibbs et al., 2014).

Our theoretical framework is that a successful scientific PhD preparatory program must not only build laboratory skills but must also enhance a scholar’s “science identity,” which includes knowledge of a discipline, ability to perform relevant professional practices, and a sense of belonging and recognition within a scientific network confirmed by oneself and others (Carlone and Johnson, 2007; Hurtado et al., 2009). Therefore, our program includes a mentored research experience but also directly provides opportunities to build skills and gain experiences outside the lab that are necessary components of success in scientific careers. These experiences occur within the context of a cohort of scholars who are intentionally provided opportunities to build community both within the cohort and also within the broader research environment at our institution. Through this program, scholars should gain increased self-efficacy in a constellation of competencies, thus contributing to increased science identity and a desire to pursue and persist in scientific research careers.

As is the goal for all NIH-funded PREPs, our goal in designing University of North Carolina (UNC) PREP (hereafter referred to as UP) was to build a program that increases the number of UR scholars who matriculate in competitive biomedical PhD programs and to prepare these scholars for subsequent success within those programs. We postulated that enabling students to “hit the ground running” in graduate school increases both the likelihood that they have a positive and productive graduate experience and that they pursue competitive postdocs and scientific positions after graduate school. We therefore designed a program that provided specific training and support in a range of areas that are necessary for entry into and success within biomedical PhD programs. Initially, the program focused on building a strong support network and acclimating scholars to the environment of full-time research at an R1 doctoral university, then preparing scholars for the biomedical PhD program application and admissions process. Concurrently, we provided professional development training in skills that are important for success in science but are not often explicitly taught in the research lab (unspoken rules of lab culture, giving an effective talk, preparing a poster, time management, scientific writing, money management, etc). By developing a program that blends lab training, support in a multitude of relevant professional proficiencies, and intentional community building among scholars and with scientifically trained program staff, our hypothesis was that an intervention of this type would lead to successful outcomes for UR scholar acceptance into and persistence in highly competitive biomedical PhD programs. In addition, we investigated which programmatic components were most beneficial to students and how the PREP intervention influenced the scientific self-efficacy of scholars. In this study, we present findings and outcomes from the first 5 yr of UP.

METHODS AND RESULTS

Program Participants

All work was carried out under the approval of the UNC Chapel Hill Institutional Review Board (study 15-2750), and all participants provided consent for data collection used in this study. Scholars completed a survey on the first day and during the final week of the program, plus an alumni survey 1 yr after program completion. Scholars also completed specific skills assessments (described later) and participated in three focus group interviews. The entry, exit, and alumni surveys and the focus group interviews were collected and analyzed by an external evaluation team, and results from these instruments were available only anonymously and in aggregate to program leaders. Results from entry and exit surveys reflect 100% of scholars. The response rate for the alumni survey was 97% (33/34) of scholars who completed the program. Scholar focus groups occurred three times per year: following the Summer session and near the end of both the Fall and Spring semesters. Focus groups were scheduled in association with weekly group meetings, so most, if not all, scholars were typically present. No project leaders or university administrators were present during the interviews. The focus group sessions lasted 60 min and were audio-recorded to enable the evaluator to focus on directing the conversation and to capture direct quotes as students spoke about their experiences with the program. Interview transcripts were analyzed via Atlas.ti, a qualitative software package that allows for assigning thematic codes to narrative. The evaluation team then reported the most common themes to program leaders, including representative quotes that were stripped of identifying information. Faculty mentors were similarly surveyed at the end of the program year by the external evaluator via electronic survey and a focus group session. Relevant or illustrative quotes from focus group interviews and qualitative survey responses are presented in this report throughout the text.

Five cohorts have completed UP to date, totaling 45 postbaccalaureate scholars. UP scholars were primarily African American or Hispanic/Latino(a) (55.6 and 44.2%, respectively), and as has been observed for other PREPs (Gazley et al., 2014), UP scholars were predominantly female (71.1%; Table 1). We have also observed this trend for our graduate programs; our incoming class of biology and biomedical PhD students is 63% female. While the intersection of underrepresentation exists among female scientists who are also from UR racial or ethnic groups, we did not directly study the impact of intersectionality here. The majority of UP scholars received their bachelor’s degree from minority serving institutions (64.4%), and approximately one-third (31.1%) attended research-intensive universities (Indiana University Center for Postsecondary Research, 2015) before joining UP. Scholar selection was competitive, with ∼100 applications for 8–10 slots (Supplemental Table S1). The majority of scholars were supported by the NIH PREP grant, with an additional one or two scholars funded from other sources, including institutional funds and individual faculty research grants. We selected scholars based on a stated desire to pursue a biomedical PhD and some evidence of having pursued research opportunities in the past, though the majority of scholars had limited research experience before entering UP. Each applicant submitted an undergraduate transcript, personal statement, and three letters of recommendation. We did not require Graduate Record Examination (GRE) scores for UP applicants. A subgroup of scholars applied to UP after being declined by UNC’s Biological and Biomedical Sciences Program (BBSP), an umbrella first-year PhD training program. Forty percent (18/45) of UP scholars joined the program after having unsuccessfully applied to PhD programs (Table 1).

Program Design

Summer Development Session (SDS).

The UP time line and program structure is highlighted in Figure 1. The first five iterations of the UP 1-yr postbaccalaureate experience began each year in early June. The first 4 wk consisted of the Summer Development Session (SDS or Summer boot camp), which included five primary components: critical analysis of scientific literature, hands-on lab skills, GRE preparation, professional development workshops, and lab selection. One goal of the SDS was to build community among the cohort and between the scholars and program staff. In addition, the SDS provided acclimation to the research institution environment and intensive skill building before the scholars began full-time research. Importantly, during the SDS, scholars met with program staff to discuss their research interests and receive guidance about lab options. Scholars then met with multiple faculty members to select a research lab that matched both their scientific interests and preferences for a training environment (mentoring style, lab culture, etc.). Once scholars joined a research lab, they continued to meet as a group with program staff once per week for the remaining 11 mo of the program for additional skill building, including training in the responsible conduct of research, oral research presentations, preparation for the graduate school application and interview process, critical analysis, and scientific writing. Importantly, since the scholars were scattered in their individual research labs during the final 11 mo of the program, the group meetings provided continued contact among the cohort and between the scholars and program leaders and maintained a sense of community among the group.

Critical Analysis of Scientific Literature.

When students join research labs, they are often first given a stack of research articles to get “up to speed.” This experience can be daunting for early-career scientists with minimal practice reading the scientific literature. To strengthen their critical analysis skills and paper-reading efficiency, UP scholars participated in an intensive primary literature–based course during the SDS. The course began with a workshop explicitly describing strategies for effectively and efficiently reading a research article. Each week of the SDS, scholars read a review article and three research articles centered on a common theme (topics included neurophysiology of alcoholism, the microbiome, cancer and DNA repair). Faculty members or postdocs led students through in-depth discussions of each article from the big picture to specific research methods used. At the end of each week, scholars completed a take-home exam consisting of questions requiring higher-order thinking such as experimental design and drawing connections between papers; the exam questions were similar to those encountered in graduate-level courses. Following the exam, students met with the course instructor to go over their answers and address any areas of weakness (including writing skills). By the end of this module, students had read in detail ∼16 articles and, as a result, expanded their knowledge of scientific vocabulary, approaches, techniques, and topics.

To measure scholars’ critical analysis baseline and their growth during the SDS, we assigned participants a primary research article on the first day of the SDS along with a series of questions that probed their ability to understand the rationale, experimental methods, controls, results, and conclusions. Scholars had 2 h to read the paper and complete the assessment. At the end of the 4-wk critical analysis module, scholars received the same paper and questions and repeated the exercise. We randomized and blindly graded the baseline and postanalysis responses using a scoring rubric to assess growth in the ability to efficiently analyze a research article. Scores significantly improved following the critical analysis block (56.7 ± 17.9 pre vs. 75.9 ± 13.4 post, p < 0.001, n = 27; Figure 2Ai). In addition, scholars reported increased confidence in all surveyed aspects of reading and understanding the research article from the preassessment to the postassessment (Figure 2B). We considered the possibility that this improvement simply reflects practice with the same paper. For this reason, we experimented with using different papers for the pre- and postassessment in the first 2 yr, but differences in the difficulty level among papers and variability in scholar background knowledge confounded assessment of the intervention. Moreover, the assessment paper was not discussed following the baseline exercise, so the students received no feedback on how well they answered the baseline assessment questions. Between the baseline and postassessment, the scholars read and discussed in-depth ∼16 articles over 4 wk, so it was unlikely that they benefited significantly from remembering the paper from the first reading. The assigned article, assessment, and scoring rubric are provided in the Supplemental Material.

Scholars reported in focus groups that the SDS critical analysis block improved not only their ability to analyze scientific papers but also their paper-reading efficiency:

The critical analysis block contributed to increasing the science identity of scholars by increasing content knowledge through the reading of papers and, perhaps more importantly, by strengthening general paper-reading skills and efficiency.

Laboratory Skills Training.

Students entered UP with varying degrees of research experience. To prepare students for entry into their PREP research lab, UP included a hands-on introductory lab skills module during the SDS. The goal of this course was to build a general knowledge foundation by introducing skills likely to be important when working in a biomedical research lab. Topics included “How does a research lab work?,” lab safety (led by UNC’s Environmental Health and Safety Office), lab math, making solutions, working with DNA, molecular cloning (including polymerase chain reaction, gel electrophoresis, plasmid preps, restriction digests, ligations, transformation, and sterile technique), techniques for measuring gene expression, and techniques for detecting protein–protein interactions. In addition, we familiarized students with experimental design, hypothesis testing, statistical analysis, and the use of Web tools like BLAST and PubMed. Students learned techniques in the context of weekly miniexperiments, such as identifying an unknown gene, statistically analyzing a phenotypic study following RNA interference (RNAi) treatment of Caenorhabditis elegans, and determining whether two signaling proteins in a pathway directly interact via coimmunoprecipitation. Students kept lab notebooks and gave a formal presentation of their findings at the end of the course. These activities ensured that scholars possessed a foundation of knowledge and understanding about the workings of the research lab and biomedical research, which gave them confidence before they joined a UP research mentor’s lab.

To determine scholar progress, on the first and last day of the lab skills course, we assessed students’ familiarity with common biomedical techniques and what they are used for (e.g., immunoblots to determine protein expression, RNAi to suppress translation). Scholars improved significantly (44.2 ± 19.8 pre vs. 64.9 ± 18.9 post, p < 0.001, n = 27) in their knowledge of common biomedical methods and techniques and what types of questions these methods can address (Figure 2Aii). The assessment used can be found in the Supplemental Material.

We presume that the hands-on lab skills training was especially beneficial for scholars with minimal previous research experience. While scholars would sometimes report that specific techniques learned during the lab skills module were directly useful in their UP research lab, an overarching reported benefit from focus groups was recognizing the importance of understanding why each step of a protocol is important:

GRE Preparation.

Despite evidence that the GRE is biased against females and UR minority groups (Miller and Stassun, 2014), the GRE is still a ubiquitous requirement of the graduate school admissions process. Thus, we recognized that significant GRE preparation is a necessary component of pregraduate preparatory programs. During the SDS, scholars received didactic instruction on quantitative and verbal concepts and general test-taking strategies. Every session began with vocabulary “challenges”; we placed students in groups to compete against one another in games that reinforced the most commonly used GRE vocabulary words (a key component to success in the verbal section). At the end of each session, we assigned practice problems that utilized the skills learned in class that day. In the afternoon, scholars had dedicated time to work on GRE problems individually and in groups. They submitted completed problem sets to the instructor, who used their responses to determine which problems were most challenging to students. These problems were then topics for class the next day.

UP scholars significantly improved both verbal (34.6 ± 21.3–47.7 ± 22.8, p < 0.001, n = 42) and quantitative (31.5 ± 17.4–41.4 ± 19.7, p < 0.001, n = 42) GRE percentile scores from the pretest taken on the first day of the SDS to the posttest taken on the final day of the SDS (Figure 2C). The pre- and posttests used were official Educational Testing Service (ETS) computer-based practice tests from the ETS Powerprep II software that simulates the actual GRE exam. Scholars took the official GRE within 2 mo of completing the SDS, and their official scores were significantly higher than their pretest scores (Figure 2C, p < 0.001, verbal and quantitative, n = 42). Actual GRE scores were also, on average, higher than the posttest scores (48.6 ± 24.8 percentile verbal, 49.2 ± 16.6 percentile quantitative, n = 42), but this difference was only significant for the quantitative section (Figure 2C, p < 0.01).

While GRE preparation was likely not any scholar’s favorite aspect of the SDS, students recognized that the GRE is currently an important component in graduate school admissions:

Professional Development Workshops.

During the SDS, scholars also participated in workshops that addressed additional skills important to being a successful student and researcher. Topics included time management, money management, an introduction to lab culture, how to keep a lab notebook, how to interview with faculty when selecting a lab, how to succeed in the research lab, and how scholars can make the most out of their UP year. These workshops addressed important professional practices that contribute to a scholar’s science identity.

Laboratory Research.

For the lab selection process, UP scholars met with two to three faculty within their research interest areas. All laboratory selections were made by the end of the SDS, in early July. Within the first 2 wk of the research experience, program staff led an orientation session describing programmatic expectations and specific expectations of scholars, faculty, and day-to-day laboratory mentors (often senior graduate students or postdocs). Because the goal of UP scholars is to matriculate and succeed in PhD programs, the orientation session emphasized the importance of independent research projects for UP scholars. During this session, we introduced the UP individual development plan (IDP) process (described in the following section), and previous UP mentors shared their experiences and advice for mentoring UP scholars.

In the first 5 yr, 34 UNC faculty participated as UP mentors, and nine out of 34 served as UP mentors in multiple years. At the end of each year, the majority of UP mentors rated their scholars’ overall research progress and growth to be “very good” or “excellent” during their PREP years (Supplemental Table S3). Overall, faculty mentors were very satisfied with the program (6.29 ± 0.94 on a seven-point satisfaction scale, n = 34; Supplemental Table S4), and 91% (31/34) of PREP mentors surveyed were “likely” or “very likely” to host another PREP scholar in the future (Supplemental Table S5). Currently, UP scholars have coauthored 24 peer-reviewed manuscripts resulting from research done during their 11-mo UP experiences, including two papers with UP scholars as the first author. In addition, UP scholars presented their research along with all first-year biological and biomedical PhD students in an end-of-semester poster session.

Individual Development Plan (IDP).

Central to each UP scholar’s training is the IDP team, consisting of the scholar, the research mentor, and the PREP director. The UP IDP evaluation instrument (which we customized and is distinct from IDPs focused primarily on long-term career goals) assesses the skills of each PREP scholar in key areas important for becoming a successful scientist, including scientific writing, presentation skills, and response to feedback (the UP IDP survey is provided in the Supplemental Material). Typically, if program staff identified a growth area, then the scholar met regularly with a member of the program staff, the research mentor, or both. Examples of development activities are scholars meeting weekly with their faculty mentors or program staff to go over research articles or writing a weekly research synopsis for the principal investigator (PI) to review. The IDP process was not only important for identifying and addressing the specific needs of each scholar but also for facilitating communication between the scholar and the research mentor and helping the program staff be fully informed about each scholar’s progress. A common theme that emerged from focus groups was that scholars often had a more negative view of their own progress than their faculty advisors did, and the IDP meetings were often a confidence booster for the scholar:

In addition, alumni recounted that the IDP process during UP provided a useful and safe venue for critically assessing their progress. Alumni also reported that the type of communication that was fostered during UP IDP meetings provided a model for communicating with their PhD advisors in graduate school:

The IDP process was also beneficial for the faculty advisors. Faculty mentors reported satisfaction with the level of communication with the program staff, which was primarily accomplished through the IDP process (Supplemental Table S3).

Graduate Course Performance.

Graduate-level course work typically differs from undergraduate course work in both format and expectations. To prepare scholars for PhD-level course work, all UP participants took a graduate-level course for credit alongside UNC biological and biomedical PhD students; National Institute of General Medical Sciences funding and institutional support provided tuition funds. Scholars chose courses based on their individual research interests and background. Graduate-level courses at UNC are graded “H” (high pass, approximately top 10%), “P” (pass), “L” (low pass), and “F” (fail). In the first 2 yr, the program provided a course coach (typically a graduate student who had previously performed well in the course) if a student performed below average on the first exam. However, we found that it was often challenging to “catch a student up” once they were behind. This issue was compounded in courses in which the first assessment did not occur until halfway through the semester. Therefore, in year 3, we began assigning all scholars a course coach at the beginning of the semester. All UP scholars met weekly in small groups with their course coaches to review notes and ask questions. On the addition of mandatory coaching, scholar course grades improved dramatically, and all scholars received at least a “P” (Figure 2D). An additional benefit of PREP scholars taking a graduate-level course is their ability to prove themselves academically to a graduate admissions committee, especially for scholars with an undergraduate grade point average (GPA) or GRE scores lower than average for admitted students at their target graduate institutions. Importantly, four of the seven scholars receiving an “H” had previously been declined admission into PhD programs before UP, which indicates that admissions committees may have difficulty predicting the future academic performance of applicants based on past academic metrics (GRE scores and undergraduate GPA).

Scholar Confidence and Self-Efficacy

Successful scientists possess attributes that transcend success at the lab bench. A previous study of successful female scientists of color determined a model of three overlapping components that contribute to “science identity”: competence in one’s discipline, performance of relevant professional practices, and recognition as a “science person” by oneself and others within the scientific community (Carlone and Johnson, 2007). An analysis of the underrepresentation of African Americans in the sciences highlighted that having a successful science career goes beyond desire or ability and also involves a social component (Lewis, 2003). “Following these lines of reasoning, fostering science identity development involves more than focusing on individual factors such as increasing one’s level of competence in science. It also involves social factors, including socialization into the sciences and making meaning of science-related experiences, such that an individual not only feels like a science person, but also acts and is seen by relevant others as a science person” (Hurtado et al., 2009, p. 192). Themes such as these greatly influenced the design of our program to include, along with an intensive research experience, a plethora of professional skill-building components and a focus on building and maintaining community during the program year. We expected that successful implementation would result in increased scholar self-efficacy in areas relevant to scientific research and, ultimately, in scholars choosing and persisting in scientific careers.

Put simply, self-efficacy is the belief in one’s own ability to successfully accomplish something (Bandura, 1997). As with most pursuits, there is increasing evidence that scientific self-efficacy is strongly tied to retention and success in science careers; for a science trainee to pursue or persist in the science-training track, it is critical that the scholar believes he or she can do it (Lent et al., 1994, 2008; Chemers et al., 2011). A growing sense of scientific self-efficacy is critical for a trainee to pursue, attempt, and then proceed in science, as it is directly linked to building science identity. With regard to programs like UP that target individuals in the early stages of the training path, any successful intervention must contribute to increasing scientific self-efficacy, either directly or indirectly. One shortcoming of the blanket term, “scientific self-efficacy” is that, on its own, it oversimplifies the complexity of skills and abilities needed as a researcher. For example, an individual could have high self-efficacy for his or her ability at the bench but be terrified to give a research talk. Other scientists are confident communicators but have less confidence in their technical abilities. Therefore, we divided scientific self-efficacy into 24 competencies (listed in Figure 3). Throughout our program design, we purposefully built opportunities for scholars to gain experience, and thus confidence, in all of these areas.

To assess the impact of UP on scholar scientific self-efficacy, participants reported their confidence level for each item on the first day of the program and again, a year later, during the last week of the program. To minimize bias and encourage honest responses, an external evaluator administered and analyzed the surveys, and scholars were aware that their replies were anonymous and would only be shared with program staff in aggregate. Scholar confidence increased significantly (p < 0.01, n = 44) for 21 of the 24 categories after completion of the UP (Figure 3). Interestingly, two of the three categories that did not show a significant increase in scholar confidence were general statements of science identity, “Having a successful science career” and “Being a scientist.” One explanation could be that these were two of the items scholars initially reported being most confident about at program outset, leaving less room for increased reported confidence (though, notably, confidence could decrease but did not in any case). The fact that scholars were initially confident in their general ability to be a scientist and have a successful career but were noticeably less confident in all of the specific pieces that go into achieving that goal could represent a general naïveté that goes along with being an early-career trainee, or it could indicate a selection bias by our program for UP scholars that strongly demonstrate a general (though not yet refined) desire to pursue a PhD and a research career.

In exit surveys and focus group interviews, scholars confirmed the increased self-efficacy observed from survey responses; they reported feelings of increased confidence in their ability to be a scientist:

Outcomes

A primary goal of UP, and all PREPs, is to transition scholars from our program to competitive biomedical PhD programs well prepared for retention and success in these programs. Over the past 5 yr, 91.1% (41/45) of UP scholars transitioned directly into biomedical PhD programs, an additional two scholars entered science master’s programs, and one went directly into an industry position (Table 2). During the first 5 yr of UP, 14/41 of the scholars who entered PhD programs joined UNC’s BBSP PhD program. While we are certainly eager to retain talented students for our own graduate programs, the goal of UP is to prepare scholars for the graduate programs that are the best fit for them professionally and personally. It is important to note that being an UP scholar is not a “rubber stamp” into UNC’s biomedical PhD programs; not all UP scholars received an offer of admission, though most were very competitive. In addition, applying to UNC for graduate school is not a requirement. UP scholars had an average of 3.4 PhD program acceptances, so typically they had multiple programs to choose from and selected the one that was the best fit for them.

The retention rate of UP scholars in PhD programs is currently 95.1% (39/41). Because our program is only 5 yr old, UP scholars from the first cohort are just beginning to graduate, but thus far, nine out of 10 have either graduated or are on track to graduate within the next year (the 10th scholar initially entered a PhD program but finished with a master’s degree and works in a research position at a leading R1 institution). This result is significant, given that the national PhD completion rate for biomedical science fields averages around 63% (Council of Graduate Schools, 2008; National Research Council, 2011), which is also the average PhD completion rate for scholars from all NIH-funded PREPs, nationally (Hall et al., 2015). There are many factors that go into assessing the quality of a research institution, but for biomedical science in the United States, NIH funding is a useful proxy for the focus and intensity of biomedical research activity taking place at an institution. More than half (23/41) of UP scholars entered PhD programs at institutions that currently rank in the top 10 for NIH funding, and 87.8% (36/41) of scholars entered PhD programs at top-20 NIH-funded institutions (Table 2). Of the scholars who had been rejected from PhD programs before UP, 17/18 were accepted into PhD programs after UP, and 16 of those scholars entered graduate programs at top-20 NIH-funded institutions. These positive outcomes highlight the potential impact of bridge programs like UP that keep talented UR trainees within the research community on paths to scientific leadership.

The benefits of UP for graduate school acceptance were often increasingly clear to scholars once they transitioned from UP into a PhD program, and this appreciation was a common theme reported by UP alumni surveyed after their first year of graduate school:

Benefit of Program Components

To assess the usefulness of specific programmatic components to participants, we surveyed scholars during the final week of the program and again after they completed their first year of graduate school. The alumni survey asked scholars to assess which interventions had persistent benefits during graduate school. On exit, scholars rated most program components as beneficial, with mean ratings for all components averaging greater than 3.5 on a five-point scale (Figure 4). In most cases, ratings of program components by alumni trended to be even higher than those for scholars just exiting the program. Responses on open-ended alumni survey items corroborate the finding that scholars considered the program even more beneficial in retrospect:

Interestingly, but perhaps not surprisingly, three of the four program components rated by scholars as most beneficial were related to interpersonal professional relationships with program staff, research mentors, and fellow UP scholars (social activities). These data indicate a phenomenon we have observed that is difficult to enumerate—the importance of relationships and social support during the training process. The importance of community and support networks is often overlooked in studies or descriptions of research programs or interventions, but it is an important theme that we intentionally strive to promote within our program through team building during the SDS, weekly meetings with all scholars and program staff, and regular social activities (planned and impromptu). Our results indicate that intentional fostering of support networks and community is a critical component of a successful research education program, which affirms the third pillar of our theoretical framework—the importance of feeling a sense of belonging within a scientific community. It is worth noting that UP scholars were included alongside our biological and biomedical graduate student population at social and academic events, including social activities, graduate courses, and poster sessions. In these ways, scholars also became integrated into the broader research community outside of the program and their own lab.

The importance of interpersonal relationships with program staff and peers came up regularly during scholar focus group interviews and open-ended responses upon program exit:

Even after leaving the program, alumni recognized the importance of social activities and community building as a key component of their success during UP. Perhaps more importantly, alumni also reported seeking out similar community and support systems they encountered during UP at their graduate institutions during their PhD training, which likely is a contributing factor to their high rates of retention in graduate school:

We consider that the best measure of program impact is feedback from scholars once they have completed a year of graduate school. By that time, scholars have a more realistic view of how well the graduate preparatory program actually readied them for graduate school. When we asked alumni how well UP prepared them overall for graduate school, alumni responded that they were extremely well prepared (6.56 ± 0.65, 7 = extremely well, 1 = not very well, n = 33) (Supplemental Table S8), indicating that the benefits of UP continued to be apparent once the scholars transitioned from UP into graduate school. Open-ended responses from the alumni survey also demonstrated this appreciation:

DISCUSSION

A recent study examined factors that led individual students with an interest in a biomedical PhD to choose a PREP instead of directly entering graduate school (Gazley et al., 2014). Not surprisingly, the authors found that scholars enter postbaccalaureate programs for many different reasons based on their specific goals and expectations and their general feeling of “readiness” for graduate school that goes beyond their preparation. In addition, PREPs provide an important safety net to catch potential postbaccalaureate researchers who are initially declined by PhD programs. As we demonstrated here, a sizable portion of scholars in our program chose PREP as a backup plan after not being accepted into graduate school.

In our experience, PREP scholars typically fall into one of two categories: 1) students who performed well academically as undergraduates but became interested in research toward the end of their undergraduate careers and thus have minimal research experience; or 2) students with consistent research experience at their home institution and/or through summer research programs but with a history of academic performance that is below what is typically required for admission into top biomedical PhD programs. Either type of student may be perfectly capable of success in a PhD program but is minimally competitive right out of their undergraduate studies because of a specific shortcoming in their graduate admissions application. PREP offers benefits to both types of students through providing a year of full-time research experience (especially important for the first group) combined with opportunities for scholars to prove themselves academically by performing well in graduate-level course work or boosting their GRE scores (especially important for the second group). Importantly, an effective PREP must also build skills and self-efficacy in the many professional practices that go along with being a scientist besides bench work and must intentionally foster a supportive network and scientific community in which the scholar develops a sense of belonging. These things together contribute to effectively forming scholars’ science identities and increasing the likelihood they will persist in science as they advance through their training.

SUGGESTIONS FOR POSTBACCALAUREATE PROGRAMS

While all PREPs and other biomedical postbaccalaureate programs contain research components and a selection of professional development activities, our analysis identified a few particular areas that go beyond skill building that have contributed to positive scholar outcomes.

Build Community into Programming

Scientific research is a challenging and sometimes stressful pursuit. There can be challenges faced by students from UR groups in addition to those faced by their majority peers. For example, UR undergraduate researchers within diversity programs indicated an enhanced need to validate their competence and affirm their identity as scientists, especially in the face of external expectations or criticism from peers and family members (Hurtado et al., 2009, p. 209). “In response to external criticism and misunderstanding, it becomes clear that students must find ways to reinforce their developing science identity. For example, to counteract negative perceptions or misunderstandings about being a scientist, students turn to the support they receive from the structured research programs in which they participate.” Indeed, outgoing UP scholars rated the program staff as the most beneficial component of the program, closely followed by the research mentoring they received, going to the Annual Biomedical Research Conference for Minority Students (ABRCMS), and social activities. These benefits were also recognized by scholars once they left the program, indicating that the relationships built during the program and the support they provided was instrumental to overall successful outcomes. Therefore, while it is certainly important to connect students with research experiences, efforts to foster community and a support network among peers and with program staff is critical for maximizing student outcomes. In UP, scholars are all required to start the program together in June. Besides the educational content, important benefits of the SDS are the bonds that are built within the cohort and with the program staff. On day 1 of the program, we include team-building games that rapidly build familiarity and trust among the scholars. Scholars and program staff continue to work closely together daily during the first 4 wk of the program. After the SDS, once scholars scatter to their respective research labs, scholars maintain connection through the weekly group meetings and regular social activities throughout the year.

Choose Program Staff with a Science Background

Most federally funded scientific diversity initiatives are led by a faculty PI and are often supported by a nonfaculty staff person who acts as a program coordinator, director, or both. In our experience, having program staff who themselves have been through scientific graduate training has been especially beneficial to students. Typically, with the exception of the faculty research mentors, the program coordinator/director has the most direct contact with the students, and this individual is often seen as a “safe” person to whom students can air frustrations or stresses they are facing. Moreover, program staff with a science background are able to provide not only social support but also direct assistance with reading and discussing scientific papers and feedback on research progress and research presentations:

Facilitate Communication between Mentors and Scholars

We have observed through our own PhD admissions process for the BBSP that the single most important component of the graduate school application is the most recent research mentor letter of recommendation. Even if other aspects of a PhD application are strong, a lukewarm letter from a research mentor can greatly hinder an applicant’s chances for admission. We found that the process of formally checking in with mentors and scholars at least once per semester (and at least 2 months before the PhD application deadline) via the UP IDP process was beneficial for facilitating communication between student and PI. Structured research programs have the advantage of liaising with the student and PI to strengthen their communication lines, which ultimately enable students to clarify and meet expectations.

Provide Professional Development Opportunities

To become successful scientists, students must master other professional skills that are just as important as competencies at the bench. While these non–bench related professional skills are often not explicitly taught, students are still expected to become proficient in them. Officially, PREPs are 75% research and 25% “further skills development” (NIH, 2016); however, programs providing the most impact likely include relevant instruction and opportunities beyond biomedical course work and GRE preparation, such as opportunities for giving research talks and receiving feedback, poster presentations, scientific writing, interview skills, navigating the research culture, and other “soft skills” that are critical for success and science identity. We found that meeting with students weekly and focusing on developing these skills was important to their success. Not surprisingly, many of these topics were reported by scholars as being most helpful (Figure 4). Alumni indicated an increased appreciation of the benefits of these professional development activities, likely due to an increase in their recognition of the importance of these skills as they matriculated through graduate school.

Recruit Participants Declined for Admission into PhD Programs

While the availability of postbaccalaureate training programs like PREP is becoming more widely known through expanded numbers of programs and recruiting efforts, we have found that many eligible students still do not know these programs exist. Because PREPs are housed at research-intensive institutions, it is likely that these universities also have their own selective biomedical PhD programs. Each year, we contact eligible UR applicants from our own graduate admissions pool who are declined admission to inform them about PREP opportunities (both our program and a link to the NIH PREP listing of all programs). Many of the students who entered our program through this mechanism were considering other career options after being initially declined from graduate programs. PREP, and other postbaccalaureate opportunities, can offer a necessary “plan B” and minimize the likelihood that talented UR scholars with an interest in biomedical research exit the scientific training path. In our experience, these scholars are extremely motivated to improve their competitiveness, and clearly they are interested in pursuing a biomedical PhD, because they already applied. Therefore, we recommend that institutions do not overlook their own PhD applicant pool when recruiting for their postbaccalaureate program.

Institutional Support

UP is intensive not only for the individual scholars but also for the staff themselves. In addition to scheduled events such as the SDS, weekly meetings, social events, IDP meetings, and so on, scholars benefit from regular unscheduled interactions with the director, coordinator, and other staff members to discuss personal challenges, choices, and successes. The director and coordinator also devote many hours each week to evaluating UP applicants, providing individual feedback on assessments and graduate school application drafts, and preparing meeting materials. UNC faculty contribute in essential ways to the program as advisors, panel discussion participants, and mock interviewers in addition to their roles as laboratory mentors, but the research and teaching commitments of tenured faculty preclude making a major time commitment to PREP. NIH funding through the R25 provides partial salary support for program staff, but we find that providing the benefits of scientifically trained staff with the primary task of supporting UP scholars requires additional institutional financial investment. We estimate that replicating UP requires at least one full-time position at a competitive salary; at UNC this effort is divided between two individuals. In addition, UNC institutional funds supplement the R25 to cover the costs of travel and housing for on-campus interviews, a new laptop computer for each scholar, attendance at ABRCMS plus a local conference, and fall semester tuition—typically at the out-of-state rate.

In conclusion, we report a model for a postbaccalaureate training program that has transitioned more than 90% of UR participants from the program into top-tier biomedical PhD programs with 95% retention over the past 5 yr. The success of these first five cohorts has generated considerable enthusiasm by UNC departments, centers, and individual faculty to expand future cohorts. It is our hope that these practices will be replicated or adapted elsewhere to increase participation and success among UR scholars in the biomedical sciences.

ACKNOWLEDGMENTS

REFERENCES