A Social Capital Perspective on the Mentoring of Undergraduate Life Science Researchers: An Empirical Study of Undergraduate–Postgraduate–Faculty Triads

INTRODUCTION

Undergraduate research experiences, during which students conduct research projects in faculty members’ laboratories, are highly valued within the science, technology, engineering, and mathematics (STEM) education community. Undergraduates who participate in research realize a variety of cognitive, behavioral, and affective outcomes, including the development of research and communication skills (Kardash, 2000; Zydney et al., 2002; Bauer and Bennett, 2003; Lopatto, 2004; Russell et al., 2007), increased ability to think and work like a scientist (Seymour et al., 2004; Hunter et al., 2007), the development of a scientific identity (Hunter et al., 2007; Adedokun et al., 2012; Robnett et al., 2015), increased scientific self-efficacy (Chemers et al., 2011; Robnett et al., 2015), and sustained or increased interest in graduate education in science (Zydney et al., 2002; Bauer and Bennett, 2003; Lopatto, 2004; Russell et al., 2007; Schultz et al., 2011; Eagan et al., 2013). The importance of undergraduate research experiences in the persistence and success of STEM students forms the basis for national calls to increase undergraduate research opportunities (e.g., American Association for the Advancement of Science [AAAS], 2011; National Research Council, 2003).

Mentoring is an integral element of the undergraduate research experience, and faculty mentorship has been proposed as an important factor for students to maximize the benefits of participating in research (e.g., Zydney et al., 2002; Taraban and Logue, 2012). However, the number of faculty available to mentor undergraduates limits the number of undergraduates who can participate in internship-style research experiences (Wei and Woodin, 2011). Graduate students and postdoctoral researchers, whom we refer to collectively as “postgraduates” to be consistent with prior research (Dolan and Johnson, 2010), often serve as mentors to undergraduate researchers in order to involve more undergraduates in research, especially at large research universities. Yet there has been little research on the effect of being mentored by a postgraduate. Initial research in this area suggests that postgraduates provide important forms of technical, psychosocial, and informational support to undergraduate researchers (Dolan and Johnson, 2010; Thiry and Laursen, 2011) but also can influence undergraduates negatively by increasing pressure to work long hours, enhancing the sense of hierarchy in lab groups, and varying in their abilities to mentor (Dolan and Johnson, 2010).

Postgraduates who mentor undergraduates at large universities are themselves mentored by faculty members (Whiteside et al., 2007; Thiry and Laursen, 2011). This creates a social structure that we refer to as a “mentoring triad” to distinguish it from dyadic mentoring, in which an undergraduate researcher is mentored directly by a faculty member. Social capital theory is useful for examining how mentoring triads function and how being in different triads may affect the outcomes undergraduates realize from participating in research. Social capital theory posits that beneficial resources are embedded in a person’s social network, and the value derived from a person’s social network is called “social capital” (Lin, 2001). A person’s social ties are considered assets, because they may provide access to helpful resources, such as information about a job opening or influence through a letter of recommendation (Lin, 2001). Thus, the more ties an individual has and the more beneficial those ties are with respect to offering information, advice, and influence, the more social capital the undergraduate is able to accumulate.

Relationships with faculty are an important source of social capital for undergraduates. Faculty serve as institutional agents, helping students find resources and make important connections within their institutions and disciplines (Crisp and Cruz, 2009; Stanton-Salazar, 2011; Eagan et al., 2013). Faculty also provide psychosocial support, such as encouragement and sharing personal stories of struggle, which is thought to be important for encouraging students to persist in the face of difficulty (Crisp and Cruz, 2009). Undergraduates’ social capital in the form of ties to faculty is known to improve their academic and professional outcomes. For example, undergraduates’ interactions with faculty have been shown to predict their retention and persistence in higher education (Pascarella and Terenzini, 2005) and their intentions to pursue graduate study (Eagan et al., 2013). For STEM undergraduate researchers, interactions with faculty have been linked to the socialization of students into science (Hunter et al., 2007; Thiry and Laursen, 2011) and the development of a scientific identity (Carlone and Johnson, 2007), which is an important predictor of students’ intentions to pursue science research–related careers (Chemers et al., 2011; Estrada et al., 2011). Many of these studies have specifically focused on the benefits of research experiences for historically underserved students in science. Because there has been no quantitative study of the influence of mentoring triads on the outcomes of undergraduate researchers, we focus here on characterizing the experiences of undergraduate life science researchers in general rather than with respect to undergraduates’ gender, race, ethnicity, or first-generation status.

Qualitative research indicates that social capital available to undergraduates through ties to postgraduates differs from social capital through ties to a faculty member (Dolan and Johnson, 2010). For example, undergraduates consider faculty mentors to be helpful for answering “big picture” questions (Dolan and Johnson, 2010; Thiry and Laursen, 2011), suggesting that faculty serve as agents for undergraduates to develop a broader conceptual understanding of their research. Undergraduates report that postgraduates are agents for information about the technical aspects of their research projects (Dolan and Johnson, 2010; Thiry and Laursen, 2011). We posit that, because of differences in social capital available from postgraduates and faculty, undergraduates who differ in their connections to postgraduates and faculty members will realize different outcomes from participating in research. Undergraduates mentored by faculty alone may come to understand and value science research but receive insufficient day-to-day guidance on technical aspects of the work that limits their ability to produce results. This could reduce undergraduates’ motivation to continue in research and reduce their scholarly productivity (e.g., authorship on conference presentations or journal publications) and their competitiveness for graduate school. Undergraduates mentored by postgraduates alone may develop technical skills but not a broader understanding of the purpose and value of the research, which may limit their interest in pursuing a research-related career (Estrada et al., 2011).

Mentoring triads can take different forms depending upon the presence of ties between each member of the triad (Figure 1). For example, an undergraduate could be mentored primarily by a postgraduate with little involvement from a faculty member (Figure 1, triad IV) or with substantive interaction between the postgraduate and faculty member about the undergraduate’s research project (Figure 1, triad VII). Members of a mentoring triad can interact directly (e.g., undergraduate to faculty or undergraduate to postgraduate) and indirectly (e.g., undergraduate to postgraduate to faculty). Indirect relationships may serve to strengthen direct relationships (Simmel, 1902). For example, a postgraduate who communicates with the faculty mentor about the undergraduate’s research project may be able to “broker” resources from the faculty mentor to the undergraduate. The postgraduate may ask the faculty mentor a conceptual question on behalf of the undergraduate and then relay the conceptual knowledge back to the undergraduate. This may negate the need for the undergraduate to be directly tied to a faculty mentor (Figure 1, triad VII). In the triad with all three ties (Figure 1, triad VIII), or the “closed” triad, postgraduate and faculty mentors who communicate regularly with their undergraduate protégé and with each other about the undergraduate’s research project may help the undergraduate realize greater outcomes. For instance, regular communication about the undergraduate’s research progress and professional development may help the undergraduate make both technical and conceptual progress in his or her research, which could lead to greater identification with the scientific community and enhanced scholarly productivity. Thus, both direct and indirect connections among members of a mentoring triad have the potential to influence the undergraduate’s outcomes. Understanding how mentoring triads contribute to the outcomes of undergraduate researchers is critical as we consider how to build capacity to effectively involve more undergraduates in research.

Mentoring triads have not been the focus of empirical studies to date. The prevalence of particular triads and the effects of triad structure on undergraduate outcomes are unknown. In addition, little is known about the effectiveness of postgraduate mentors compared with faculty mentors. In this study, we begin to address this knowledge gap by 1) characterizing the prevalence of different mentoring triads at research universities, and 2) identifying the empirical associations, if any, between undergraduates’ membership in particular mentoring triads and the outcomes they realize from participating in research.

We first established which mentoring triads (Figure 1) are common at research universities, where undergraduates are likely to be mentored by postgraduates. We then documented the outcomes of undergraduates in different triads, specifically, undergraduates’ reports of outcomes that are associated with participating in research: 1) ability to think and work like a scientist, 2) scientific self-efficacy, 3) scientific identity, 4) satisfaction with their research experience, 5) career and education preparation, 6) scholarly productivity, and 7) intentions to enroll in a STEM PhD program. Finally, we compared the outcomes of undergraduates in different triads to test hypotheses based on expected differences in the social capital available to undergraduates depending on their triad membership (Table 1). In the following sections, we describe each hypothesis and explain which triads we compared to test the corresponding hypothesis.

Table 1. Triad comparisons used to test hypotheses based on expected differences in the social capital available to undergraduates depending on their triad membership

Mentorship by a Faculty Member versus a Postgraduate

Undergraduate researchers with only one mentor may realize distinct outcomes depending on whether the mentor is a faculty member or postgraduate. For example, an undergraduate may report higher scientific identity following recognition from a faculty mentor than a postgraduate mentor because he or she perceives the faculty member as more of a science expert or as better representing the scientific community. In this case, the scientific identity of undergraduates in the faculty-only triads (triads III/VI) would be greater than those of undergraduates in the postgraduate-only triad (triad IV). In contrast, an undergraduate’s ability to think and work like a scientist may be greater if mentored by a postgraduate than a faculty member, since postgraduates are more likely to spend more time mentoring undergraduates on day-to-day research activities. Here, the undergraduates in the postgraduate-only triad (triad IV) would report higher gains in thinking and working like a scientist than those reported by undergraduates in faculty-only triads (triads III/VI). Although resources from faculty may be brokered through postgraduates, the quality of resources are thought to deteriorate as they are passed from one person to the next (Baker, 1984; Baker and Iyer, 1992). Thus, if undergraduates in the postgraduate-only triad (triad IV) report less positive outcomes compared with undergraduates in faculty-only triads (triads III/IV), outcomes of undergraduates in the triad missing the undergraduate–faculty tie (triad VII) would also be less positive. Hypothesis 1. Direct mentorship by the faculty member versus the postgraduate leads to distinct outcomes (Table 1, panel 1).

Mentorship by Two Mentors versus One Mentor

Mentorship by both a postgraduate and a faculty member, compared with mentorship by solely a postgraduate or a faculty member, gives rise to two competing hypotheses. On the one hand, undergraduates with two ties may realize greater outcomes than undergraduates with one tie simply because they have access to more resources (Table 1, panel 2A). If so, the outcomes of undergraduates with ties to both a postgraduate and a faculty mentor (triad V) should exceed the outcomes of undergraduates with only a faculty mentor (triads III/VI) and undergraduates with only a postgraduate mentor (triad IV). In contrast, undergraduates mentored by both a postgraduate and a faculty member who do not communicate with each other about the undergraduate’s project may receive conflicting information or advice and thus realize lesser outcomes (Table 1, panel 2B). If so, the outcomes of undergraduates in faculty-only triads (triads III/VI) and the postgraduate-only triad (triad IV) would exceed those of undergraduates with both a postgraduate and faculty mentor (triad V). Hypothesis 2A. Undergraduates with two mentors realize greater outcomes than undergraduates with only one mentor (Table 1, panel 2). Hypothesis 2B. Undergraduates with two mentors realize lesser outcomes than undergraduates with only one mentor (Table 1, panel 2).

Mentorship by a Faculty Member and a Postgraduate Who Communicate about the Undergraduate’s Project (Closed Triad)

Coleman (1988, 1990) argues that networks with closure, or groups of people who are all connected to one another as in triad VIII (Figure 1), are the source of social capital. Research in the corporate sector shows that network closure allows for more direct access to information and thus less deterioration of information as it moves from person to person (Baker, 1984; Baker and Iyer, 1992). Research in industry, friend groups, and other social networks (e.g., athletic teams) demonstrates that closure facilitates trust (Granovetter, 1985; Coleman, 1988; Burt, 2000, 2001) and norm setting (Coleman, 1990). This leads to greater cooperation (Jones and George, 1998) and common thinking, knowledge, and values (Sundstrom et al., 1990) that can result in improved team performance (Erdem et al., 2003; Lusher et al., 2014). Although these phenomena have not been examined in academic research settings, we propose these results have relevance to the experience of undergraduate researchers. Specifically, we posit that undergraduates in closed triads not only benefit from direct interaction with faculty and postgraduate mentors but also from direct interaction between the postgraduate and faculty mentors. Discussions between the postgraduate and faculty mentors may yield better insight into what the undergraduate needs and how to help the undergraduate obtain it. If closed mentoring triads foster the development of trust among members and reinforce group norms, undergraduates could realize more positive outcomes, such as greater scientific self-efficacy, scientific identity, and scholarly productivity. If so, the outcomes of undergraduates in the closed triad (triad VIII) should exceed the outcomes of undergraduates in all other triads. Hypothesis 3. Undergraduates mentored by both a postgraduate and faculty member who themselves interact about the undergraduate’s research (“closed triad”) realize greater outcomes than undergraduates in all other triads (Table 1, panel 3).

Here, we present data that were collected as part of a larger study aimed at understanding mentoring of undergraduate life science researchers by postgraduates and faculty. We focus on results that address the goals of this study: characterizing the prevalence of mentoring triads at research universities and identifying any relationships between undergraduates’ triad types and their outcomes.

METHODS

Participants

Participants were U.S. undergraduates who had completed at least one semester or summer of research in the life sciences in the past 2 yr and who reported previously or currently working with a postgraduate and a faculty member in a research experience. We recruited undergraduates directly to minimize bias resulting from faculty choosing to respond about their best undergraduate researchers. We distributed study invitations with a link to the online survey to undergraduate researcher listservs maintained by universities or National Science Foundation–funded Research Experience for Undergraduates (REU) programs. We also recruited at the annual meeting of the Society for the Advancement of Chicanos and Native Americans in Science (SACNAS) and the Association of Biomedical Researchers Conference for Minority Students (ABRCMS). Recruiting was targeted to undergraduates at institutions classified as having very high research activity and high research activity and at doctoral degree–granting institutions to maximize the likelihood of identifying students mentored by postgraduates and faculty rather than by faculty alone. All participants were offered a $20 cash incentive for participating. This study was approved by institutional review boards at the University of Georgia (MOD00000565) and the University of Texas at Austin (2014-06-0094).

A total of 842 undergraduates participated, representing more than 50 institutions in more than 25 states; demographic information for all student respondents are presented in Appendix A in the Supplemental Material. Approximately 2% of the participants were recruited through REU programs and responded about their REU experience; the vast majority of participants responded about research experiences offered through programs at their home institutions. Approximately 86% of the institutions in the data set are public institutions, 14% are Hispanic-serving institutions, and 3% are historically black colleges or universities. The Carnegie classifications of the institutions at which responding students conducted their research are provided by triad type in Table 2. As explained below (see Mentoring Triad Type, Control Variables, and Data Analysis sections), some student responses are not included in the analyses presented here. Demographic information for students included in these analyses is outlined in Table 2. We collected few responses from students who identified as American Indian, Native American, African American, Native Hawaiian, Pacific Islander, Hispanic, or Latino/a. These student populations have been historically underserved by institutions of higher education (Ladson-Billings, 2006). We combined these students into one category termed “underserved” in order to have a sufficiently large sample to conduct the analyses presented here, but we recognize that there may be distinct and important differences among these populations. We categorized students as first-generation to go to college if they indicated that both their mother’s and father’s (or guardian’s, if applicable) educational level was “high school/GED,” “some high school,” or “trade/technical school.”

Table 2. Characteristics of the undergraduate participants (n = 748) used in the analyses: overall and by triad type^a

^aOnly students in triads III/VI, IV, V, VII, and VIII and who had complete demographic data were used. For college GPA, we present means. Institution type is based on the university where the undergraduate conducted research. Research institutes are biomedical institutes for research. “International” indicates students conducted research at a university outside of the United States.

Measures

We collected data via a survey administered through the online, secure, survey service Qualtrics. We queried each undergraduate about his/her mentoring triad and seven possible outcomes he or she could have realized from participating in research. Specific measures are described below, and the items for each outcome measure are included in Appendix A in the Supplemental Material.

Mentoring Triad Type.

We presented the diagram (Figure 1) of all eight triad types to students and asked them to indicate which triad best represented their relationships with the postgraduate and faculty members with whom they worked. Based on preliminary analyses, we used only six of the eight mentoring triad types in the final analyses. We did not include the triads in which the undergraduate had no ties to the faculty and postgraduate mentors (Figure 1, triads I and II) due to low sample size (n = 1 and n = 10, respectively). Additionally, we combined the two triads in which the undergraduate is only connected to a faculty mentor (Figure 1, triads III and VI, which we refer to as triads III/VI), because we thought it unlikely that postgraduates were passing social resources to the undergraduates through the faculty mentors in a way that would influence undergraduate outcomes. To ensure this was a valid assumption, we checked for significant differences between these two triads for all outcomes but found no differences. Finally, we dropped students who chose “Other” or “Prefer not to respond” when asked to identify their mentoring triad (n = 11).

To determine whether undergraduates accurately assessed the presence of a postgraduate–faculty connection of which they were not directly a part, we surveyed undergraduate participants’ postgraduate mentors about the mentoring triads in which they work with an undergraduate and faculty member (n = 511). Of the postgraduates who reported a postgraduate–faculty tie, ∼90% of their undergraduate protégés also reported a postgraduate–faculty tie (unpublished data). Thus, the high congruence suggests undergraduates are generally able to accurately perceive the presence of postgraduate–faculty ties, even though they are not a direct part of these dyadic relationships.

Thinking and Working Like a Scientist.

We measured students’ perceptions of the gains they made in their higher-level research skills using the Thinking and Working Like a Scientist scale from the Undergraduate Research Student Self-Assessment instrument (URSSA; Hunter et al., 2009; Weston and Laursen, 2015). This scale asks students to rate the extent of their gains (from 1 = no gain to 5 = great gain) on eight items such as “analyzing data for patterns,” “formulating a research question that could be answered with data,” “identifying limitations of research methods and designs,” and “understanding the theory and concepts guiding my research project.” Results from Weston and Laursen (2015) indicated that items from the Thinking and Working Like a Scientist scale were highly correlated with items that measured gains in higher-order skills on the URSSA Skills scale. Therefore, we added one item from the Skills scale, “defending an argument when asked questions,” because we felt this was an essential higher-level skill for scientists. We conducted confirmatory factor analysis to ensure that the scale was functioning as a measure of Thinking and Working Like a Scientist with the additional item (see Appendix A in the Supplemental Material for additional details). Cronbach’s α for the nine-item scale was 0.90 (n = 785), indicating high internal consistency.

Scientific Self-Efficacy.

We measured students’ scientific self-efficacy using the six-item Scientific Self-Efficacy Scale (Estrada et al., 2011), which asked students to rate their level of confidence (from 1 = not confident to 5 = very confident) to do scientific tasks, such as their ability to “generate a research question to answer” or “figure out what data/observations to collect and how to collect them.” This scale showed high internal consistency (n = 822; α = 0.90).

Scientific Identity.

We measured student’s scientific identity using the five-item Scientific Identity Scale (Estrada et al., 2011), which asked students to indicate their level of agreement (from 1 = strongly disagree to 5 = strongly agree) to items such as “I have a strong sense of belonging to the community of scientists” and “I have come to think of myself as a ‘scientist.’” This scale showed high internal consistency (n = 811; α = 0.86).

Research Satisfaction.

We measured undergraduates’ satisfaction with their research experience using a modified measure of an academic satisfaction scale (Volkwein and Carbone, 1994), which asked students to indicate their level of agreement (from 1 = strongly disagree to 5 = strongly agree) with these three items: “I am satisfied with my research experience in general,” “I am satisfied with the extent of my intellectual development during my research experience,” and “My research experience has had a positive influence on my intellectual growth.” This scale showed high internal consistency (n = 835; α = 0.91).

Career and Education Preparation.

We used six items to measure undergraduates’ perceptions of how well their research experience prepared them for a career or future educational pursuits. Undergraduates indicated their level of agreement (from 1 = strongly disagree to 5 = strongly agree) with five items from the URSSA (Hunter et al., 2009): “My research experience has prepared me for a job,” “Doing research has confirmed my interest in my field of study,” “Doing research has clarified for me which field of study I want to pursue,” “My research experience has prepared me for advanced coursework or thesis work,” and “My research experience has prepared me for graduate school”; and an additional item: “My undergraduate research experience has prepared me to succeed in an academic career.” We conducted confirmatory factor analysis to ensure that the scale was functioning as a measure of career and education preparation with the additional item (see Appendix A in the Supplemental Material for additional details). These six items showed high internal consistency (n = 708; α = 0.89).

Scholarly Productivity.

We asked students to report how many times (from 0 to 5+ times) they completed each of the following professional activities: “presented a poster or talk as part of a local program or event,” “presented a poster at a regional, national, or international conference,” “presented a talk at a regional, national, or international conference,” “participated in writing a manuscript for publication in a peer-reviewed journal,” and “published an article in a peer-reviewed journal.” We used these data to create a scholarly productivity score for each student. Because the items represented activities considered by the scientific community to range from less influential (presenting a talk or poster at a local event) to more influential (publishing a manuscript), the score represents the level of influence associated with a student’s scholarly products: 0 = no scholarly productivity; 1 = presented at a local event; 2 = presented at a regional, national, or international conference; or 3 = coauthoring a manuscript. A student’s score did not take into account the number of times he or she was involved in creating any particular scholarly product.

Intentions to Enroll in a STEM PhD Program.

We used an item from the URSSA (Hunter et al., 2009; Weston and Laursen, 2015) to measure the impact of the research experience on undergraduates’ intentions to enroll in a STEM PhD program. Students were asked to compare their current intentions with their intentions before the research experience by indicating how much more likely (from 1 = not more likely to 5 = extremely more likely) they were to “enroll in a PhD program in science, mathematics, or engineering.”

Control Variables

We anticipated that students’ characteristics beyond their mentoring triad would affect the outcomes they realized from participating in research. Thus, we included gender, race/ethnicity, first-generation status, college grade point average (GPA; mean-centered), number of prior research experiences, duration of the research experience, and whether the student was in an honors program as control variables in the regression analyses described in the Data Analysis section. Because few students chose “Other” for gender or race/ethnicity, we omitted these students from the analyses due to concerns of low statistical power (n = 7). We combined students who had two prior research experiences with students who had three or more prior research experiences, because very few students fit the latter category. We included prior research experience and duration of the research experience in the regressions to control for differences in outcomes that may arise if the outcome accrues over the number or length of the research experience. Institutional characteristics have been found to influence undergraduate researcher outcomes (e.g., Eagan et al., 2013), but we found more variation among individuals at an institution than across institutions (maximum intraclass correlation coefficient among continuous outcomes: 0.022). In addition, several institutions were represented by only one student in the sample. Thus, we chose not to use multilevel models in which the institution where the undergraduate did research was included as a random variable.

Important variation can exist among different student groups at the intersection of race/ethnicity and gender and race/ethnicity and first-generation status. For example, gender and race/ethnicity interact to influence the highest math course taken in high school (Riegle-Crumb, 2006), the likelihood of majoring in a STEM field (Riegle-Crumb and King, 2010; O’Brien et al., 2015), and levels of STEM self-confidence (Litzler et al., 2014). Additionally, race/ethnicity and first-generation college status interact to influence the saliency of a student’s identity as a first-generation college student (Orbe, 2004). There has been little work done specifically exploring the effect of intersectionality on undergraduate researcher outcomes, but we considered it important to account for any variance in outcomes associated with intersectionality. Thus, for each regression, we tested whether to include a race/ethnicity × gender interaction term and a race/ethnicity × first-generation interaction term in the model using Wald tests and likelihood ratio tests (see Appendix A in the Supplemental Material for details). A significant race/ethnicity × first-generation interaction effect was found for the Thinking and Working Like a Scientist and the Career and Education Preparation scales, so this interaction term was included as a control variable in these two analyses.

Data Analysis

We included only students who were not missing data from any of the independent variables or from any item on the scale of the dependent variable in the analyses. We checked for multicollinearity among the independent variables in each final model by calculating the generalized variance-inflation factor (Fox and Monette, 1992) for each variable, but we found no problems with multicollinearity. A correlation matrix that includes both the independent and dependent variables is presented in Appendix B in the Supplemental Material. All analyses were conducted in R, version 3.1.2 (R Core Team, 2014).

We analyzed the seven undergraduate outcomes with separate regressions followed by contrast analyses to test the three hypotheses about triad membership. In regression, a reference group must be established for each categorical predictor variable. The regression output provides statistical information comparing each group of the categorical predictor with the designated reference group. We used the closed triad (triad VIII) as the reference group for our triad predictor variable in all regressions. Thus, all other triad types were compared with the closed triad, testing hypothesis 3 simply by running the regressions. To test hypotheses 1 and 2, we conducted four contrasts (hypothesis 1: triads III/VI vs. IV and III/VI versus VII; hypothesis 2: triads V vs. III/VI and V vs. IV; see Table 1) for each regression model by comparing the relevant triad regression coefficients (multcomp package; Hothorn et al., 2008). When doing multiple contrasts, the probability of a type I error (a false positive) increases with the number of contrasts conducted. Therefore, it is important to control the overall α level across all contrasts (i.e., the family-wise error rate) at 0.05, rather than use an α of 0.05 for each individual contrast. We controlled the family-wise error rate for the four contrasts at an overall α level of 0.05 by using a single-step p-value adjustment (Hothorn et al., 2008). Contrast estimates are the difference in regression coefficients between the two triads being compared and are equivalent to the regression coefficients that would be obtained if one of the triads used in the contrast was designated the reference group in the regression.

We used multiple linear regression to predict the effects of mentoring triad membership on the five outcomes that were continuous variables: thinking and working like a scientist, scientific self-efficacy, scientific identity, research satisfaction, and career and education preparation. We took the mean of the responses to the items on each scale as a measure of the outcome (range = 1–5). We checked the assumptions for each linear regression, and in most cases, the error terms were heteroskedastic. Although heteroskedasticity does not result in biased parameter estimates, it does result in biased SEs, upon which tests of significance are based (Long and Ervin, 2000). Therefore, we used heteroskedastic-consistent SEs, calculated using the HC3 estimator (MacKinnon and White, 1985), to test for the significance of variables in the regression. Mentoring triad effect sizes were calculated for each outcome using Cohen’s f². R² from regression models with (full) and without (nested) triad in the model were used to calculate Cohen’s f²: (R²_full − R²_nested)/(1 − R²_full) (Cohen, 1988; Selya et al., 2012). Cohen (1988) suggests that effect sizes of 0.02 are small, 0.15 are medium, and 0.35 are large.

We used ordinal logistic regression (MASS and ordinal packages; Venables and Ripley, 2002; Christensen, 2015) to examine the effect of triad membership on scholarly productivity, because the scholarly productivity score is an ordered categorical variable. Ordinal logistic regression assumes proportional odds. That is, the relative odds of a response of a dependent variable level at least one level higher is the same for all levels of the dependent variable. This assumption results in fewer parameter estimates than a multinomial model, in which categorical outcomes are not ordered and separate relative odds must be calculated for each dependent variable level compared with a reference level. We used Akaike’s information criterion (AIC) values to determine whether the more complex multinomial model was a better fit to the data than the simpler ordinal logistic model (Fox and Weisberg, 2011). The ordinal logistic model had a lower AIC value than the multinomial model (1922.04 vs. 1938.07, respectively), indicating the ordinal logistic regression model was appropriate.

We used binary logistic regression to examine the effect of an undergraduate’s triad on his or her intentions to enroll in a PhD program in science, mathematics, or engineering. The original survey item had five response options. However, for some of the triads with low sample sizes, there were few responses for some of the response options. Therefore, we combined response options to create two outcome groups: not more likely and a little/somewhat/much/extremely more likely. Because the item was meant to determine whether the research experience had influenced their decisions to enroll in a PhD program, combining all of the positive responses creates a “more likely” category that we can easily compare with a “not more likely” category.

RESULTS

Undergraduate researchers most frequently reported being part of the closed triad (triad VIII; Figure 2). Almost twice as many students reported being in a closed triad compared with the second most common triad (triad VII), in which the undergraduate is tied to the postgraduate and the postgraduate is tied to the faculty member, but there is no tie between the undergraduate and faculty member. The remaining triads contained fewer than 40 students each (<5% of students in each triad), indicating students did not frequently experience these mentoring triads at research universities.

In general, students reported high gains in being able to think and work like a scientist, scientific identity, research satisfaction, and career preparation (Table 3). They reported moderately high levels of scientific self-efficacy. These values are similar to the values observed for other national samples of undergraduate researchers using the same or very similar measures (T. Weston, personal communication concerning unpublished data from URSSA; Estrada et al., 2011). Consistent with our overarching hypothesis for this study, undergraduates’ outcomes varied by their triad type; triad type was a significant predictor in all models. The effect sizes of triad type ranged from small (f² = 0.041 for scientific self-efficacy) to medium-small (f² = 0.087 for thinking and working like a scientist) for the linear models (Appendix B in the Supplemental Material), which is not surprising, given the many facets of a research experience that are likely to influence undergraduate researchers’ outcomes. Appendix B provides the regression tables, which include regression coefficients for triad and all control variables.

Hypothesis 1. Direct mentorship by the faculty member versus the postgraduate leads to distinct outcomes

The undergraduates in this study who were mentored solely by a faculty member versus solely by a postgraduate realized the same positive outcomes (Table 4). However, small sample sizes of undergraduates in less common triads (triads III/VI and IV) increase the probability of a type II error (false negative). A post hoc power analysis, using G*Power (Faul et al., 2007) indicated that, to achieve an 80% chance of correctly rejecting the null hypothesis with an α = 0.05, the effect size would have to be medium (Cohen’s d = 0.5) or greater for the triads III/VI versus VII contrasts and medium-large or greater for the triads III/VI versus triad IV contrasts. This is a conservative estimate, since each contrast for an outcome used an adjusted α < 0.05 for significance tests to preserve the overall family-wise error rate at 0.05. Small effect sizes can be important in education research and are common in complex social science models. Therefore, rather than solely relying on significance tests that cannot detect small effects, we examined the contrast estimates holistically and found that undergraduates in faculty-only triads (triads III/VI) generally reported more positive outcomes than undergraduates in postgraduate-only triads (triads IV and VII). Additional studies with larger sample sizes are necessary to determine whether this trend is noteworthy.

Table 4. Contrast tests to determine whether direct mentorship by a faculty member versus a postgraduate leads to distinct outcomes^a

^aContrast estimates (± SE) based on differences in the regression coefficients for the named triads are shown. All control variables were included in the estimates. Positive numbers indicate that undergraduates in the left triads reported more positive outcomes than undergraduates in the right triad. Negative numbers indicate that undergraduates in the left triads reported less positive outcomes than those in the right triad. †, continuous variable; ‡, ordinal variable; §, dichotomous variable.

Hypothesis 2A. Undergraduates with two mentors realize greater outcomes than undergraduates with only one mentor

For the most part, undergraduates mentored by both a postgraduate and a faculty member (triad V) reported statistically similar outcomes to undergraduates mentored solely by a postgraduate (triad IV) or solely by a faculty member (triads III/VI; Table 5). However, undergraduates in the triad with both postgraduate and faculty mentors (triad V) were approximately four times more likely than undergraduates with just a postgraduate mentor (triad IV) to report increased intentions to enroll in a PhD program after completing the research experience. Having two mentors may be additive in terms of an undergraduate’s intentions to enroll in a PhD program. Alternatively, the undergraduates’ direct connection to faculty mentors may be driving this result. Specifically, undergraduates with a faculty mentor (triads III/VI) were approximately three times more likely than undergraduates with a postgraduate mentor (triad IV) to report increased intentions to enroll in a PhD program after completing the research experience (Table 4). Although this difference is not statistically significant (p = 0.094), it suggests that faculty may influence students’ intentions to enroll in a PhD program, especially given the small sample sizes for the triads in this contrast.

Table 5. Contrast tests to determine whether undergraduates realize different outcomes when they have more than one mentor^a

^aContrast estimates (± SE) based on differences in the regression coefficients for the named triads are shown. All control variables were included in the estimates. Positive numbers indicate that undergraduates in the left triad reported more positive outcomes than those in the right triad(s). Negative numbers indicate that undergraduates in the left triad reported less positive outcomes than those in the right triad(s). †, continuous variable; ‡, ordinal variable; §, dichotomous variable.

Hypothesis 2B. Undergraduates with two mentors realize lesser outcomes than undergraduates with only one mentor

Undergraduates who reported having a postgraduate and a faculty mentor who did not interact directly with each other (triad V) showed a trend toward lower outcomes than undergraduates with just a faculty mentor. Although these differences were not significant, the small sample sizes for all triad types in these contrasts resulted in a low probability of detecting a true significant result unless the effect sizes were medium-large or greater. Postgraduates and faculty who do not communicate with each other about undergraduates’ research may give conflicting advice, resulting in a less positive research experience. Alternatively, the lack of communication between postgraduate and faculty mentors may be a sign of a less positive work environment in general, which would limit the quality of the undergraduates’ research experiences and their outcomes. Additional data are needed from undergraduates who experience this triad to determine whether this trend holds with a larger sample.

Hypothesis 3. Undergraduates mentored by both a postgraduate and a faculty member who themselves interact about the undergraduates’ research (“closed triad”) realize greater outcomes than undergraduates in all other triads

Undergraduate researchers in the closed triad often had greater outcomes than students in the other triads (Table 6). In particular, all undergraduates’ outcomes were significantly greater in closed triads (triad VIII) compared with triads in which the undergraduate was mentored solely by a postgraduate (triad IV). The outcomes of undergraduates in closed triads were also greater than the outcomes of undergraduates in triads with indirect ties to the faculty mentor through the postgraduate (triad VII). The positive outcomes of undergraduates in closed triads could not be explained simply as having access to two mentors, because three of the seven outcomes were greater for undergraduates in the closed triad (triad VIII) than for undergraduates tied to both a postgraduate mentor and a faculty mentor who were not tied to each other (triad V). For the most part, undergraduates who were directly tied to a faculty member (triads III/VI) realized similar outcomes to undergraduates in the closed triad (triad VIII). These results suggest that much of the value undergraduates reap from being in the closed triad may be through direct interaction with the faculty mentor. Alternatively, closed triads may be an indicator of faculty members’ involvement or investment in the research group or the esprit de corps of the lab, which results in a higher-quality experience for the undergraduate researcher. The only exception was for the outcome of thinking and working like a scientist; undergraduates in closed triads reported greater gains in this area than undergraduates in any other triad.

Table 6. Contrast tests to determine whether undergraduates in a closed triad realize greater outcomes than undergraduates in all other triads^a

^aContrast estimates (± SE) based on differences in the regression coefficients for the named triads are shown. All control variables were included in the estimates. Positive numbers indicate that undergraduates in the left triad reported more positive outcomes than those in the right triad(s). Negative numbers indicate that undergraduates in the left triad reported less positive outcomes than those in the right triad(s). †, continuous variable; ‡, ordinal variable; §, dichotomous variable.

Limitations

We used several methodological approaches to ensure the validity of our results. First, when possible, we made use of valid and reliable measures of the outcomes undergraduates realize from participating in research. We chose to focus on outcomes that were well established in the literature on undergraduate research experiences to allow for comparison of our work with published research. We identified participants by using diverse recruiting strategies and included data from a nationally representative sample of undergraduate life science researchers at research universities. We used several statistical tests to determine the appropriateness of the assumptions we made in conducting the regression analyses. We recognize, however, that our study design and methods are inherently limited in ways that may affect the interpretation of our results.

We surveyed undergraduates once after the completion of at least one semester of undergraduate research. This cross-sectional approach does not account for what students brought into the research experience but rather assumes outcomes were the result of the research experience the undergraduates were reporting. For some measures, this is likely valid, as we explicitly asked students to reference their research experience when responding to items (see Appendix A in the Supplemental Material). However, the scientific self-efficacy and scientific identity measures were more general and were not explicitly tied to a student’s research experience. To account for this, we included the number of research experiences and duration of research experiences as control variables in all analyses. Yet we cannot say with certainty that membership in a triad preceded the outcomes within the research experience itself. For example, students with high self-efficacy or identity may seek out opportunities to interact with faculty directing their research, thus generating a closed triad situation. Future research should employ longitudinal studies that survey students before and after their research experiences and make use of structural equation modeling to determine directionality.

We made use of measures that rely on undergraduates’ self-reporting on their research outcomes. Concerns have been raised by us and others about the validity of self-reported learning gains, since correlations vary widely between self-reported measures of learning and more direct measures such as tests or expert judgments (Falchikov and Boud, 1989; Corwin Auchincloss et al., 2014). Only one outcome reported here is a self-report of learning: thinking and working like a scientist (Hunter et al., 2009; Weston and Laursen, 2015). This measure is used to document outcomes of students in all NSF-funded BIO REU sites. By using this measure, our results can be compared with results from national samples of undergraduate researchers in the life sciences. Similar concerns could be raised about relying on students’ intentions to enroll in a PhD program rather than tracking students to document actual enrollment. Eagan and colleagues (2013) also relied on students’ graduate school aspirations in their national study of the impact of undergraduate research programs, arguing that intentions to enroll in graduate school were the best predictors of actual enrollment and an essential first step for subsequent enrollment. Self-report is widely accepted for documenting psychosocial outcomes such as scientific self-efficacy and scientific identity.

We included specific language in the study invitation aimed at encouraging participation by students with poor research experiences, but our data set contains few undergraduates who reported having a negative experience. Only 49 students indicated their satisfaction with their research experience was less than neutral (score < 3). Thus, the prevalence of triads as well as the outcomes of undergraduates in different triads reported here may be biased by underrepresentation of students with negative research experiences. It is possible that undergraduates with negative experiences opted out of our study because they had to call to mind the postgraduates and faculty with whom they worked. Undergraduates with negative experiences may also have been less motivated to respond due to concerns about potential retaliation by the postgraduate or faculty mentor. This adds a layer of sensitivity that might not be an issue in more general studies of undergraduate research experiences. It is important to note that undergraduates in our study reported average outcomes that were very similar to average outcomes reported by other national samples of undergraduate researchers (e.g., T. Weston, personal communication concerning unpublished data from URSSA; Estrada et al., 2011). This indicates that any bias due to undergraduates with negative experiences opting out of the study is not unique to our sample.

Finally, our study examined mentoring triads primarily at research universities. Our inferences about undergraduate–faculty interactions and their effects are not likely to apply to undergraduates who participated in research in labs without a postgraduate, either at research universities or at other types of institutions. Further research is needed to compare the effects of different mentoring structures across institution types.

DISCUSSION

Postgraduates, who are themselves mentored by faculty members, commonly serve as mentors to undergraduate life science researchers at research universities (Whiteside et al., 2007; Thiry and Laursen, 2011). This is the first quantitative study to examine how this triadic mentoring relationship contributes to the outcomes undergraduates realize from participating in research, even though this relationship is commonplace at research universities. Although undergraduates in this study reported being in all eight possible mentoring triad structures (Figure 2), only two were commonly reported: the closed triad, in which all three members are tied (triad VIII), and the triad with undergraduate–postgraduate and postgraduate–faculty ties, but no undergraduate–faculty tie (triad VII).

Undergraduates in the most common triad (VIII), the closed triad, reported more positive outcomes than undergraduates in the second most common triad (VII), the triad with an undergraduate–postgraduate connection and a postgraduate–faculty connection but no undergraduate–faculty connection. Differences in outcomes between these two triads were found even after controlling for differences among students in cultural background and social identity (gender, race/ethnicity, first-generation status), in academic achievement (college GPA, honors status), and in the extent of their research experience (number of prior research experiences, duration of research experience). These results indicate that postgraduates are not able to fully broker the resources that faculty can offer. The benefit of the closed triad could not be fully explained by the presence of the undergraduate–faculty tie, since undergraduates in the closed triad reported greater gains in thinking and working like a scientist (p < 0.05) than undergraduates mentored by faculty alone (triads III/VI). Nor could the benefit of the closed triad be fully explained by simply having two mentors, as undergraduates in the closed triad reported greater gains in thinking and working like a scientist (p < 0.001), greater science self-efficacy (p < 0.01), and greater research satisfaction (p < 0.05) than undergraduates mentored by a postgraduate and a faculty member who themselves did not interact (triad V). In fact, our data suggest that a triad in which undergraduates have both postgraduate and faculty mentors who do not interact with each other about the undergraduate’s research (triad V) may be less favorable than a triad in which an undergraduate has only a faculty mentor. These triad V undergraduates reported less positive outcomes than undergraduates who were mentored directly by faculty alone (triads III/VI). Although these contrasts were not significant, we did not have the sample sizes necessary to obtain the power needed to identify significant effects of small or medium magnitude. Future research should examine this trend with larger sample sizes.

The beneficial nature of the closed triad is consistent with Coleman’s (1988, 1990) argument that closed networks are the source of social capital. Dense social networks, which are characterized by the prevalence of closed triads, result in mutual trust and obligation among all members (Coleman, 1988). With respect to the undergraduate–postgraduate–faculty relationship, a closed triad may allow for the development of greater rapport between the undergraduate and his or her mentors, which leads to the provision of greater professional opportunities and the realization of more positive outcomes. For example, a faculty member and postgraduate who trust an undergraduate researcher more may be willing to give him or her more complex and rewarding research tasks or allow him or her to make independent decisions, leading to greater learning, more research progress, and new professional opportunities (e.g., Seymour et al., 2004). Additionally, in a closed network, everyone is more likely to be “on the same page,” because everyone is communicating and working toward a collective goal (Coleman, 1988). Faculty and postgraduate mentors who do not communicate with one another about their undergraduate protégés’ research are more likely to provide conflicting advice or information. An undergraduate may then become unsure of who should or should not be a source of advice or information. Based on our findings, we recommend the formation of closed triads in which postgraduates play a prominent role in the mentoring of undergraduate researchers but faculty also interact directly with undergraduates. We also recommend that postgraduates and faculty set regular meetings to discuss undergraduate researchers’ projects and progress and for faculty to provide guidance to postgraduates on how to mentor. Further research is needed to discern the frequency and nature of undergraduate–faculty interactions and delineate the specific functions of postgraduate–faculty interactions (e.g., information sharing, mentoring) necessary for undergraduates to maximally benefit from their research experiences.

If we look holistically at the triads compared in Table 6, there is evidence that faculty offer unique resources that help undergraduates benefit maximally from research experiences. Undergraduates in the closed triad (VIII) and the faculty-only triads (III/VI) achieved the most positive outcomes, while undergraduates with a direct tie only to a postgraduate (triads IV and VII) achieved the least-positive outcomes. We likely did not find differences between faculty-only (III/VI) and postgraduate-only (IV and VII) triads because the power of this analysis was limited by the small sample sizes of faculty-only triads (n = 40–45). However, consistently positive contrast estimates (Table 4) suggest that undergraduates who work directly with faculty may experience greater outcomes than those who work only with a postgraduate. For example, scientific identity was greater in students with a faculty mentor than a postgraduate mentor (triad IV: contrast estimate = 0.375, p = 0.104; triad VII: contrast estimate = 0.223, p = 0.146). Development of a scientific identity is particularly important for the persistence of students in STEM. Students who report greater scientific identity are more committed to pursuing a science career after college (Chemers et al., 2011; Estrada et al., 2011). Even small differences in scientific identity can discriminate between students who stay in science and students who leave science after college (Estrada et al., 2011). Recognition from established members of the scientific community has been shown to influence a student’s scientific identity (Carlone and Johnson, 2007; Hurtado et al., 2011). Because faculty are more established members of the scientific community than postgraduates, faculty recognition may be an important form of capital for students to develop their own identities as scientists. Studies that include more faculty-only triads are necessary to test this hypothesis.

Given the positive outcomes of undergraduates in closed triads and the trend in positive outcomes reported by undergraduates who interact directly with faculty, future research is needed to understand the factors that influence whether an undergraduate reports a tie with his or her faculty mentor. It is standard practice in social network analyses to measure the strength of a tie based on the frequency of interaction or the “closeness” of the relationship (Granovetter, 1973; Marsden and Campbell, 1984). The size of the research group and norms within the particular life science disciplines are likely to influence this relationship and affect what an undergraduate researcher might consider a meaningful tie with a faculty member. Perhaps one intensive week of side-by-side fieldwork is sufficient for an ecology undergraduate to perceive a tie with a faculty member, while weekly one-on-one interactions are necessary for a biochemistry undergraduate to report a faculty tie. An undergraduate conducting research in a large group may report a faculty tie based on only a handful of interactions over the semester, while an undergraduate in a small research group may not perceive a tie unless he or she interacts with a faculty member every week. Alternatively, it may be necessary for an undergraduate to feel some level of comfort or deep level similarity to a faculty member to form a productive tie (Eby et al., 2013). Future research should explore how interaction frequency and relationship quality, as well as other contextual factors such as discipline and research group size, influence undergraduates’ perceptions of their ties to faculty mentors and their realization of particular outcomes.

Even though our results indicate that direct undergraduate–faculty interaction is important, undergraduates in this study reported positive outcomes regardless of triad type. This suggests that postgraduates are able to provide the social resources (e.g., information, advice, psychosocial support) that undergraduates need to reap many of the benefits of research experiences. Even undergraduates with a single postgraduate mentor reported generally positive outcomes. For example, the average undergraduate researcher in triad VII (i.e., undergraduate–postgraduate and postgraduate–faculty ties, but no undergraduate–faculty tie, with all control variables taken into account) reported high gains in thinking and working like a scientist (mean = 3.889 on scale of 1–5) and a high scientific identity score (mean = 3.868 on scale of 1–5). These results provide the first empirical support from the undergraduate perspective for involving postgraduates as mentors in order to broaden the availability of undergraduate research opportunities (AAAS, 2011; President’s Council of Advisors on Science and Technology, 2012). The positive outcomes of undergraduates mentored by postgraduates should assuage concerns about postgraduates’ capabilities to mentor undergraduate researchers (Dolan and Johnson, 2010).

Mentoring triad membership accounted for a small proportion of the variance associated with each outcome, likely due to the many factors that influence the outcomes an undergraduate realizes from participating in research. Factors outside the research experience per se are also likely to be influential. For example, Eagan and colleagues (2013) found that initial aspirations for an advanced degree, race/ethnicity, and mother’s education level affected a STEM student’s intentions to pursue a graduate or professional degree above and beyond the effect of a research experience. In our analyses, student demographics significantly predicted some of the outcomes. Yet, even including all control variables, the total variance explained in undergraduates’ outcomes was relatively small (range of R² for regressions: 0.090–0.140; see Appendix B in the Supplemental Material). Finer characterization of research experiences in general and research mentoring in particular is likely to be necessary to more fully explain undergraduate researchers’ outcomes. For example, Thiry and Laursen (2011) found that undergraduate researchers benefited from particular types of mentoring help, such as professional socialization, intellectual support, and personal support. Future research should move beyond linking the presence of a mentor to undergraduate researchers’ outcomes and elucidate aspects of the mentoring relationship that are important for undergraduates’ intellectual, personal, and professional growth.

ACKNOWLEDGMENTS

REFERENCES