Increasing Research Productivity in Undergraduate Research Experiences: Exploring Predictors of Collaborative Faculty–Student Publications

INTRODUCTION

Considering the rapid pace of innovation in medical science, technology, and practice, developing a pool of talented biomedical scientists in the United States is of paramount importance (Kaiser, 2011; Pool et al., 2016). Research experiences at the undergraduate level are essential to nurturing biomedical science talent (National Science Foundation [NSF], 2003, 2004; Jones et al., 2010). Benefits of participation in undergraduate research for students have been well documented (Kardash, 2000; Seymour et al., 2004; Hunter et al., 2007; Kuh and Nelson Laird, 2007; Bauer and Bennett, 2008; Cole and Espinoza, 2008; Russell, 2008; Espinosa, 2009; Laursen et al., 2010; Lopatto, 2010; Adedokun et al., 2013; Eagan et al., 2013; Madan and Teitge, 2013; Linn et al., 2015). Only a few studies have been conducted on the benefits of undergraduate research for faculty mentors. For example, Adedokun et al. (2010) identified contributions to faculty research and interpersonal gains as two types of benefits for faculty mentors. Dolan and Johnson (2010) found that faculty members develop more sustained and intimate relationships with undergraduates through research and that undergraduate students bring diversity to faculty-led research groups.

Studies of undergraduate research have rarely included a focus on collaborative faculty–student publications, yet publishing original research together is valuable for faculty mentors and their students, and collaboration tends to enhance overall research productivity (Pravdić and Oluić-Vuković, 1986; Bozeman and Corley, 2004; Lee and Bozeman, 2005; Wuchty et al., 2007). Publishing research is of central importance to most faculty members, because peer-reviewed publications are usually the main factor considered in faculty hiring, tenure, and promotion (Jacobs and Winslow, 2004; Potter et al., 2011; van Dijk et al., 2014). In some institutions, faculty–student collaborative publications are an especially valued activity for faculty promotion and tenure. Scholarly publishing helps student mentees gain critical, logical, analytic, and scientific thinking skills (Lei and Chuang, 2009); compete more successfully as applicants for graduate school (Davis and Warfield, 2011); and become more successful in their research careers (Laurance et al., 2013; van Dijk et al., 2014; Horta and Santos, 2016). In sum, faculty–student publications are an important outcome of undergraduate research, yet factors predicting such publications have been largely overlooked in the literature.

While no studies have examined correlates of faculty–undergraduate student publications, many have identified salient faculty characteristics that influence research publications. These factors are likely important in understanding faculty–undergraduate student publishing and include demographic attributes (Sax et al., 2002; Pashkova et al., 2013; Tomei et al., 2014; Tschannen et al., 2014), years of employment (Taylor et al., 2006; Huang et al., 2015), and funding/principal investigator (PI) status (Gaughan and Bozeman, 2002; Godin, 2003; Lee and Bozeman, 2005). In addition, faculty members’ commitment to undergraduate students and the scientific community may also influence their likelihood of publishing with students. Sax and colleagues (2002) found a positive effect of faculty commitment to student development on faculty research productivity. Other studies suggest that biomedical faculty who place greater value on diversity in the academy are more likely to involve undergraduates in their research teams (Morales et al., 2016, 2017), which might increase their likelihood of publishing papers with students. Involving minority students could also improve the productivity of the research teams, because team diversity has been shown to improve research performance (Barjak and Robinson, 2008). Mentoring minority students also helps faculty members to improve their strategies for faculty–student research collaborations (Muller, 2006). Aside from a faculty mentor, an undergraduate student might also be mentored by a graduate student or postdoctoral researcher (often referred to as a bench mentor). Previous studies show that the research experiences of both faculty mentors and undergraduate mentees may be greatly influenced by bench mentors (Dolan and Johnson, 2010). Hence, the involvement of bench mentors could also influence faculty–undergraduate collaborative publishing. Finally, institutional and disciplinary affiliations also influence faculty publications beyond the individual level (Sax et al., 2002; Taylor et al., 2006).

This study focuses on predicting collaborative biomedical faculty–student publications resulting from mentored undergraduate research across 13 U.S. research universities. It is responsive to Burks and Chumchal’s (2009) call to conduct studies on this topic. We answer the question: Why are some biomedical faculty mentors more productive than others, in terms of publishing research findings with their undergraduate student mentees?

METHODS

Data Collection

We collected our data through a cross-sectional institutional review board–approved Web survey (IRB of the University of Texas at El Paso protocol no. 00001224). We designed the survey to cull information on potential faculty mentors at Arizona State University, Baylor College of Medicine, Clemson University, Rice University, the University of Arizona, the University of Connecticut Institute for Clinical and Translational Science, the University of New Mexico Main Campus, the University of New Mexico Health Sciences Center, the University of Texas–Austin, the University of Texas–Arlington, the University of Texas at El Paso (UTEP), the University of Texas–Southwestern, and the University of Texas Health Sciences Center at Houston–School of Public Health. These institutions are currently linked through the National Institutes of Health (NIH)-funded BUILD (Building Infrastructure Leading to Diversity) program, which is an initiative to address the lack of diversity in the U.S. biomedical research workforce (Kaiser, 2011; Oh et al., 2015).

To create the sampling frame, we asked our primary contact at each institution for a list of faculty members who conducted biomedical research, which resulted in 887 potential respondents. To administer the survey, we followed established Web survey protocols (Cook et al., 2000; Dillman, 2007; Manfreda and Vehovar, 2008). Faculty completing the survey in its entirety received a $10 gift card via a follow-up email. The survey was conducted in two rounds: the first round included six institutions and was open from mid-November through mid-December 2013; and the second round included another seven institutions and was open from mid-January through mid-February 2014. We used Qualtrics Survey Software and pilot tested the survey with a group of faculty at UTEP. Overall, the response rate across all institutions was 60% (with range of 42–100%). A total of 536 faculty members completed the survey.

Variables

Table 1 provides descriptive statistics for each variable. The dependent variable was constructed from a question that asked, “Over the past 5 years, how many publications have you coauthored with undergraduate students?” The variable has four categories (0 = no papers; 1 = 1–4 papers; 2 = 5–9 papers; 3 = 10 or more papers). Among the 536 faculty members, 39% had no publication; 42% had 1–4 publications; 11% had 5–9 publications; and 8% had 10 or more publications. Independent variables were grouped into three categories: faculty–student collaboration, faculty commitment to undergraduate students, and faculty characteristics.

Faculty–Student Collaboration Variables

Average Duration of Relationship.

Based on the survey question “Over the last 5 years, what was the average duration of your typical research mentoring relationship with an undergraduate student?,” we recoded responses into three categories (each coded 0 = no or 1 = yes): “1 summer or less,” “more than 1 summer and equal to or less than 1 year,” and “more than 1 year.” The reference group is “more than 1 summer but less or equal to 1 year.”

Number of Undergraduate Students.

This variable was constructed from a question that asked, “Over the past 5 years, how many undergraduates have you worked with on research?” The variable is coded based on four survey response options (0 = no students; 1 = 1–4 students; 2 = 5–9 students; 3 = 10 or more students).

Number of Graduate Students.

This variable was constructed from a survey question: “Over the past 5 years, how many graduate students have you worked with on research?” The variable is coded based on four survey response options (0 = no students; 1 = 1–4 students; 2 = 5–9 students; 3 = 10 or more students).

Formal Program.

This variable was constructed from the survey question “Have you ever mentored undergraduate student researchers through any formal programs (e.g., Research Initiative for Scientific Enhancement (RISE) program, the Minority Access to Research Careers (MARC) program, the Louis Stokes Alliance for Minority Participation (LSAMP) program, the Research Experiences for Undergraduates (REU) program,¹ or any internal undergraduate research programs)?” Responses were coded into two categories (0 = no or 1 = yes).

Faculty Commitment to Undergraduate Students Variables

Benefits of Mentoring.

We used two survey items prefaced with the statement “Please rate the extent to which the following items are benefits that you receive from working with undergraduate students on research projects,” which were “I enjoy teaching students about research” and “I receive help from undergraduates on my research.” Both variables were rated on four-point Likert scales (1 = strongly disagree to 4 = strongly agree).

Barriers to Mentoring.

We used two survey items prefaced with the statement: “Please rate the extent to which the following items are barriers that you face in including undergraduate students in your research projects,” which were “supervising undergraduate research is time-consuming” and “research by undergraduates is often of low quality.” Both variables were rated on four-point Likert scales (1 = strongly disagree to 4 = strongly agree).

Demographics of Undergraduates.

We created six variables related to the demographic characteristics of each faculty member’s undergraduate mentees by using responses from the following questions: “Have you ever mentored a… 1) Native American student/2) Hispanic/Latino student/3) Black/African-American student/4) female student/5) LGBT [lesbian/gay/bisexual/transgender] student/6) student with disability…on a research project?” Each variable had two response options (0 = no or 1 = yes).

Faculty Characteristics

Sex.

The sex variable was constructed from the survey question “What is your sex?” Females were coded as 1 and males were coded as 0.

Race/Ethnicity.

We used two survey questions: “What is your race?” (Response options: white, Black, American Indian, Asian, Pacific Islander, and other) and “Are you of Hispanic, Latino or Spanish origin?” (Response options: yes or no). The cell sizes for Blacks (2% of sample), Native Americans (1%), Pacific Islander (0%), and others (1%) were too small to analyze separately. Thus, we recoded the data into four mutually exclusive categories (each coded 0 = no or 1 = yes): Asian non-Hispanic (11%), Hispanic (20%), and other (which includes Black and Native American; all are nonwhite and non-Hispanic) (4%); and white non-Hispanic (53%), which is used as the reference group.

H-Index Score.

We used the H-index (Hirsch, 2005) as a measure of underlying research productivity, which gauges scientific output based on the number of indexed papers and the number of citations each paper receives. We retrieved H-index scores for all respondents by searching respondent names in Scopus. Scopus provides reliable and reproducible H-index scores (Jacsó, 2008) and has been used in other studies of faculty research productivity (e.g., Eloy et al., 2012; Svider et al., 2013).

NIH Funding.

We constructed a measure of NIH funding by using the NIH research portfolio online reporting tool, RePORT. We collected the amount of NIH grant money (in U.S. dollars) each faculty member had received as a project PI from fiscal year 2008–2013.

Years in Higher Education.

We used the survey question “How many years of experience do you have in higher education as a faculty member?” to determine the faculty member’s length of career.

Statistical Analysis

The missing values of the analysis variables were multiply imputed before the multivariate analyses were started. The percent missing for these variables ranged from 0 to 15% (Table 1). Multiple imputation involves creating multiple sets of values for missing observations using a regression-based approach. It is currently considered a best practice for addressing missing data in statistical analysis (Enders, 2010). Using IBM SPSS, version 22, statistical software, 20 imputed data sets were specified to increase power and 200 between-imputation iterations were used to ensure that the resulting imputations were independent of one another (Enders, 2010). When imputed data are used, it is recommended that originally ordinal measures (i.e., the dependent variable: the two benefits items; the two barrier items: number of undergraduates, and the number of graduate students) be analyzed as continuous predictors, which we do in our model. This approach is considered a best practice when imputing missing data and estimating model parameters, because rounding off imputed values based on discrete categorical specifications has been shown to produce more biased parameter estimates in analysis models (Horton et al., 2003; Allison, 2005; Enders, 2010; Rodwell et al., 2014). Before analyzing the multiply imputed data sets, we excluded faculty who did not mentor any undergraduate students over the past 5 years (n = 46) and cases with a relatively high proportion of missing data (i.e., more than 40% missing for the variables included in the analysis) (n = 22), which resulted in the loss of 68 cases.

Then, multiply imputed data (n = 468) were analyzed using a generalized estimating equation (GEE) with a robust covariance estimator, which models the independent variables as predictors of collaborative faculty–student publications. SPSS generates pooled GEE results for the 20 data sets, which is what we report. GEEs provide a general method for analyzing clustered variables and relax several assumptions of traditional regression models (Liang and Zeger, 1986; Zeger and Liang, 1986; Diggle et al., 1995). We defined clusters of faculty mentors by their disciplinary area—including clinical/medical science (n = 214), life science (n = 156), social science (n = 59), and engineering (n = 39)—and then by their institutions (n = 12). This cluster-definition method yielded 28 total clusters (because all institutions did not have faculty in all four disciplinary areas). Based on those clusters, our GEE model statistically adjusts for variation by academic field and institution, which is important, because those contextual attributes strongly influence faculty teaching loads, publication norms and expectations, and rates of undergraduate research participation. Because our focus is on predictors of faculty–student publications at the faculty level, not on higher-level (e.g., institutional or disciplinary) effects, GEEs are most appropriate, because the intracluster correlation estimates are adjusted for as a nuisance and not modeled (as in hierarchical linear models). GEEs are also appropriate for this study because they imply no strict distribution assumptions for independent variables, and one of the independent variables, “NIH funding” is skewed. Some faculty members administered major NIH grants as PIs over the past 5 years, while many others received zero research dollars from NIH.

GEEs also require specification of an intracluster dependency correlation matrix (Liang and Zeger, 1986; Zeger and Liang, 1986). In this study, we specified the exchangeable correlation matrix, which assumes constant intracluster dependency, such that all off-diagonal elements of the correlation matrix are equal. This specification has been used in related studies (Daniels et al., 2016; Collins et al., 2017). Because the dependent variable was treated as continuous, we tested normal, gamma, and inverse Gaussian distributions with logarithmic and “identity (linear) link” functions to select the best-fitting models (Garson, 2012). We report results from the GEE using a normal distribution with an identify link function, because it yielded the lowest quasi-­likelihood under the independence criterion value, meaning it was the best fitting. Based on variance inflation factor, tolerance, and condition index criteria, inferences from the GEE are not affected by multicollinearity. Continuous independent variables were standardized before being included in the model.

RESULTS

Table 2 presents results for the GEE model. Considering variables related to faculty–student collaboration, the average duration of the mentoring relationship exhibits a positive and statistically significant association with faculty–student publications. Compared with faculty who on average mentored undergraduates for more than 1 summer but less or equal to 1 year (i.e., moderate duration), faculty who usually mentored undergraduates for more than 1 year (i.e., long duration) were significantly (p < 0.001) more productive with their students; in contrast, faculty who usually mentored for 1 summer or less (i.e., short duration) were significantly (p = 0.004) less productive in terms of collaborative faculty–student publications. Faculty who had mentored more undergraduate students published significantly (p < 0.001) more with their undergraduate students. We did not find an association between mentoring undergraduates through formal programs and collaborative faculty–student publications. The number of graduate students was also not a significant predictor.

With regard to faculty commitment to undergraduate students, faculty who agreed more strongly that they enjoy teaching students about research published significantly (p = 0.006) more coauthored papers with mentees. Faculty who had mentored an African-American undergraduate student on a research project published significantly (p = 0.009) more with their undergraduate students. Two variables in this group approached statistical significance and were positively related to publications. They were “receiving help from undergraduates on research” (p = 0.053) and “have mentored an undergraduate student with a disability” (p = 0.056). Other mentoring barrier and student characteristics variables did not approach significance as predictors of faculty–student publications.

Results for faculty characteristics demonstrated that faculty with higher H-index scores, more years of experience working in higher education, and who were recent recipients of more grant money from NIH as project PIs published significantly (p < 0.001; p = 0.025; p = 0.001) more with their undergraduate students. Sex and race/ethnicity were not statistically significant predictors in the model.

DISCUSSION AND CONCLUSIONS

Our results reveal that faculty members who were more productive in publishing with students shared certain characteristics. These faculty mentors, not surprisingly, worked with more undergraduate students and typically worked with them for more than 1 year on average. Given the relative rarity of publications that include undergraduate student authors, it stands to reason that working with more students provides a faculty mentor with greater potential for having authored more papers with students and also leads faculty to develop effective strategies for integrating undergrads in productive ways within their ongoing research. Working with students for more than 1 year allows time for the students to become more proficient researchers and complete a research project (Hunter et al., 2007; Thiry et al., 2012; Adedokun et al., 2014).

With regard to faculty commitment, we found that biomedical faculty who enjoyed teaching students about research were more productive in terms of collaborative faculty–student publications, which was consistent with Sax and colleagues’ (2002) unexpected finding that general commitment to students was a positive influence on faculty research productivity. Interestingly, faculty members who had mentored Black or disabled students were more productive in publishing with their undergraduate students. We see four possible explanations for this surprising finding. First, having mentored Black students or students with disabilities might indicate the mentor’s commitment to helping undergraduate students more generally, because faculty members might need to spend more time or provide additional support when mentoring more socially marginalized students. Previous studies show that faculty commitment to undergraduates is positively related to research productivity (Sax et al., 2002). Second, faculty members who have mentored Black or disabled students may be more adept at communicating and collaborating with undergraduates than other faculty members, because mentoring minority students has been shown to improve faculty members’ strategies for faculty–student collaboration (Muller, 2006), which is highly related to faculty–­student publication level Third, team diversity has been shown to improve research performance, because a broader range of knowledge, skills, and contacts in the group contribute to more successful research (Barjak and Robinson, 2008). Involving minority and disabled undergraduate students could improve team-based research productivity, because those students can bring a broader array of perspectives and skills. Fourth, for some faculty members, having experiences of mentoring Black students or students with disabilities might suggest that they manage large labs/research teams. With the benefits of a large lab/research team (e.g., funding, equipment, graduate students/postdocs), such faculty members may be more likely to publish papers with their students. However, this explanation is less likely than the other three, because our model controlled for the number of undergraduate students, the number of graduate students, and the amount of NIH funding, which would correspond with funding for postdocs and lab technicians, in addition to graduate and undergraduate students. The other undergraduate mentee demographic variables did not approach significance. We believe this may related to the characteristics of our sample. Across the 13 research-intensive institutions (11 of which are located in the U.S. Southwest), female and Hispanic students are not socially marginalized to the same degree as Black students or students with disabilities. In fact, among the 536 faculty members we surveyed, more than 70% had mentored female undergraduates and more than 65% had mentored Hispanic students. Therefore, the experience of mentoring those students (female/Hispanic) is likely less related to the mentor’s commitment to helping undergraduates or development of improved collaboration strategies. And, in the U.S. Southwest, female and Hispanic students might not contribute to the team diversity in the same way as Black and disabled students. However, it is not clear why mentoring Native American students or LGBT students was not associated with increased faculty–undergraduate publications. More research is needed to advance understanding of the role of mentee diversity in faculty–student research productivity.

Findings indicate that biomedical faculty members who believed more strongly that they received help from undergraduates on their research were more productive in terms of coauthoring papers with undergraduates; this likely reflects the help that they actually receive from students (Chopin, 2002; Eagan et al., 2011; Webber et al., 2013). Our findings suggest that receiving help from undergraduates not only motivates biomedical faculty to mentor more undergraduate students (Morales et al., 2016, 2017) but also contributes to their research productivity with students. Although faculty members might have concerns with regard to working with undergraduate students (e.g., excessive time demands, insufficient skills), our study suggests that those concerns did not directly result in less collaborative faculty–student publication productivity.

Faculty members who had received more research money from the NIH as PIs in the past 5 years were more likely to publish papers with their undergraduate students, which aligns with other studies linking funding status to publishing prowess (Gaughan and Bozeman, 2002; Lee and Bozeman, 2005; Godin, 2003). Specifically, our results suggest that faculty members who had received more dollars from the NIH were more like to publish papers with undergraduate students. Funding likely provides access among undergraduate students to higher-­quality research resources. As a result, the overall research productivity of the lab/research team may increase. When predicting faculty–student collaborative publications, the H-index was also significant in the model, which is not surprising, because the H-index is a measure of faculty research productivity. Controlling for other factors, this finding indicates that a faculty mentor who was a more productive researcher was more likely to publish papers with undergraduate students. Years of experience in higher education was also significant in the model, which indicates that when predicting faculty–student collaborative publications, the amount of time faculty members have worked in their roles is important. This suggests that as faculty members spend more time in their roles, they accrue skills that help them to be more productive mentors. The faculty member’s sex was not significant. While previous studies show that male faculty members tend to be more productive than female faculty members, even after controlling for personal, professional, and environmental factors, such as age, rank, department, tenure status, funding status, or institutional factors (e.g., Sax et al., 2002; Fox, 2005; Symonds et al., 2006; Hunter and Leahey, 2008; Padilla-Gonzalez et al., 2011), we did not find males to be more productive in terms of publishing with students. It may be that female faculty are generally more committed to publishing collaboratively with undergraduate students, such that they publish as much with undergraduates as their male colleagues, even if their overall productivity is lower.

This study has limitations that should be addressed via future research. Several variables for faculty commitment to undergraduates are derived from yes/no survey items that correspond to the entire duration of the faculty member’s career. Because the dependent variable is the number of collaborative publications over the past 5 years, variables focusing on student engagement (e.g., with Black student mentees) over the past 5 years would provide better alignment. Because bench mentors may directly influence faculty–undergraduate student collaborative publications, variables that provide improved measures of the involvement of bench mentors should be used in future studies. In this study, we used the number of graduate students the faculty member had worked with during the past 5 years as a proxy for this, but better measures are needed. For example, future analysts should collection information on how many graduate students and postdoctoral personnel work with the average undergraduate in each faculty member’s lab/research team and how much time they tend to spend together. Finally, more research is needed to examine relationships between student demographics and faculty–student publications. To summarize, future researchers who are interested in examining faculty–undergraduate publications should seek to completely align independent variables and productivity outcomes in temporal terms and consider interactions between undergraduate students and other research team members besides the faculty mentor and graduate students.

In conclusion, faculty–student peer-reviewed publications are tangible products of undergraduate research experiences, and publishing collaborative research is valuable for both biomedical faculty mentors and undergraduate mentees. By publishing original research with their faculty mentors, undergraduates may develop their scientific and technical human capital (Bozeman and Corley, 2004), and student success contributes to future increases in this capital across biomedical fields (Lee and Bozeman, 2005). Practically, results suggest that college administrators and research program directors should strategically aim to encourage more faculty members to work with undergraduate students, identify mechanisms to sustain faculty–student collaborations beyond 1 year, and promote faculty awareness that undergraduate students can contribute productively to their research. Additionally, undergraduate research programs that aim to enhance faculty–student publishing should seek to match students with faculty mentors who demonstrate enjoyment with student interactions, have more funding and experience, and are generally productive scholars, because those characteristics are associated with a greater likelihood of publishing.

FOOTNOTES

ACKNOWLEDGMENTS

REFERENCES