ASCB logo LSE Logo

General Essays and ArticlesFree Access

Evaluation of a Culturally Responsive Mentorship Education Program for the Advisers of Howard Hughes Medical Institute Gilliam Program Graduate Students

    Published Online:https://doi.org/10.1187/cbe.21-11-0321

    Abstract

    Effective mentorship is critical to the success of trainees in research career pathways, significantly impacting their research productivity, academic and research self-efficacy, and career satisfaction. Research faculty may be unaware of or unprepared to address mentor–mentee dynamics in mentoring relationships, especially those that go beyond traditional scientific skill development. Addressing mentorship dynamics can be even more challenging for mentors from well-represented backgrounds working with mentees from historically excluded racial/ethnic groups. The Howard Hughes Medical Institute supports programmatic interventions, like the Mentorship Skills Development (MSD) course, an innovative program that aims to advance the mentorship competencies and cultural diversity awareness of mentors. Between 2015 and 2020, more than 200 faculty mentors participated in the MSD. Quantitative and qualitative data reveal significant gains in mentorship skills and cultural awareness, with mentors reporting increases in their confidence to have conversations around race and culture with their mentees. More than 85% reported actual or intended changes to their cultural responsiveness or mentorship behaviors. Importantly, behavioral changes were also observed by their mentees. These data indicate that culturally responsive mentorship education can increase knowledge and efficacy in effective mentorship practices and improve mentorship experiences of both mentors and mentees.

    INTRODUCTION

    Effective mentorship plays a critical role in the long-term persistence and academic success of trainees (hereafter referred to as “mentees”) in research career pathways, having a significant impact on mentees’ research productivity, academic and research self-efficacy, and career satisfaction (National Academies of Sciences, Engineering, and Medicine [NASEM], 2019). However, many faculty advisers (hereafter referred to as “mentors”) may be unaware of or unprepared to address mentor–mentee dynamics in research mentoring relationships, especially those that go beyond traditional scientific skill development. Addressing mentorship dynamics can be even more challenging for mentors from well-represented backgrounds, many of whom will have experiences distinct from those of their mentees due to the ways in which race, ethnicity and other aspects of identity can influence interactions within the research and training environment. These challenges have become even more pronounced with recent, national social unrest that has increased awareness about policies and practices that serve to exclude students identified as Black, Native American, Native Pacific Islander, and Hispanic/Latine and/or from other marginalized sociocultural groups (Blake-Beard et al., 2011; Asai, 2020; McGee et al., 2021). For the purposes of this article, we use some terms as they are most recently defined by the U.S. Census Bureau (www.census.gov/topics/population/race/about.html).

    The authors recognize that the use of terms related to identity, particularly those related to race and ethnicity, are constantly evolving (Harmon, 2021). We have elected to use the term “historically excluded” to refer to racial and ethnic groups that are underrepresented in science, technology, engineering, and mathematics (STEM) fields. The underrepresentation of persons from these groups is the result of a culture of systematic exclusion manifested by practices and policies that were not designed by or for them. Though we use the term “historically excluded” to capture the accumulation of past exclusionary cultures, practices, and policies, we acknowledge their ongoing impact that creates underrepresentation of various racial and ethnic groups in STEM fields. We understand that the terms used may not be preferred by some groups; thus we will continue to interrogate our nomenclature and adapt terminology appropriately as we continue our work.

    The NASEM concluded that challenges due to cultural diversity factors operating in interpersonal interactions, including race/ethnicity, can compromise the persistence of mentees from groups historically excluded in STEM fields (National Research Council, 2011). These challenges may include research training environments that implement exclusionary practices. Such practices differentially impact students from certain racial/ethnic groups wherein they encounter implicit cultural biases, stereotypes, and even encounter direct experiences of racism and discrimination from peers and faculty (Colón Ramos and Quiñones-Hinojosa, 2016; Acosta and Ackerman-Barger, 2017; Puritty et al., 2017).

    The NASEM consensus study titled The Science of Effective Mentorship in STEMM (National Academies of Sciences Engineering and Medicine, 2019) emphasized the importance of culturally responsive mentorship wherein mentors see themselves as cultural beings and value their mentees’ identities, both cultural and scientific. Byars-Winston et al. (2018) summarized research stating that cultural awareness is the foundational element needed to enact culturally responsive practice. Simply put, one cannot respond to cultural diversity matters of which one is not first aware. Evidence suggests that mentorship that explicitly addresses cultural diversity as well as psychosocial needs of trainees is positively correlated with increases in trainees’ science identity and commitment to/satisfaction with a research career, particularly for those from historically excluded racial/ethnic groups (Haeger and Fresquez, 2016). Increasing research mentors’ capacity to effectively navigate cultural diversity in their mentoring relationships is thus critical to advance scientific workforce diversity. This work is one key component in “addressing equity and inclusion in the research training ecosystem” (https://diversity.hhmi.org; www.nih.gov/ending-structural-racism/unite) with movements toward inclusive excellence (www.aacu.org/making-excellence-inclusive).

    Changing the research training enterprise requires attention to the relationships and cultural contexts in which that training occurs. The Howard Hughes Medical Institute (HHMI) has stated that it is committed to advancing equity in research environments that enables students and scientists from all backgrounds to thrive (https://diversity.hhmi.org). In support of this commitment, HHMI is investing in programmatic efforts to promote consistent, effective mentorship. One such program is the Mentorship Skills Development (MSD) course embedded in the Gilliam Program (www.hhmi.org/science
-education/programs/gilliam-fellowships-advanced-study). Gilliam Fellows (mentees) are accomplished predoctoral trainees from groups that are underrepresented in the sciences due to historical and continued exclusion, including students from ethnic and racial groups and students with disabilities. The Fellows engage in multiple years of research training and professional development activities guided by mentors who predominantly come from ethnic and racial groups that are well represented in the sciences. Authors of this paper created the innovative, intensive MSD course to develop its mentors in evidence-based culturally responsive mentorship competencies. Acceptance of the Gilliam award requires that mentors participate in the MSD course to maximize their cultural responsiveness in support of their and their mentees’ success.

    The MSD course for HHMI Gilliam mentors includes: 1) foundational mentor training that teaches specific competencies and characteristics of effective instrumental and psychosocial mentorship roles; 2) extended training concentrated on building the cultural responsiveness of mentors to address cultural diversity in their research mentoring relationships; and 3) targeted work on mentorship relationship dynamics. This program is composed of several extensively studied mentor training interventions that have been combined into one full-year educational experience including curricular elements from: Entering Mentoring (Handelsman et al., 2005; Pfund et al., 2006, 2014; Pfund, 2015), Optimizing the Practice of Mentoring (Weber-Main et al., 2019), Promoting Mentee Research Self-Efficacy (Butz et al., 2018), Culturally Aware Mentor (CAM) Workshop (Byars-Winston et al., 2018; Womack et al., 2020), and Mentor Training Implementation Planning (Pfund et al., 2015; Spencer et al., 2018).

    The purpose of this paper is to introduce the MSD model implemented as part of the HHMI Gilliam Program and to present outcome data that demonstrate its effectiveness on mentors’ attitudes and behaviors relative to mentorship competencies and culturally responsive practices. First, we describe the components and design of the model. Next, we summarize findings from analyses of both quantitative and qualitative evaluation data documenting the impact of the program on mentors’ attitudes and behaviors as well as the impact on mentees. We conclude the paper with a discussion of the findings for both future practice and research. These findings have great potential to inform other interventions aimed at supporting the capacity of faculty research mentors to optimize their mentorship practices with all students, particularly those from populations historically excluded due to specific identities including ethnicity or race.

    METHODS

    Participants

    The Gilliam Fellow (mentee) participants for this sample include 119 individual Gilliam Fellows from 50 institutions who participated between 2016 and 2018. Descriptive demographics for these mentees were collected on their applications to the Gilliam Fellows Program and are shown in Table 1. Ninety-seven percent of these mentees are from groups historically excluded by ethnicity, race and disability in the biomedical sciences.

    TABLE 1. Demographics of Gilliam Fellows (2016, 2017, and 2018 cohorts; participants were allowed to choose more than one category)

    201620172018Total
    Asian American1102
    Black10121335
    Native American2035
    Native Pacific Islander0011
    White0134
    Hispanic/Latine21192464
    Disabled1236
    Female18182965
    Male16211754

    The Gilliam mentor participants for this sample include 116 faculty who served as mentors to the 2016, 2017, and 2018 Gilliam Fellows cohorts and completed the indicated elements of the MSD course. Demographic data collected via survey from 109 of these mentors are shown in Table 2. The majority of mentors in these three cohorts identified as White (81%) and more than half, as male (57%).

    TABLE 2. Demographics of Gilliam mentors (2016, 2017, and 2018 cohorts; participants were allowed to choose more than one category)

    201620172018Total
    Asian American3418
    Black2136
    Native American0022
    Native Pacific Islander0001
    White26313289
    Hispanic/Latine0134
    Multiple1113
    Other1315
    Prefer not to report1102
    Female10131145
    Male20241862
    Prefer not to report1102

    Data from certain years (2016–2018) were used for the described analyses instead of data from the entire program (2015–2020), because we prioritized investigation of the impact of the training over multiple points in time (up to 3 years post training).

    The Gilliam mentor and Fellow (mentee) participants included in the sample came from 50 unique institutions. Forty-three of these institutions are Carnegie R1 (doctoral institutions with very high research activity). Twenty-three are publicly supported, and twenty-two are private—these include seven Ivy League institutions plus six minority-serving institutions. Incorporating MSD into the Gilliam program was an intentional strategy. It is one way in which HHMI can leverage its relationship with institutions with very high research activity to drive culture change.

    Intervention

    We designed the MSD course for the HHMI Gilliam Program to be implemented through a cohort model of faculty participants, parallel to the admission of Gilliam Fellow cohorts annually. The cohort model allowed us as program leaders to build a learning community for mentors in which they engage in mentorship education as a group. The cohort approach has been shown to increase motivation, help participants build professional networks, and serve as a mutual support resource (Opacich, 2019). Gilliam mentors participated in the MSD course with their assigned cohort for a full year, which included engagement in four webinars (synchronous online), two self-paced online learning modules (asynchronous online), and two separate face-to-face workshops (Figure 1).

    FIGURE 1.

    FIGURE 1. Overview of multifaceted evidence-based MSD course for HHMI Gilliam mentors.

    Total engagement for each faculty mentor was roughly 25–30 hours across the year, not including additional reading and reflection participants may do or the time they spend mentoring. Following the evidence for validated, published mentorship interventions summarized in the NASEM (2019) mentorship report, we selected topics and activities for the MSD course that include components from the following curricula: Entering Mentoring, Optimizing the Practice of Mentoring, Promoting Mentee Research Self-Efficacy, Culturally Aware Mentor Training, and Mentor Training Implementation Planning. We also developed specific curricular items to address current events and societal challenges that are specifically relevant to the lived experiences of the mentors and Fellows. The various components of this multifaceted program and evidence of their effectiveness are listed in Table 3.

    TABLE 3. Overview of the Mentorship Education Program for Gilliam mentors

    DateLengthFormatTopicsReferences
    October60 minutesSynchronous onlineIntroduction to the Science of MentorshipNASEM, 2019
    November90–120 minutesAsynchronous onlineOptimizing the Practice of MentorshipWeber-Main et al., 2019
    December60 minutesSynchronous onlineLeveling the Playing Field by Articulating ExpectationsHandelsman et al., 2005; Pfund et al., 2006, 2014; Pfund, 2015
    January60 minutesSynchronous onlineImproving Communication with Your MenteeHandelsman et al., 2005; Pfund et al., 2006, 2014; Pfund, 2015
    February60 minutesSynchronous onlinePeer Mentor Training (topics vary in response to challenges expressed by participants)
    March90–120 minutesAsynchronous onlineIntroduction to Culturally Aware Mentoring 
(iCAM)Byars-Winston et al., 2018
    April2 daysFace-to-faceExploring Your Institutional Data on Diversity and InclusionExamining Your Own Cultural Heritage: Culture BoxThe Experience of Being an “Other”Promoting Mentee Research Self-EfficacyAdvancing Culturally Responsive CommunicationPfund et al., 2015; Butz et al., 2018; Byars-Winston et al., 2018; Poodry and Asai, 2018
    September1.5 daysFace-to-faceNavigating Imposter PhenomenonNavigating the Changing Development Stages of Your MenteeReducing Stereotype Threat and MicroaggressionsFostering Mentee’s Career ResilienceOptimizing Mentoring Relationships in Your Own Program/DepartmentSpencer et al., 2018

    Assessments

    We collected data from mentors via surveys administered immediately following the April and September face-to-face sessions. We administered an additional survey annually to each cohort roughly 17 months after the start of their time in the Gilliam Program and subsequently each year for 3 years. Data were initially collected via paper survey and then via online survey platforms, either Qualtrics or an assessment platform from the Center for the Improvement of Mentored Experiences in Research (www.cimerproject.org). Survey questions included assessment of program components and outcomes from mentors. Mentee surveys, administered annually, included questions about their experiences in their mentoring relationships. Specific factors assessed in each survey are listed in Table 4.

    TABLE 4. Overview of assessment surveys and response rates

    SurveyaFactors assessed and included in this studyResponse rate per cohort
    Midpoint mentor survey (April)DemographicsSatisfaction with workshop activitiesCAM skills (Byars-Winston et al., 2018)Intent to change/actual change in mentorship behaviors (Pfund et al., 2014; Byars-Winston et al., 2018)2016: 84%2017: 92%2018: 98%
    Endpoint mentor survey (September)Satisfaction with workshop activities2016: 77%2017: 71%2018: 70%
    Annual mentor and mentee survey T1 (December/January)Satisfaction with program components, including all webinars and resourcesQuality of mentoring relationshipb (Byars-Winston et al., 2015)Cultural diversity awarenessb (Byars-Winston and Butz, 2020)2016: 81%2017: 85%2018: 38%

    aFor the April and September sessions, a small number of mentors for Fellows in the Burroughs Wellcome Fund Postdoctoral and Graduate Diversity Enrichment Programs attended. Their immediate postsession survey data on satisfaction and learning are included in these results; however, these mentors and mentees were not tracked longitudinally.

    bAssessment by both mentor and mentee.

    Data Analyses

    We compared post-training scores to retrospective pre-training scores for individual subitems in each scale using the Wilcoxon rank-sum test for comparison across nonparametric paired samples. Mentor behaviors and mentorship relationship quality averages were compared with corresponding mentee scores using the Mann-Whitney U-test for comparison across nonparametric independent samples given the lack of paired data for every mentor–mentee pair. Confidence scale subitems for mentors were compared across multiple cohorts using a Kruskal-Wallis test for multiple, nonparametric sample means comparison. Qualitative data from open-ended survey questions were analyzed using an iterative coding methodology (Saldaña, 2009). For the first round of coding, open coding was employed by F.S., followed by code consolidation. Using the codebook created by F.S., both C.P. and F.S. independently employed focus coding to assign categories to each response. Interrater reliability was calculated after this step using the simple proportion agreement method given the exploratory nature of this study (Campbell et al., 2013).

    Composition of the Team

    The intervention implementation team comprises five facilitators (four women, one man; two individuals identified as White, two individuals identified as Black, and one individual identified as Hispanic/Latine) from varying disciplines (biology, genetics, psychology) and career paths (e.g., faculty, research staff, administrators) and from two U.S. research-intensive universities, a biomedical research institute, and an institute supporting higher education and biomedical research. The facilitation team originally began as two facilitators and expanded as the scope of the intervention and the size of the participant pool grew larger. All facilitators have many years of experience in designing, implementing, studying, and administratively coordinating professional development and training interventions for individuals in academia, government, and the private sector, including research mentors and mentees in the sciences. It is important to note that facilitators were experienced in culturally responsive facilitation (S, House, A, Byars-Winston, D, Azurdia… and C, Sorkness, unpublished data). Culturally responsive facilitation skills include the ability to be responsive in real time to dynamics reflecting race and other cultural factors unfolding in the training. We believe that this level of facilitation experience was critical to the overall effectiveness and successful implementation of the intervention. The qualitative data were analyzed by one white female and one nonbinary Hispanic/Latinx researcher. The remaining members of the team include an Asian American male and a White female.

    RESULTS

    Participants

    Between 2015 and 2020, 232 individuals from 62 different institutions served as mentors to Gilliam Fellows; 84% of these mentors fully completed the mentorship education component of the Gilliam Program. Here, we report on 116 faculty who served as mentors to the 2016, 2017, and 2018 Gilliam Fellows cohorts and who completed the indicated elements of the MSD course. The majority of mentors in these three cohorts identified as White (81%) and male (57%; Table 2).

    Satisfaction

    Mentors in each cohort reported high levels of satisfaction with almost all elements of the MSD course (Figure 2). The face-to-face sessions were the most highly rated. The shared resources that include webinar slides, references, tools, and links to online resources were rated the lowest. All cohorts reported similar levels of satisfaction across the course components.

    FIGURE 2.

    FIGURE 2. Satisfaction with MSD course components. Rating of satisfaction from 1 (very unsatisfied) to 6 (very satisfied) with the indicated MSD course components. Note that the assessment of the iCAM online module completed in March was included as part of the April session rating.

    In their open-ended responses about the strengths of program components, mentors noted:

    “The workshop gave me specific tools, and useful advice on how to build a more inclusive environment in my lab and my larger department.”

    “There were many specific scenarios and activities that I plan to use with my trainees to enhance my ability to be an effective, culturally aware mentor…. I feel that this is the beginning of a new journey for me and my lab.”

    “I value the personal insight provided by my colleagues.”

    Mentors also suggested areas for improvement such as:

    “I wish we had a little longer to discuss some topics.”

    “A few more examples of specific problems faced by URM [sic]1 in the lab setting.”

    “More advice on how to contribute to this effort beyond the lab ie to the program or school… more resources for explicit implementation… perhaps experiences of past mentor attendees that have been implemented/work.”

    Learning

    At the conclusion of the face-to-face training session, upon completion of nearly 20 hours of engagement in the MSD course over 8 months, mentors were asked to retrospectively rate their skill levels in several areas before and after the training. Specifically, mentors were asked to self-report gains in 15 culturally aware mentoring skills (Byars-Winston et al., 2018). Gains for each skill were significant (p < 0.001) for each cohort. Figure 3 shows the mentors’ self-reported gains for four selected skill areas that have previously been shown to significantly increase with culturally aware mentoring (CAM)-based mentorship education (Byars-Winston et al., 2018).

    FIGURE 3.

    FIGURE 3. Self-report skill gains following CAM-focused mentorship education sessions with Gilliam mentors. Box plots of retrospective skill ratings across three mentor cohorts (A) 2016, (B) 2017, and (C) 2018, in response to the indicated subscale question “Please rate how skilled you were BEFORE the training and how skilled you feel you are NOW in each of the following items.” Blue box plots represent responses to “BEFORE” and orange responses to “NOW.” Likert-scale responses ranged from 1 (not at all skilled) to 7 (extremely skilled). The four items shown are: 1) respectfully broaching the topic of race/ethnicity in my mentoring relationships; 2) encouraging mentees to think about how research relates to their own lived experience; 3) intentionally creating opportunities for my mentees to bring up issues of race/ethnicity when they arise; 4) going outside of my comfort zone to help mentees feel included in the lab. All population means of pre- and posttest scores for each subscale were statistically significant using Wilcoxon sum rank test for nonparametric paired samples.

    FIGURE 4.

    FIGURE 4. Self-reported confidence in CAM-specific behaviors following engagement in the yearlong MSD course. Mentors were asked to rate their confidence in their ability to do the indicated action on a five-point Likert type scale from 1 (not at all confident) to 5 (completely confident). Items assessed include: 1) discuss with mentees how it feels to be underrepresented in science; 2) take advantage of opportunities to address race/ethnicity in the research mentoring relationship; 3) recognize aspects of the research experience (e.g., lab or fieldwork) that may make racially/ethnically minoritized students feel vulnerable to confirming stereotypes; 4) provide opportunities for mentees to talk about their racial/ethnic identities as they relate to their research experiences should the occasion arise; 5) notice interactions in the mentoring relationship that could be insulting or dismissive to mentees. There were no statistically significant differences in population means of subitem scores across cohorts using a Kruskal-Wallis test for multiple nonparametric samples.

    Confidence

    Upon completion of the yearlong MSD course, mentors were asked to rate their confidence in their ability to effectively mentor trainees from different racial/ethnic backgrounds. The average self-reported confidence for this single item across cohorts was 5.82 ± 0.82 on a seven-point scale from “no confidence” to “extremely confident”. Mentors in each cohort were also asked to rate their confidence for five CAM-specific behaviors on the annual survey (Figure 4). Mentors reported high levels of confidence in each area, with the highest confidence in their ability to provide opportunities for mentees to talk about their racial/ethnic identities as they relate to their research experiences should the occasion arise.

    BEHAVIOR CHANGES

    When asked about the frequency with which specific CAM behaviors occurred in their research mentoring relationships, mentors self-reported engaging in these behaviors some of the time or frequently (Figure 5). Mentors reported “creating opportunities for my mentees to bring up issues of race/ethnicity as they arose” and “raising the topic of race/ethnicity in their research mentoring relationships when it was relevant” with some frequency. They reported more frequently approaching the topic of race/ethnicity with their mentee(s) in a respectful manner (also known as “broaching”).

    FIGURE 5.

    FIGURE 5. Mentor and mentee reports of engagement in culturally aware mentoring behaviors (cultural responsiveness). Gilliam mentors and mentees were asked to report how frequently each of the behaviors related to cultural responsiveness occurred in their mentoring relationships. Mentors were asked to report how frequently they engage in the behavior, and mentees were asked how frequently the behavior occurred with their primary research mentors. Frequencies were rated on a five-point Likert-type scale: 1 (never), 2 (rarely), 3 (sometimes), 4 (frequently), 5 (all of the time). Behaviors assessed were 1) I/My mentor approached the topic of race/ethnicity with my mentee/me in a respectful manner; 2) I/My mentor raised the topic of race/ethnicity in our research mentoring relationships when it was relevant; 3) I/My mentor created opportunities for my mentee/me to bring up issues of race/ethnicity as they arose. Ns for each cohort are listed as mentor/mentee. Population means for each subscale were compared between mentors and mentees using the Mann-Whitney U-test for comparison across nonparametric, independent samples. Statistically significant differences (p < 0.05) are marked with an asterisk (*).

    To determine whether these self-assessments aligned with what was experienced by their mentees, we asked Gilliam Fellows to indicate how frequently each of the following occurred in their relationships with their primary research mentors (Figure 5). Across most of the reported actions, mentors and mentee assessments of frequency did not statistically differ at the population level, but there was a slight trend for mentees reporting lower frequency of behaviors than what mentors self-reported. The exception was the rating of “raising the topic of race/ethnicity in our research mentoring relationships when it was relevant” for the 2017 and 2018 cohorts.

    To more deeply explore changes in mentor behaviors following engagement in the MSD course, mentors were asked to report whether they planned to or actually implemented changes to their mentorship following the yearlong training. These data were collected on the annual surveys starting in 2018. Of the 2017 cohort, 87.1% of mentors (27/31) reported having made changes or their intent to make changes to their mentorship after the full year of engagement. Of responding mentors in the 2018 cohort, 94.1% (16/17) reported having made changes or their intent to make changes to their mentorship. Mentors were asked to describe these changes, and their open-ended responses were coded using thematic analysis. Table 5 shows the frequency of coded responses with examples combined from both the 2017 and 2018 cohorts of the changes described across the two cohorts. Open-ended responses from subsequent longitudinal surveys were also analyzed, and similar patterns were seen, with a slight reduction in the number of instances across most categories.

    TABLE 5. Mentors’ self-reported behavioral changes

    Mentorship behaviorNo. of mentors including behavior in their responsesExample response
    Cultural/background awareness13“I try to consider the mentees [sic] background when giving advice or discussing research, and how my attitudes might be perceived by others.”
    Engaging in effective communication9“Developed new ways to communicate and strategies to deal with conflict”
    Aligning expectations8“I am much more explicit with expectations.”
    Providing psychosocial support7“I have realized that my mentee needs a great deal of emotional support and I have tried to make sure that I ask about well-being and express support for them as a person very frequently.”
    Proactive/intentional mentorship7“More structure; formal documentation of goals, projects, etc.; more focused discussion of areas for development”
    Using a Mentorship Compact5“Made and distributed a lab compact—got input from my Gilliam Fellow on the content”
    Creating an Inclusive Environment5“I also plan on including regular activities with my research group that promote an inclusive atmosphere in my lab.”
    Providing career support4“I have also put a stronger emphasis and more time on professional and career development in my mentoring my students, whereas my previous focus was more heavily weighted towards scientific development.”
    Using an IDP4“Professional development plans”
    Increasing self-efficacy3“I think I have been more patient and also focused on developing his developing self-confidence.”
    Providing personalized mentoring2“Trying to be more attentive to individual needs and challenges of each mentee”
    Promoting professional skill development2“We now explicitly discuss career goals, training in professional skills such as presentation and writing, the importance of going to seminars and reading literature in the field, etc.”
    Broaching uncomfortable topics2“I also find it easier to talk with my Gilliam fellow and other URM [sic] graduate students about difficult issues, sometimes using examples of things that were discussed in the Gilliam training sessions.”
    Fostering independence2“Giving him autonomy to make decisions”
    Other10“Increased meeting frequency/mentor availability; becoming a department change agent; being more open to feedback; focusing on the big picture; providing more resources”

    The Gilliam mentees were asked to describe any changes they saw in their Gilliam mentors over the previous year in the context of the mentoring relationship. Table 6 shows the range of behaviors coded from their open-ended responses with an example of each.

    TABLE 6. Mentors’ behavioral changes as reported by mentees

    Mentoring behaviorNo. of mentees including this behavior in their responsesExample response
    Cultural/background awareness8“More aware of issues that impact students of color; obviously incorporates HHMI training in mentoring”
    Effective communication4“My mentor is more sensitive to what is said in lab…. Is also more active while listening.
    Increased availability4“My mentor has made a point to be around more.”
    Inclusive environment3“My mentor has always been very involved with my development and tried their best to be as inclusive and supportive as possible. Since the Gilliam, this has only gotten better as I feel they have a better vocabulary for doing much of the same things they were doing already, fortunately.”
    Improved/more mentorship strategies3“My adviser was already very progressive with addressing diversity/confidence concerns, but they were able to implement new techniques to help me address these concerns.”
    Psychosocial support2“My mentor has been much more aware of and supportive of my personal struggles with identity and imposter syndrome. They have found better ways to be supportive in those instances (i.e., by sharing personal experiences).”
    Proactive/intentional mentorship2“I think my mentor started investing more time and effort in mentoring me. I am very grateful for this because I know I thrive on positive feedback and a supportive environment.”
    Fostering independence2“They treat me more like an equal, they will seek out my comments on writing and presentations, will take my feedback and boundaries very seriously.”
    Change agent (mentor)2“My mentor is trying to now integrate this form of training for all T32 mentors on campus, which I think is great.”
    Career support1“They also made a stronger effort to help other students meet their career goals. I was likely one of the first students to frequently bring up career goals during meetings, and they extended this type of conversation to students who normally did not talk about their own personal goals.”
    Personalized mentoring1“My mentor has made steps to ensure that the lab as a whole is more aware of each individual’s background and strengths that they bring to the lab, and their individual working styles; my mentor has made the lab space a more welcoming and inclusive environment.”
    Increasing self-efficacy1“They praise my work more often or tries to highlight my unique contributions/skill sets.”
    Professional skill development1“My mentor has come in with new skill- and confidence-building techniques to every meeting, that have seemed to have helped improve my skills tremendously.”
    General improvement4“My mentor has been more collaborative, patient, and understanding since the Gilliam Fellowship training.”
    Other2“More receptive to feedback, helps expand my mentoring network”

    One mentee described a transformation in her mentor after the mentor training experience:

    “Before the fellowship, my mentor and I did not talk about some of the topics listed above. I find them to be more mindful of my experience and concerned about life–work balance. Before, I was pushed to work weekends and long hours, whereas now there are conversations about engaging in a healthier work style. We talk about how my experiences as a URM [sic] at my institution affected me and how their leadership on campus/in the department can contribute to improving the experience of future grad students. After the mentor meeting in person, I truly saw an interest and a spark in my mentor for truly engaging in the DandI activities. They even teamed up with another Gilliam mentor to include more family participation in our work and for the very first time on our campus, they are hosting a STEM family weekend. In addition to their increased mindfulness, they are also assuring that there is more equity for graduate student evaluation as she has found through mine and other URM [sic] graduate students that there is a different level of expectation and differences in how we are being evaluated. To assure there is equal evaluation they have volunteered to sit in ALL of the qualifying exams for this coming year. Their goal is to have a clear evaluation form that all the professors can follow and extend equal evaluation from, their plan is to develop this form so everyone can be treated equally. By volunteering their time they are assuring that there is constant and equal evaluation and that URM [sic] students aren’t being failed because they lacked confidence in their presentation. They are truly motivated to make a space for URM students in this department that is inclusive.”

    However, not all mentees reported positive changes in their mentors, with 8% of the comments noting less than optimal or unsustained changes. One mentee reported “There were some changes initially but they didn’t last.” Another noted that the mentor had taken on a lot more responsibility in the institution and therefore had become more forgetful and detached. Finally, one mentee noted that the mentor tried to lead diversity training for the lab, but the experience was uncomfortable.

    Mentoring Relationship Quality

    Both mentors and mentees were asked to assess the quality of their mentoring relationships annually following their first year in the Gilliam Program. Mentors and mentees similarly rated the quality of their working relationships as good. Shown in Figure 6 are mean ratings of quality by cohort at the furthest longitudinal survey time point (1-, 2-, and 3-years post program engagement). Overall, mentors and mentees rated the quality of their relationships similarly, with no statistical significance between mentor and mentee ratings for each year.

    FIGURE 6.

    FIGURE 6. Quality of the mentoring relationships. Ratings of mentoring relationship quality by both mentors and mentee cohorts at the furthest longitudinal survey time point. Mentorship quality was rated on a four-point scale from 1 (poor), to 2 (fair), to 3 (good), to 4 (excellent) with the question: “How would you rate the overall quality of your mentoring in your relationship with your Gilliam fellow?” Ns for each cohort are listed as mentor/mentee. There were no statistically significant differences in quality reported by mentors and mentees for each cohort using the Mann-Whitney U-test for comparison across nonparametric, independent samples.

    DISCUSSION

    If we are to advance scientific workforce diversity, we need to increase the capacity of research mentors to engage with cultural diversity in their mentoring relationships. Trainees from racial and ethnic groups historically excluded in the sciences who leave these fields typically do so not only because their interest in science changes. They leave because of fatigue from the myriad cultural stereotypes and narratives about their competence that compromise their ability to focus on their research training (Gibbs et al., 2014; Colón Ramos and Quiñones-Hinojosa, 2016; Ruiz and Zárate, 2019). However, empirical evidence indicates that when mentors intentionally address cultural diversity in 1) their mentees’ experience of doing science and 2) in the research training environment, such cultural responsiveness positively contributes to their mentees’ academic and career intentions to remain in science pathways (Haeger and Fresquez, 2016). This paper provides 3 years of evaluation data from a national sample of biomedical research faculty participating in the novel HHMI Gilliam MSD course, a yearlong, intensive, multifaceted faculty development program to promote culturally responsive mentorship. To our knowledge, this is the first example of a single mentorship education program combining multiple components of evidence-based mentor training curricula into one comprehensive offering. Our findings can inform other efforts to build research mentors’ competence and confidence to engage in effective, culturally responsive mentorship practices. We highlight here five key findings from this evaluation study.

    First, faculty participants were highly satisfied with all of the training sessions offered, similar to high satisfaction ratings of mentorship education reported in previous studies (Pfund et al., 2014; Byars-Winston et al., 2018; Weber-Main et al., 2019). Second, faculty participants reported skill gains across core mentorship competencies such as aligning expectations and improving communication as well as promoting mentees’ research self-efficacy (unpublished data). These gains mirrored assessment results from other national samples of mentor participants engaged in the Entering Mentoring–based curriculum (Butz et al., 2018; Pfund et al., 2014). Our findings add to the growing evidence that mentorship education increases faculty mentors’ understanding and application of specific mentorship competencies known to contribute to effective mentorship (Pfund et al., 2016). The NASEM (2019) report on effective mentorship called for a competency-based approach to mentorship, because competency-based practice in any domain is known to contribute to more effective performance outcomes (Parson et al., 2018).

    Third, mentors reported many intended or actual behavioral changes in their mentorship practices 17 months after the start of their yearlong engagement in the Gilliam MSD course. Some behaviors reported linked directly to the core Entering Mentoring–derived components of the program, such as use of mentoring compacts and individual development plans, promoting professional development skills for mentees, improving communication, and addressing equity/inclusion in the mentoring relationship. Other reported behaviors were linked to added mentorship topics such as building research self-efficacy and implementing plans to affect departmental climate at their home institutions. Some behaviors linked directly to those highlighted in the CAM curricular content, such as heightened mentor awareness about their cultural backgrounds, intentionally creating an inclusive environment, and providing personalized mentorship and psychosocial support. In fact, the category of behavioral changes related to cultural awareness had the largest number of mentor responses. Combined, these data indicate that mentors intended to or actually engaged in new mentorship practices.

    Fourth, consistent with the focus of the Gilliam MSD course on culturally responsive mentoring, we evaluated gains in cultural diversity awareness outcomes and related confidence and behaviors as reported by both the mentor participants and their mentees (i.e., Gilliam Fellows). Our results align with the original CAM training study that demonstrated similar retrospective pre–post gains in a variety of CAM skills, including “intentionally creating opportunities for my mentees to bring up issues of race/ethnicity when they arise” and “respectfully broaching topics related to race/ethnicity” (Byars-Winston et al., 2018). Mentors also reported an increase in their self-efficacy for enacting culturally aware mentoring, as indicated by both their overall confidence to mentor trainees from different racial/ethnic backgrounds and for multiple behaviors previously shown to be impacted by CAM training (Byars-Winston et al., 2018). Self-efficacy beliefs are significant predictors of action and achievement and are therefore drivers of behavioral change (Bandura, 1997). Building on our assertion in the Introduction that cultural awareness facilitates cultural responsiveness, we expect self-efficacy for culturally aware mentoring to lead to eventual culturally responsive mentoring behaviors. We observed evidence that Gilliam mentors are frequently engaging in many CAM-related behaviors, which are similar to those reported by participants in the original CAM interventions 18–24 months after participation in the CAM training (Womack et al., 2020).

    Finally, we investigated the experience of the Gilliam mentees with their primary mentors, asking mentees to describe the mentorship-related behavioral changes they experienced. Similar themes emerged in the codes for both mentors and mentees, suggesting that the behavioral changes made by mentors were noticed by the mentees. The specific behaviors that emerged from our data codes provide a list of possible concrete actions mentors can take to advance their mentoring relationships in culturally responsive, impactful ways. We also used parallel survey questions about the frequency with which mentees rated their mentors as being engaged in specific CAM-related behaviors and found similar frequencies of each behavior to those self-reported by their mentors.

    Although our results are encouraging, we know that there is much work to be done, as the sciences are vulnerable to structural racism, bias, and discrimination. Since the spring of 2020, which ushered in heightened visibility to racialized violence and individual and systemic racism, scores of publications from trainees and faculty from groups historically excluded by race and ethnicity in the sciences have demanded rooting out racism and other forms of oppression embedded in academic environments that compromise the career development of people from such groups (e.g., Platt, 2020). Some science faculty are responding to this demand with increased willingness to take action to transform STEM fields toward inclusive excellence (Barber et al., 2020). This demand may account for our observation of higher baseline scores, higher confidence ratings, and behavior frequencies of culturally aware mentoring in later cohorts of our data. Though the Gilliam MSD course may be improving over time, these data trends may also be due to a period effect of societal trends in recognizing and attempting to rectify racial and social inequities.

    In summary, our evaluation data indicate that the Gilliam MSD course offers an impactful faculty development opportunity for research mentors to learn by reflecting on their own mentorship practices in research training environments with a diversity and equity lens, doing so in a community of peers. Three key components of reflective practice included in the course design are reflection in action (analysis of what is being done), reflection on action (analysis of a past situation), and reflection for action (new learnings applied to future actions; Schon [1983] as cited in Steinert, 2010). Combined, these three reflective practice components engender the critical analysis, self-awareness, and collective peer community necessary for research mentors to strategize transformative behaviors at the individual and systems levels for cultivating equitable scientific environments.

    Limitations

    We acknowledge that those who participated in the Gilliam MSD course may have been motivated to respond if they had positive updates to share. Those with less favorable experiences may not have provided open-ended responses, and we note the potential non-response bias in our findings. We also acknowledge the limitations of self-reported mentor data. We included mentee data about mentor behaviors to provide additional evidence of mentors’ mentoring practices. While these data are informative and encouraging, we understand the power dynamic between mentors and mentees. Despite assurances of anonymity given to mentees and mentors, this concern could influence the willingness of mentees to respond.

    Future Directions

    Although not reported in this paper, the Gilliam Fellowship Program also includes leadership training for mentees. The Gilliam Leadership Training equips mentees with skills to use their personal values to guide their interactions and professional decisions as they advance through their scientific careers. We plan to examine the impact of this mentee training on the mentees and their relationships with their mentors. Our limited sample size did not allow us to fully analyze dyadic data from mentor and mentee pairs in the present paper, though our continued data collection will allow us to study alignment and impact of that alignment between mentor and mentee pairs on mentoring relationships and mentee outcomes. We will also continue longitudinal investigation of changes in mentor behaviors and investigate the impact of the mentoring relationship on mentees’ progress and career decisions, and we will track mentors’ efforts to change the mentorship culture at their home institutions. Due to sample size restrictions, we were unable to further explore the paired mentor and mentee responses, including how individual characteristics play into training effectiveness within mentoring relationships.

    CONCLUSION

    To advance diversity and equity goals in graduate STEM education, we need faculty who are willing to grapple with and critically examine the cultural contexts of research training environments. This requires a paradigm shift that focuses on fixing inequitable systems and not the students who occupy them. As McGee et al.’s (2021) research with Black doctoral students in STEM indicates, it is impossible for Black students to avoid the racialized stress they experience due to academic and research environments that were not designed for their success. The onus to redress and ameliorate hostile and inequitable research training environments is on members of the faculty, who hold great influence in shaping the climates of graduate education environments (NASEM, 2018). HHMI’s Gilliam MSD course is a promising model of science faculty development for culturally responsive mentorship. In October 2021, HHMI announced its 10 diversity, equity and inclusion goals, including providing mentorship training to all HHMI scientists modeled on the MSD (https://diversity.hhmi.org). This is one strategy to build equitable research training environments so that all trainees may thrive.

    FOOTNOTES

    1 The authors avoid using the term minority such as in “URM” because it has been used pejoratively and is vague (see Flanagin et al., 2021). Read authors’ use of the term “historically excluded” in the Introduction.

    ACKNOWLEDGMENTS

    We acknowledge the foundational contributions of Drs. Clif Poodry, George Langford, Alfred Mays, and Jo Handelsman. We also acknowledge those who supported implementation of the Gilliam Program and the MSD course, including Leah Nell Adams and Megan Katz. This work was supported by a grant to the University of Wisconsin–Madison from HHMI through the Science Education Program. This work was reviewed by our IRB and is covered under protocols no. 2016-1517 and no. 2017-1336.

    REFERENCES

  • Acosta, D., & Ackerman-Barger, K. (2017). Breaking the silence: Time to talk about race and racism. Academic Medicine, 92(3), 285–288. https://doi
.org/10.1097/ACM.0000000000001416 MedlineGoogle Scholar
  • Asai, D. J. (2020). Race matters. Cell, 181(4), 754–757. https://doi.org/
10.1016/j.cell.2020.03.044 MedlineGoogle Scholar
  • Bandura, A. (1997). Self-efficacy: The exercise of control. W.H. Freeman/Times Books/Henry Holt. Google Scholar
  • Barber, P. H., Hayes, T. B., Johnson, T. L., & Márquez-Magaña, L., & 10,234 signatories. (2020). Systemic racism in higher education. Science, 369(6510), 1440–1441. https://doi.org/10.1126/science.abd7140 MedlineGoogle Scholar
  • Blake-Beard, S., Bayne, M. L., Crosby, F. J., & Muller, C. B. (2011). Matching by race and gender in mentoring relationships: Keeping our eyes on the prize. Journal of Social Issues, 67(3), 622–643. https://doi.org/10.1111/
j.1540-4560.2011.01717.x Google Scholar
  • Butz, A., Branchaw, J., Pfund, C., Byars-Winston, A., & Leverett, P. (2018). Promoting STEM trainee research self-efficacy: A mentor training intervention. Understanding Interventions, 9(1), 3730. Google Scholar
  • Byars-Winston, A., & Butz, A. R. (2021). Measuring research mentors’ cultural diversity awareness for race/ethnicity in STEM: Validity evidence for a new scale. CBE—Life Sciences Education, 20(2), ar15. LinkGoogle Scholar
  • Byars-Winston, A., Womack, V. Y., Butz, A. R., McGee, R., Quinn, S. C., Utzerath, E., ... & Thomas, S. (2018). Pilot study of an intervention to increase cultural awareness in research mentoring: Implications for diversifying the scientific workforce. Journal of Clinical and Translational Science, 2(2), 86–94. https://doi.org/10.1017/cts.2018.25 MedlineGoogle Scholar
  • Byars-Winston, A. M., Branchaw, J., Pfund, C., Leverett, P., & Newton, J. (2015). Culturally diverse undergraduate researchers’ academic outcomes and perceptions of their research mentoring relationships. Understanding Interventions, 37(15), 2533–2554. Google Scholar
  • Campbell, J. L., Quincy, C., Osserman, J., & Pedersen, O. K. (2013). Coding in-depth semistructured interviews: Problems of unitization and intercoder reliability and agreement. Sociological Methods & Research, 42(3), 294–320. https://doi.org/10.1177/0049124113500475 Google Scholar
  • Colón Ramos, D., & Quiñones-Hinojosa, A. (2016, August 4). Racism in the research lab. New York Times. Retrieved July 6, 2022, from https://
archive.nytimes.com/kristof.blogs.nytimes.com/2016/08/04/racism-in
-the-research-lab/ Google Scholar
  • Flanagin, A., Frey, T., Christiansen, S. L., & Bauchner, H. (2021). The reporting of race and ethnicity in medical and science journals: Comments invited. Journal of the American Medical Association, 325(11), 1049–1052. Google Scholar
  • Gibbs, K. D. J., McGready, J., Bennett, J. C., & Griffin, K. (2014). Biomedical science Ph.D. career interest patterns by race/ethnicity and gender. PLoS ONE, 9(12), e114736. https://doi.org/10.1371/journal.pone.0114736 MedlineGoogle Scholar
  • Haeger, H., & Fresquez, C. (2016). Mentoring for inclusion: The impact of mentoring on undergraduate researchers in the sciences. CBE—Life Sciences Education, 15(3), ar36. https://doi.org/10.1187/cbe.16-01-0016 LinkGoogle Scholar
  • Handelsman, J., Pfund, C., Lauffer, S. M., & Pribbenow, C. M. (2005). Entering mentoring: A seminar to train a new generation of scientists. Madison, WI: University of Wisconsin Press. Retrieved July 6, 2022, from https://www.hhmi.org/science-education/programs/entering-mentoring Google Scholar
  • Harmon, A. (2021, November 1). BIPOC or POC? Equity or equality? The debate over language on the left. New York Times. Retrieved May 25, 2022, from www.nytimes.com/2021/11/01/us/terminology-language-politics.html Google Scholar
  • McGee, E., Botchway, P. K., Naphan-Kingery, D. E., Brockman, A. J., Houston, S., II, & White, D. T. (2021). Racism camouflaged as impostorism and the impact on black STEM doctoral students. Race Ethnicity and Education, 25(4), 487–507. https://doi.org/10.1080/13613324.2021.1924137 Google Scholar
  • National Academies of Sciences, Engineering, and Medicine (NASEM). (2018). Graduate STEM education for the 21st century. Washington, DC: National Academies Press. https://doi.org/10.17226/25038 Google Scholar
  • NASEM. (2019). The science of effective mentorship in STEMM. Washington, DC: National Academies Press. https://doi.org/10.17226/25568 Google Scholar
  • National Research Council. (2011). Research training in the biomedical, behavioral, and clinical research sciences. Washington, DC: National Academies Press. Google Scholar
  • Opacich, K. J. (2019). A cohort model and high impact practices in undergraduate public health education. Frontiers in Public Health, 7, 132. https://doi.org/10.3389/fpubh.2019.00132 MedlineGoogle Scholar
  • Parson, L., Childs, B., & Elzie, P. (2018). Using competency-based curriculum design to create a health professions education certificate program the meets the needs of students, administrators, faculty, and patients. Health Professions Education, 4(3), 207–217. https://doi.org/10.1016/j.hpe.2018.03.008 Google Scholar
  • Pfund, C. (2015). Entering mentoring (1st ed.). New York, NY: Freeman. Google Scholar
  • Pfund, C., Byars-Winston, A., Branchaw, J., Hurtado, S., & Eagan, K. (2016). Defining attributes and metrics of effective research mentoring relationships. AIDS and Behavior, 20(Suppl. 2), 238–248. MedlineGoogle Scholar
  • Pfund, C., House, S. C., Asquith, P., Fleming, M. F., Buhr, K. A., Burnham, E. L., …& Sorkness, C. A. (2014). Training mentors of clinical and translational research scholars: A randomized controlled trial. Academic Medicine, 89(5), 774–782. https://doi.org/10.1097/ACM.0000000000000218 MedlineGoogle Scholar
  • Pfund, C., Pribbenow, C. M., Branchaw, J., Lauffer, S. M., & Handelsman, J. (2006). Professional skills: Enhanced: The merits of training mentors. Science, 311(5760), 473–474. MedlineGoogle Scholar
  • Pfund, C., Spencer, K. C., Asquith, P., House, S. C., Miller, S., & Sorkness, C. A. (2015). Building national capacity for research mentor training: An evidence-based approach to training the trainers. CBE—Life Sciences Education, 14(2), ar24. LinkGoogle Scholar
  • Platt, M. O. (2020). We exist. We are your peers. Nature Reviews Materials, 5(11), 783–784. https://doi.org/10.1038/s41578-020-00248-x Google Scholar
  • Poodry, C. A., & Asai, D. J. (2018). Questioning assumptions. CBE—Life Sciences Education, 17(3), es7. https://doi.org/10.1187/cbe.18-02-0024 LinkGoogle Scholar
  • Puritty, C., Strickland, L., Alia, E., Blonder, B., Klein, E., Kohl, M., ... & Gerber, L. (2017). Without inclusion, diversity initiatives may not be enough. Science, 357(6356), 1101–1102. https://doi.org/10.1126/science.aai9054 MedlineGoogle Scholar
  • Ruiz, J. P., & Zárate, S. (2019). Latinx Scientists in the United States. The POSTDOCket, 17(7). Retrieved July 6, 2022, from www.nationalpostdoc
.org/page/POSTDOCket_1707 Google Scholar
  • Saldaña, J. (2009). The coding manual for qualitative researchers (3rd ed.). London: Sage. Google Scholar
  • Spencer, K. C., McDaniels, M., Utzerath, E., Rogers, J. G., Sorkness, C. A., …& Pfund, C. (2018). Building a sustainable national infrastructure to expand research mentor training. CBE—Life Sciences Education, 17(3), ar48. https://doi.org/10.1187/cbe.18-03-0034 LinkGoogle Scholar
  • Steinert, Y. (2010). Faculty development: From workshops to communities of practice. Medical Teacher, 32(5), 425–428. https://doi.org/10.3109/
01421591003677897 MedlineGoogle Scholar
  • Weber-Main, A. M., Shanedling, J., Kaizer, A. M., Connett, J., Lamere, M., & El-Fakahany, E. E. (2019). A randomized controlled pilot study of the University of Minnesota Mentoring Excellence Training Academy: A hybrid learning approach to research mentor training. Journal of Clinical and Translational Science, 3(4), 152–164. https://doi.org/10.1017/cts.2019.368 MedlineGoogle Scholar
  • Womack, V. Y., Wood, C. V., House, S. C., Quinn, S. C., Thomas, S. B., McGee, R., & Byars-Winston, A. (2020). Culturally aware mentorship: Lasting impacts of a novel intervention on academic administrators and faculty. PLoS ONE, 15(8), e0236983. https://doi.org/10.1371/journal.pone.0236983 MedlineGoogle Scholar