From the NSF: The National Science Foundation’s Investments in Broadening Participation in Science, Technology, Engineering, and Mathematics Education through Research and Capacity Building
Abstract
The National Science Foundation (NSF) has a long history of investment in broadening participation (BP) in science, technology, engineering, and mathematics (STEM) education. A review of past NSF BP efforts provides insights into how the portfolio of programs and activities has evolved and the broad array of innovative strategies that has been used to increase the participation of groups underrepresented in STEM, including women, minorities, and persons with disabilities. While many are familiar with these long-standing programmatic efforts, BP is also a key component of NSF’s strategic plans, has been highlighted in National Science Board reports, and is the focus of ongoing outreach efforts. The majority of familiar BP programs, such as the Louis Stokes Alliances for Minority Participation (now 25 years old), are housed in the Directorate for Education and Human Resources. However, fellowship programs such as the Graduate Research Fellowships and Postdoctoral Research Fellowships under the Directorate for Biological Sciences (and parallel directorates in other STEM disciplines) are frequently used to address underrepresentation in STEM disciplines. The FY2016 and FY2017 budget requests incorporate funding for NSF INCLUDES, a new cross-agency BP initiative that will build on prior successes while addressing national BP challenges. NSF INCLUDES invites the use of innovative approaches for taking evidence-based best practices to scale, ushering in a new era in NSF BP advancement.
INTRODUCTION
The National Science Foundation (NSF) has a long history of investing in efforts to broaden the participation of groups underrepresented in the science and engineering enterprise. Broadening Participation at the National Science Foundation: A Framework for Action (NSF, 2008) defined NSF’s activities in this area as encompassing programs for individuals, institutions, and geographic regions. A Congressional Research Service (CRS) report on NSF policy and activities further expanded the scope of efforts to broaden participation in science, technology, engineering, and mathematics (STEM) to include the following: “faculty diversity, the potential for bias in grant-making and ‘family friendly’ work environments for scientists and engineers, as well as teacher quality in schools that serve minority students, parental involvement and support for STEM-interested youth, and access to STEM-related opportunities and support programs” (Gonzalez, 2014, p. 13). The CRS report also noted that, because gaps in STEM achievement can be associated with socioeconomic status and geographic location, these issues should also be included under the umbrella of potential initiatives for broadening participation (BP).
Few would challenge the necessity for increased diversity as a means of strengthening STEM research and enlarging the future workforce or the ability of BP to contribute to the overall global competitiveness of the United States (National Academy of Sciences, National Academy of Engineering, and Institute of Medicine, 2011; Ferrini-Mundy, 2013; Campaign for Science and Engineering, 2014; Valantine and Collins, 2015). When the Framework was published, the NSF’s Broader Impacts merit review criterion (the standard by which all submitted proposals are assessed) included a question about the ability of the proposed activities to broaden the participation of groups underrepresented in science and engineering.1 Although the Broader Impacts criterion has since been modified to omit this direct question, BP is still included in the scope of activities that constitute broader impacts.2 Unfortunately, principal investigators appear to have good intentions in this area that are not often fully realized, as demonstrated in an evaluation that examined the implementation of broader impacts included in funded grant proposals in environmental biology (Watts et al., 2015).
While progress has been made in BP in STEM, the National Center for Science and Engineering Statistics (NCES, 2015) data show that those groups identified as being underrepresented in STEM (racial and ethnic minorities, women, and persons with disabilities) still lag behind their majority counterparts in STEM degree attainment and representation in the STEM workforce. For example, African Americans and Hispanics comprise 13 and 16% of the U.S. population, respectively, but together they earned less than 20% of the science and engineering bachelor’s degrees in 2012, and doctoral attainment remained at ∼7% for all underrepresented ethnic minorities between 2010 and 2012 (NCES, 2015). While the news is better for women, who earned 50.5% of the bachelor’s degrees and 41.1% of doctoral degrees in science and engineering in 2012, the number of undergraduate and graduate degrees awarded to women in computer science and mathematics and statistics declined between 2002 and 2012, and physics undergraduate degree attainment also decreased. The difference in bachelor’s degree attainment by the age of 24 for dependent family members from the uppermost and the lowest income quartiles increased from sixfold in 1970 (40 vs. 6% degree completion) to eightfold in 2013 (77 vs. 9% degree completion; Cahalan and Perna, 2015).
Although NSF-funded programs and the power to convene world-renowned researchers have enabled NSF to make great strides in BP, the agency has been encouraged by the federally appointed Committee on Equal Opportunities in Science and Engineering (CEOSE) to do even more. In the 2011–2012 biennial report to Congress, CEOSE called for the creation of a “bold, new initiative for broadening participation” (CEOSE, 2014). The committee envisioned large-scale centers that would focus on transforming STEM education and show both immediate and long-term impact on efforts to broaden participation nationally. In the subsequent CEOSE report (2015), the committee again highlighted progress, while iterating the need for an agency-wide effort, and noted that NSF must lead the way among federal agencies and beyond to impact the sometimes stagnant figures on STEM degree attainment and workforce diversity.
The purpose of this article is to review NSF’s past and current BP efforts, examine various approaches and strategies for BP support within programs, and describe the resulting portfolio and future directions at NSF. We aim to help generate an understanding of the additional efforts needed and the federal government’s response to the challenge of diversifying the STEM workforce. It is beyond the scope of this article to cover all approaches to BP identified by the CRS. Instead, the focus will be primarily on current NSF-funded programs and activities, with the hope that our readers will join us in this important task.
NSF’S HISTORY OF PROGRAMS FOR BP
The first NSF annual report covered two years, 1950–1951, and focused on planning and the need to develop “a national policy in basic research and education” (NSF, 1951). This report also indicated that there was a critical shortage of scientific manpower. That shortage was the impetus for creating the long-standing Graduate Research Fellowship Program (GRFP), which continues to play a vital role in supporting STEM graduate students today. While there was no mention of the need to diversify the STEM workforce in the early years, the report did suggest that women were “a potential source of scientific talent that can be utilized more fully” (NSF, 1951). Although women comprise nearly 30% of today’s science and engineering workforce, they are still not equally represented across all fields (NCES, 2015).
It was not until 20 years later that a concerted BP effort was undertaken by NSF, when agency director William D. McElroy announced that all programs would be asked to “give special consideration to the research needs of the black academic institutions, their faculty and students” (NSF, 1972). The next major milestone was the establishment of CEOSE by Congress in 1980 to advise NSF on “policies and activities” to ensure that all segments of the American public can fully participate in the science and engineering enterprise, including women, minorities, and persons with disabilities (U.S. Congress, 1980). This legislation (42 U.S.C.§1885(c.))3 provided guidance on the formation of CEOSE as part of the Science and Engineering Equal Opportunities Act and charged the committee to address, in national and international science and economic policy, equal access to STEM “education, employment, and training” for women, minorities, and persons with disabilities.
CEOSE’s influence is evident in the subsequent expansion of programs targeting underrepresented groups that were launched in the 1990s and in the call for research on the status of minorities in STEM education and in the U.S. STEM workforce. For example, in the 25 years since the creation of the Louis Stokes Alliances for Minority Participation program (LSAMP), nearly 500,000 undergraduate and graduate STEM degrees have been earned by underrepresented students supported by the program as documented in annual reports. Additionally the Program for Persons with Disabilities, which focused on increasing the quality and quantity of students participating in STEM who have disabilities, was also created during this period. (That program was later renamed Research in Disabilities Education, or RDE, to emphasize the shift to capacity building models.) The Minority Postdoctoral Research Fellowship program continues today as Postdoctoral Research Fellowships in Biology (PRFB) and supports multiple areas including, but not limited to, BP of groups underrepresented in biology. RDE has been incorporated into the Education and Human Resources Core Research Program (ECR), which includes a component on BP and institutional capacity.4
NSF’s strategic plans emphasize integrating education and research as a mechanism for building a diverse STEM workforce and include performance goals and strategic objectives that address both areas of NSF investment (NSF, 2014a). For example, desired outcomes demonstrating broader participation are found in the 2006–2011 strategic plan, which focuses on inclusiveness in the STEM enterprise and maintains a performance goal of BP. Additionally, as indicated in the 2014–2018 strategic plan, NSF-funded work will
Integrate education and research to support development of a diverse STEM workforce with cutting-edge capabilities.
Build an increasingly diverse, engaged, and high-performing workforce by fostering excellence in recruitment, training, leadership, and management of human capital.
STRATEGIES FOR ADDRESSING BP IN STEM
Although the majority of NSF’s BP efforts are in the form of programs, the agency also has published a number of reports on this topic in addition to the Framework for Action cited above. For example, the National Science Board’s Committee on Education and Human Resources hosted a workshop on Broadening Participation in Science and Engineering Faculty in 2004. The proceedings included among its recommendations the expansion of programs that increase degrees earned by individuals from underrepresented groups. Refer to Figure 1 and Table 1 for information on demographic shifts in degree recipients since 2000. Because many projects require an evaluation plan, a framework for evaluating BP efforts was also published in 2009 (NSF, 2009). National Science Board reports often discuss BP, since it is an important component of the agency’s strategic plans, as previously noted. It is also a topic discussed by directorate advisory committees. More recently, the NSF Federal Advisory Committee for the Directorate for Education and Human Resources (EHR) completed a strategic re-envisioning document that includes a chapter on BP. Not surprisingly, due to the substantive investments in BP made by the EHR, the EHR Advisory Committee encouraged increased leadership in BP across the agency in all directorates and also recommended that an effort be made to “enhance the visibility of broadened participation STEM and education workforce” issues (NSF, 2014b). NSF envisions that this may be accomplished by building on its decades of BP investments and offering alternatives to the long-standing but ineffectual deficit approach to more asset-based approaches that highlight the benefits and contributions of broadened participation in all aspects of STEM.
Figure 1. Percent change from the year 2000 in degrees awarded by race and ethnicity for (a) associate’s, (b) bachelor’s, (c) master’s, and (d) doctoral degrees. Table 1 compares these data with the population growth documented in the 2010 census. Source: National Science Board (2016), appendix tables 2-18, 2-23, 2-29, and 2-33.
| Population Δ% (census) | Associate’s degree Δ% | Bachelor’s degree Δ% | Master’s degree Δ% | Doctoral degree Δ% | |
|---|---|---|---|---|---|
| White | 6 | 46 | 21 | 25 | 12 |
| Asian or Pacific Islander | 43 | 40 | 42 | 42 | 36 |
| Black or African American | 12 | 69 | 32 | 85 | 51 |
| Hispanic | 43 | 93 | 66 | 96 | 69 |
| American Indian or Alaska Native | 18 | 50 | 39 | 64 | 48 |
| Other/unknown race or ethnicity | — | 257 | 164 | 179 | 129 |
Outreach to engage institutions serving diverse student populations with NSF programs is exemplified by a November 2015 cross–federal agency meeting for minority-serving community colleges to provide technical assistance in writing and submitting proposals and encourage the development of communities of practice.5 Dear Colleague Letters (DCL), first released in 2014 and again in 2015 and 2016, have been used to expand the pool of applications for projects to increase the participation in STEM of students at 2-year Hispanic-serving institutions (HSIs), including plans to increase access to research experiences, strategies to increase matriculation to 4-year degree programs, and research and conferences on evidence-based practices that have been particularly helpful for students at HSIs (NSF 15-063 [NSF, 2015a], 15-078 [NSF, 2015b], and 16-094 [NSF, 2016]). As part of the White House College Opportunity Summit in December 2014, NSF released a DCL inviting work to pilot innovations with high-impact potential for helping students learn the mathematics generally taught in the first 2 years of both 2-year and 4-year postsecondary institutions (NSF 15-026). Students from underrepresented groups are disproportionately likely to require tutoring to gain the needed skills in mathematics, and mathematics is a gateway to college success (Pryor et al., 2007).
Awareness of opportunities alone is not always sufficient for institutions to be successful in obtaining NSF funding. One model for technical assistance that is increasing the success of community colleges in gaining NSF funding is the Mentor-Connect program funded through the Advanced Technological Education (ATE) program.6 The project provides one-on-one mentoring on all aspects of grant preparation for principal investigators from institutions that have not received NSF funding in the past decade. While ATE is a workforce program, rather than a BP program, 45% of the participating students are members of underrepresented groups in STEM.7 Virtual networks of communities, including STEM Central,8 can also support implementation of effective BP practices. STEM Central originally supported STEM Talent Expansion Program (STEP) grantees but has expanded its reach to support all who are interested in implementing evidence-based and high-impact practices. Many of these practices, including student–faculty research and academic civic engagement, have been shown to result in increased student grades and increased persistence into the second year of college for students from underrepresented groups (Kuh, 2009). Integrating these practices into the STEM curriculum is an element of several NSF BP programs, including the NSF Scholarships in Science, Technology, Engineering, and Mathematics (S-STEM, for low-income students), the Centers of Research in Excellence in Science and Technology (CREST), the Tribal Colleges and Universities Program (TCUP), the Historically Black Colleges and Universities Undergraduate Program (HBCU-UP) and the long-standing LSAMP.
As illustrated with the mathematics DCL, some BP challenges are disciplinary specific. Very few minority-serving institutions (MSIs) and community colleges offer geoscience degrees, which creates a workforce challenge in this field. The Improving Undergraduate STEM Education: GEOPATHS (IUSE:GEOPATHS)9 program in the Geoscience Directorate aims to broaden participation of students from groups underrepresented in the geosciences, especially those at MSIs and community colleges. The IUSE: REvolutionizing Engineering and Computer Science Departments (IUSE:RED) program focuses on the second and third years of the undergraduate program to increase the graduation rates of women and underrepresented minorities.10 IUSE:RED emphasizes the development and engagement of the entire faculty within a department, identifying and dissipating departmental cultural barriers to inclusion of students and faculty from different backgrounds, and developing coherent technical and professional threads that include internship opportunities in the private and public sectors and research opportunities with faculty. Within the life sciences, an undergraduate mentoring program for students underrepresented in biology ran from 1999 until 2010, with an initial focus on environmental biology.11 The scaling of effective teaching practices for all undergraduate biology students is supported by the work of the Vision and Change Fellows.12
NSF’S BP PORTFOLIO
The BP portfolio was originally characterized in the Framework for Action as follows (NSF, 2008):
BP-focused programs: these programs have an explicit BP goal and 100% of the budget funds BP activity.
BP-emphasis programs: the review criteria are focused on BP, but projects may include a variety of elements; 50% or more of the budget funds BP activity.
BP potential: projects likely to address BP, but the percentage of the budget invested in BP varies.
Other: may include DCL or other mechanisms to address BP.
Although less than 10 years have passed since the Framework for Action was published, the 2015 summary of BP programs looks quite different from the original listing due to program retirements and more extensive use of DCLs. The current NSF BP portfolio is quite robust and represents an investment of more than $800 million (CEOSE, 2015). The updated listing shows just three categories and currently lists 16 BP-focused projects, 11 BP-emphasis projects, and 21 DCLs (see NSF BP portfolio: www.nsf.gov/od/broadeningparticipation/bp_portfolio_dynamic.jsp).
The CEOSE report, available on the NSF website, also includes a table of BP-focus and BP-emphasis programs, but does not incorporate DCLs, and categorizes EPSCoR as the sole geographic diversity program. This summary of the BP portfolio counts supplements and special program tracks separately, which results in slightly higher totals for BP-focus (18) and BP-emphasis (13) programs (see Table 2). NSF’s BP investment was more than $607 million in FY2013 and more than $638 million in FY2014 (CEOSE, 2015). However, the FY2015 request shows a total of $823.03 in the BP portfolio. Table 2 summarizes both sets of data by NSF directorate; no investment totals are shown for the website listings, since the amount funded for DCLs varies greatly and is included in program investments. The main point is that, while there are fewer programs than in prior years, the overall investment continues to increase annually.
| Directorate | BP-focused (NSF website) | BP emphasis (NSF website) | BP activities (DCLs) (NSF website) | CEOSE BP focus | CEOSE BP emphasis | CEOSE investment FY2015 request(millions) |
|---|---|---|---|---|---|---|
| BIO-Directorate for Biological Sciences | 1 | 1 | 1 | – | $2.50 | |
| CISE-Directorate for Computer and Information Science and Engineering | _ | 1 | 1 | 1 | $49.74 | |
| EHR-Directorate for Education and Human Resources | 6a | 2 | 5 | 7 | 9 | $541.23 |
| ENG-Directorate for Engineering | 2 | 1 | 9 | 3a | $13.43 | |
| GEO-Directorate for Geosciences | 1a | _ | _ | 1 | 1 | $3.82 |
| MPS-Directorate for Mathematics and Physical Sciences | 2a | 1a | 1a | 3 | $1.15 | |
| SBE-Directorate for Social, Behavioral, and Economic Sciences | 1 | _ | 2 | $3.0 | ||
| Offices-Office of Integrative Activities and Office of International Science and Engineering | 1 | 1a | $159.69 | |||
| All | 2 | 6 | 6 | 1 | 2 | $48.47 |
| Total | 16 | 11 | 21 | 18 | 14 | $823.03 |
The bulk of NSF’s BP investment resides in the Directorate for Education and Human Resources and includes previously mentioned BP-focus programs such as LSAMP, HBCU-UP, TCUP, CREST, S-STEM, and others that are included in both the CEOSE and BP website listings.13 These programs support a variety of evidence-based approaches designed to improve STEM undergraduate education and student retention and completion rates and may address both short-term goals and comprehensive institutional change. However, enduring fellowship programs such as the GRFP and the Postdoctoral Research Fellowships in Biology (PRFB), as well as the Geosciences Postdoctoral Fellowships, and the Social, Behavioral, and Economic Sciences (SBE) Postdoctoral Research Fellowships (all included in the CEOSE listing) are an effective means by which the programs in NSF’s disciplinary-focused directorates support BP by encouraging the participation of students from underrepresented groups, whereas many of EHR’s BP programs focus on institutions with high minority enrollment. Initiatives such as the SBE and EHR’s Science of Broadening Participation (initiated in 2011, currently soliciting proposals through jointly released DCL NSF 15-066; NSF, 2015c) and the Engineering Directorate’s Partnerships for Research and Education in Materials (PREM) are other examples of programs that are BP focused and invest 100% of their funds in BP through research. In the case of PREM, research collaborations between MSIs and research centers or similar facilities are funded by the Division of Materials Research.
Using the publicly available tool Deep Insights, Anytime, Anywhere (DIA2; www.dia2.org) to search the NSF award database, we searched for awards containing the phrase “broadening participation” and documented a steady increase over the past decade (Figure 2a). From 2003 through the first quarter of FY2016, 1002 awards were made, supporting 1713 collaborators and funded by programs across the agency. A broad range of research networks emerge from an analysis of the award principal and coprincipal investigators (Figure 2b).
Figure 2. Search results for NSF awards containing the term “broadening participation” between 1995 and 2016 using the DIA2.org search engine. (a) Number of awards per year. (b) Networks of collaborations among award recipients.
We continually learn from both individual projects and programs as a whole through program-level evaluation, integrating that knowledge into solicitation revisions. For example, from our STEP and S-STEM programs, the importance of focusing on success, not just access, for students from underrepresented groups led to an emphasis on pathways through the undergraduate years, reflected in the most recent S-STEM solicitation. From ADVANCE research and evaluation, a tool kit of Strategies for Effecting Gender Equity and Institutional Information emerged, with guidance on integrating different combinations of efforts to achieve specific goals in a given institutional context.14
LOOKING FORWARD
The NSF portfolio of projects that carry out research on strategies for BP and do capacity building continues to grow and generate new knowledge about increasing the participation of all in STEM and STEM education. For example, the evidence base for supporting the success of academic women has emerged through research in the NSF ADVANCE program.15 Both the success and barriers to success for undergraduate students from groups underrepresented in STEM fields are analyzed in the recent NSF-funded report, Barriers and Opportunities for 2-Year and 4-Year STEM Degrees: Systemic Change to Support Students’ Diverse Pathways (Malcom and Feder, 2016).
Many challenges remain for developing a diverse STEM workforce and a diverse, STEM-savvy public. To accelerate the inclusion of underrepresented groups in STEM, the FY2016 NSF budget request calls for a $15.5 million investment in a new initiative, the NSF Inclusion across the Nation of Communities of Learners of Underrepresented Discoverers in Engineering and Science (NSF INCLUDES).16 This is one of four priority areas for the agency for 2016. The goal is to set a bold vision for the nation that will accelerate adoption of practices that promote inclusive participation in science, scaling up our efforts across a range of institutions.
NSF director France Córdova convened an INCLUDES workshop on June 3, 2015, that resulted in the workshop report INCLUDES: Achieving Scale for Inclusion in STEM.17 The workshop focused on systems approaches with three themes: collective impact, scaling local success, and catalytic innovation. The workshop report synthesizes participant input and identifies potential critical levers that could drive a coordinated, transformative agenda for BP. Concurrent with the release of the NSF INCLUDES solicitation (NSF 16-544), Director Cordova released a DCL (NSF 16-048) inviting college presidents, chancellors, and other university leaders to become part of this new initiative.18
NSF INCLUDES will move forward with participation across all NSF directorates, building on existing programs and initiatives. For biologists, biology educators, and biology education researchers, there are multiple funding opportunities that can be used to tackle the challenge of BP. The Broader Impacts element of any NSF proposal can also focus on BP.
A suite of programs creates opportunities for individuals from underrepresented groups to receive support for their engagement in STEM learning in high-quality learning environments. Research Experiences for Undergraduates site and supplement awards can be used to increase the participation of students from underrepresented groups; early research experiences increase the likelihood of persistence in STEM majors (Hathaway et al., 2002; Eagan et al., 2013). Likewise, Research Opportunity Awards allow faculty at institutions with more limited research infrastructure to spend time working with an NSF-funded investigator elsewhere, bringing along one or two undergraduates. The Robert Noyce Teacher Scholarship Program prepares STEM teachers to teach in high-needs school districts, while Research Experiences for Teachers engages teachers in research experiences with the aim of helping them integrate that experience back into their schools. For students pursuing STEM degrees, the S-STEM program provides scholarships for cohorts of academically talented students with demonstrated financial need, while generating new knowledge about curricular and cocurricular supports that create optimal pathways to insure student success in STEM majors. At the graduate level, we can all encourage talented undergraduates and beginning graduate students to consider applying for the Graduate Research Fellowship Program.
For those interested in change at the level of one or more institutions or across biology departments nationally, the Research Coordination Networks in Undergraduate Biology Education program and the Improving Undergraduate STEM Education: EHR (IUSE:EHR) program’s Institutional Transformation track are options, along with LSAMP, ADVANCE, HBCU-UP, CREST, TCUP, and Alliances for Graduate Education and the Professoriate (AGEP). The ATE program can be used to build capacity at minority-serving community colleges in biotechnology, bioinformatics, and other life sciences-related fields in which there are strong demand for highly skilled technologists. ATE has the option to scale to regional and national centers and networks. Public participation in biological research (citizen science) is funded by both the Biology Directorate (for research that advances the biological sciences) and the EHR Advancing Informal STEM Learning (AISL) program. The latter program investigates learning in the context of the scientific research project and could advance our collective understanding of how to engage a more diverse populace in doing science. All institutional transformation efforts build on existing knowledge and are expected to generate new knowledge about STEM learning.
To fully advance a BP agenda, we need to continue building the knowledge base on effective strategies. Basic research in this area is encouraged in all of the divisions of the SBE Directorate and in EHR, especially in the EHR Core Research program. For more applied BP research, consider the IUSE:EHR program and most of the HRD programs. For example, HBCU-UP and TCUP include a Broadening Participation Research track that is designed to support STEM education research on questions specific to the populations served by those programs. It is a promising time to systematically and at scale engage all with an interest in biology more fully in learning and doing biology.
FOOTNOTES
1 NSF’s merit review criteria were revised in 2007 to include language on transformative research and then again in 2013 following a National Science Board (NSB) task force report released on this topic in December 2011; see www.nsf.gov/nsb/publications/2011/meritreviewcriteria.pdf.
2 See NSF Grant Proposal Guide, chap. 2, sect. C.2d.i: www.nsf.gov/pubs/policydocs/pappguide/nsf13001/gpg_2.jsp#IIC2d.
4 See Postdoctoral Research Fellowships in Biology: www.nsf.gov/pubs/2015/nsf15501/nsf15501.htm; and EHR Core Research: www.nsf.gov/pubs/2015/nsf15509/nsf15509.htm.
5 http://sites.ed.gov/octae/2015/12/03/minority-serving-community-colleges-meeting-the-future-now.
7 www.evalu-ate.org/wp-content/uploads/formidable/2015-ATE-Survey-Report-FINAL.pdf.
13 NSF program acronyms and titles: ATE, Advanced Technological Education; AISL, Advancing Informal STEM Learning; AGEP, Alliances for Graduate Education and the Professoriate; ADVANCE, Increasing the Advancement and Participation of Women in Academic Science and Engineering Careers; CREST, Centers of Research Excellence in Science and Technology; ECR, EHR Core Research Program; EPSCoR, Experimental Program to Stimulate Competitive Research; GRFP, Graduate Research Fellowship Program; HBCU-UP, Historically Black Colleges and Universities Undergraduate Program; NSF INCLUDES, Inclusion Across the Nation of Communities of Learners of Underrepresented Discoverers in Engineering and Science; IUSE, Improving Undergraduate STEM Education; IUSE:GEOPATHS, Improving Undergraduate STEM Education: Pathways into Geoscience; IUSE:RED, Improving Undergraduate STEM Education: REvolutionizing Engineering and Computer Science Departments; LSAMP, Louis Stokes Alliances for Minority Participation; PREM, Partnerships for Research and Education Materials; PRFB, Postdoctoral Research Fellowships in Biology; RDE, Research in Disabilities Education; S-STEM, Scholarships in Science, Technology, Engineering, and Mathematics; STEP, STEM Talent Expansion Program; TCUP, Tribal Colleges and Universities Program.
15 www.portal.advance.vt.edu/index.php/categories/filetype/annotated-bibliography.
17 www.informalscience.org/sites/default/files/INCLUDES_Convening_Synthesis.Sep1.pdf.
