Reading Primary Scientific Literature: Approaches for Teaching Students in the Undergraduate STEM Classroom

INTRODUCTION

The reading of primary scientific literature (PSL) is an essential scientific skill. For instance, academic scientists report reading several hundred articles a year, and most scientists surveyed identify the reading of PSL as important for scientific training (Tenopir et al., 2009, 2015; Hubbard and Dunbar, 2017). Similarly, instructors of science, technology, engineering, and math (STEM) courses report that science process skills associated with the reading of PSL, such as the interpretation of graphs and tables, are among the most important skills for undergraduate students to acquire (Coil et al., 2010). The importance of reading PSL stems from the many cognitive and affective benefits that students gain when reading and engaging with it, including increased knowledge of scientific process skills and improved self-confidence in science (Brownell et al., 2013; Krontiris-Litowitz, 2013; Sato et al., 2014). These benefits for students have sparked multiple calls to enhance instruction on reading PSL in the undergraduate STEM classroom, including recommendations from Vision and Change and the American Association of Medical Colleges and the Howard Hughes Medical Institute’s Scientific Foundations for Future Physicians (Alpern et al., 2009; American Association for the Advancement of Science [AAAS], 2011).

In light of the documented benefits of reading PSL, there are various instructional approaches and interventions to guide students’ development of reading PSL published in the education literature. However, these approaches vary widely in their scope and target populations, as well as the level of assessment and outcomes measured. Additionally, to our knowledge, no previous comprehensive attempts have been made to compile these approaches into an accessible framework for instructors, with past work only including a limited number of approaches and the assembling of a framework that examines outcomes, but not target population and type of assessment data (Sloane, 2021). Similarly, despite these published approaches, instructors still face multiple barriers for incorporating evidence-based practices (including the reading of PSL) in the undergraduate STEM classroom, suggesting an urgent need for additional resources to support PSL in STEM education (Brownell and Tanner, 2012).

In this Essay, we provide an overview of the current literature on reading PSL in undergraduate STEM courses. We discuss studies examining how students read PSL as well as the benefits and challenges of reading PSL. Next, we review and compile published approaches for teaching PSL, incorporating a framework that allows instructors to easily identify approaches for teaching PSL that are potentially compatible with their classes. As part of this framework, we summarize the methods, outcomes, target populations, and assessments of a wide variety of practices designed to bring PSL into the undergraduate STEM classroom. We conclude by discussing implications and recommendations for instructors as well as the science education research community.

BENEFITS AND CHALLENGES OF TEACHING PSL

Reading PSL in STEM classes leads to a range of benefits for students, including both cognitive and affective gains. For instance, students who read PSL demonstrate an increased understanding of the scientific process, including better comprehension in developing research questions and hypotheses, interpreting figures and experimental results, and drawing inferences and conclusions from data (Wenk and Tronsky, 2011; Wagoner, 2016; Sloane, 2021). Similarly, reading PSL can lead to increased familiarity with and comprehension of content knowledge (DebBurman, 2002; Brownell et al., 2013; Krontiris-Litowitz, 2013; Murray, 2014; Yeong, 2015; Kararo and McCartney, 2019; Chatzikyriakidou et al., 2021). In addition, reading PSL can also improve students’ data analysis and critical thinking, increase their scientific and information literacy, augment interest and excitement in science, and contribute to higher self-efficacy and confidence in their ability to succeed in science (Carter and Wiles, 2017; Schmid et al., 2021; Sloane, 2021). These benefits can persist in the long term, with students demonstrating improved scientific process skills and academic advantages in subsequent courses (Kozeracki et al., 2006; Sato et al., 2014). Remarkably, programs for undergraduates centered around reading primary literature have also led to students experiencing greater success while pursuing graduate work in science (Kozeracki et al., 2006).

Despite these benefits, there have been multiple barriers identified to teaching PSL. For instance, when queried why they did not teach PSL, instructors at universities with very high research activity (i.e., R1 Carnegie Classification) stated that they felt that the courses they were teaching were not appropriate for having students read PSL (Wagoner, 2016). Instructors similarly identified time limitations for the course, as well as the perceived challenging nature of PSL, as reasons preventing them from implementing the reading of PSL in classes. These barriers are exacerbated by the limited time, lack of training, and dearth of incentives for many faculty to reform their teaching (Brownell and Tanner, 2012). Consequently, these challenges have led to discrepancies in how different instructors perceive the importance of PSL and how much their classes emphasize the reading of PSL (Hubbard and Dunbar, 2017; Hubbard, 2021). In addition to these barriers for faculty, undergraduate students can also be resistant to reading PSL. For instance, many students report that reading such papers can be difficult and frustrating, which may in turn dissuade faculty from continuing to teach PSL (Hubbard and Dunbar, 2017; Lie et al., 2016; Howard et al., 2021). Many of these barriers persist even for graduate students reading PSL in STEM courses, suggesting that teaching PSL in undergraduate STEM courses will likely have both cognitive and affective benefits for graduate students as well. For instance, while a survey of biology PhD students at one university revealed higher self-efficacy in reading PSL than undergraduates, other studies identified that master’s students in biology report having similar challenges as undergraduates in comprehending the experiments, figures, and language used in PSL, and may also have relatively low self-efficacy in some of the skills needed to read PSL (Abdullah et al., 2015; Lie et al., 2016; Hubbard and Dunbar, 2017).

HOW DO STUDENTS READ, PROCESS, AND LEARN ABOUT PSL?

Learning to read PSL is a complex cognitive activity that is influenced by multiple factors (Nelms and Segura-Totten, 2019; Hubbard, 2021). Several theoretical frameworks explain how students develop skills to read PSL, including Alexander’s model of domain learning (MDL; Lie et al., 2016; Hubbard and Dunbar, 2017; Hubbard, 2021). Under the MDL framework, students undergo three phases of development when learning to read PSL: acclimation, competency, and proficiency (Alexander, 1997, 2003, 2005). Students first must become familiar with new concepts and new jargon presented by the paper they are reading (acclimation). Given their unfamiliarity with such concepts and jargon, students largely rely on “superficial processing strategies” that lead to poorly formed mental frameworks of the concepts of the paper (Brill et al., 2004; Lie et al., 2016; Hubbard, 2021). Consequently, students report gaining only a superficial understanding of papers they read in class if they are not provided with explicit instruction or guidance on how to read a paper (Brill et al., 2004). However, given guidance, students can progress to forming more complex mental models of the concepts and can develop more personal interest in the topics (competency), before engaging with the concepts in a deeper and broader manner (proficiency; Alexander, 1997, 2003, 2005). These differences between those who are at the acclimation stage and those who are at the proficiency stage become apparent when comparing how novice readers of PSL and “expert” readers of PSL (e.g., faculty and instructors) approach PSL. For example, expert readers of PSL rely on more complex schemas—that is, mental structures and strategies for engaging with a cognitive task (Piaget, 2003)—when reading PSL, thus lowering the cognitive load of reading such articles (Nelms and Segura-Totten, 2019). Faculty tend to reread sentences more frequently and are more likely to summarize and take notes when reading PSL than novice readers of PSL (Nelms and Segura-Totten, 2019). Novice readers, in contrast, tend to ignore unfamiliar terms or concepts and need to be prompted more frequently to aid their comprehension by taking notes and creating diagrams when reading PSL (Brill et al., 2004). Similarly, novice readers perceive methods and results as the most challenging sections of PSL to read and understand and thus underemphasize the importance of the results section. Novice readers are also likely to identify incorrect conclusions and supporting evidence for those conclusions (van Lacum et al., 2012; Hubbard and Dunbar, 2017). Expert readers, in contrast, focused on a more holistic approach to reading PSL and reported using selective reading of different sections of PSL to guide their comprehension (Hubbard and Dunbar, 2017).

As students acquire practice in reading and processing PSL, they experience affective changes as well. For instance, many students report reading PSL for extrinsic reasons, for example, reading PSL to identify answers to homework assignments and to prepare for examinations, although there are differences in students’ levels of interest and perceived utility for reading PSL (Hubbard and Dunbar, 2017; Chatzikyriakidou and McCartney, 2022). However, some students and expert readers of PSL report more intrinsic motivations for reading PSL, citing innate interest and enjoyment of reading the PSL and wanting to learn more about a given study (Hubbard and Dunbar, 2017; Howard et al., 2021). These differences suggest that instruction on reading PSL can also influence how students perceive PSL and impact their motivations for reading PSL. Indeed, PSL-based interventions in single courses have been shown to change students’ motivations and interests in reading PSL (Chatzikyriakidou and McCartney, 2022).

IDENTIFYING APPROACHES TO TEACHING PSL

We reviewed the literature for published approaches for teaching PSL in the undergraduate STEM classroom by conducting a keyword search using Google Scholar, PubMed, and ERIC (Education Resources Information Center). In addition, we compiled additional papers by referencing methods and approaches cited by other papers. We limited our search to papers describing methods for teaching PSL in undergraduate STEM classrooms, given the differences in how primary literature is structured in non–STEM fields as well as the unique set of challenges facing instructors who wish to introduce PSL in primary, secondary, and graduate education (Koeneman et al., 2013; Fankhauser and Lijek, 2016). We acknowledge that there may be broad disciplinary differences in how different fields of STEM conceptualize PSL, as well as variation in the emphasis that each field places on teaching PSL in the undergraduate classroom. However, rather than limiting our search to specific disciplines of STEM, we opted to conduct a broad, inclusive search spanning approaches for teaching PSL across STEM to gain a better understanding of which fields have the most published approaches for teaching PSL. This approach also allows us to include some published approaches that span STEM disciplines. We then contextualize our results and any differences in rates of publication of approaches for teaching PSL by discussing these potential differences in disciplinary approaches to PSL.

In addition, we limited our search to approaches for teaching PSL and did not include published curricula in which the authors provided specific handouts and instructions on the use of a specific PSL article in the classroom, given that such curricula are only designed to be implemented with one piece of literature and are not broadly generalizable. We also did not include approaches that focused on teaching students how to read secondary sources (e.g., newspaper articles, popular science media, and reviews), which have a different intended audience than PSL and are thus usually written in different structures and language than PSL (Yarden, 2009). While several of our identified approaches involved the instructor asking students to find relevant secondary articles or to read secondary sources to read to complement the reading of PSL (Hostettler and Wolfe, 1984; de Silva, 2018; Mitra and Wagner, 2021), we do not include approaches that focus solely on secondary scientific literature, given that past work has identified differences in students’ cognitive and affective behaviors when engaging with PSL versus reading secondary sources (Baram-Tsabari and Yarden, 2005). For instance, a study of biology high school students demonstrated that students tend to report understanding secondary sources better than PSL but those students who read secondary sources exhibited lower gains in inquiry skills than their peers who read PSL (Baram-Tsabari and Yarden, 2005). Given these differences between PSL and secondary sources, we focus only on approaches for teaching PSL here. We thus included any peer-reviewed manuscript that described instructional techniques or approaches with the objective of guiding students toward reading and/or interpreting any part of PSL, including techniques designed to introduce students to analysis of PSL figures and data.

We classified each approach by three aspects:

• Details of intervention: In addition to summarizing the approach itself, we also noted the length of the intervention as well as the modality of approach.

  • ○ Disciplinary context: We noted for each instructional approach the disciplinary context that it fit into, such as biology or chemistry. Some of the approaches may span several disciplines (e.g., approaches that described students reading PSL in biology classes but with the learning objective of developing mathematical and quantitative reasoning skills); in these cases, we noted the discipline of the class in which the approach was initially situated.

  • ○ Length of intervention: We noted whether the instructional approach was intended for a single class period, multiple class periods (but lasting no more than 4 weeks), or integrated approaches that spanned half the length of an entire course or more.

  • ○ Modality of approach: We similarly recorded whether the intervention took place primarily in class or outside class as homework. Some of the pedagogical approaches were nearly evenly split between both in-class and outside class time, so these were binned as mixed in modality.

• Target student population: We characterized instructional approaches by the level of course that the intervention is geared for, the type of college or university that the intervention was developed for, and the size of the class in which the intervention was deployed.

  • ○ Size of class: We documented whether each technique was implemented and assessed using classes that were small (fewer than 25 students, including larger courses divided into smaller sections), medium (25–50 students), or large (more than 50 students).

  • ○ Level of course: We logged whether each intervention was targeted at students enrolled in STEM courses for nonmajors, introductory-level STEM courses (defined as courses taken by STEM majors primarily in their first or second year as the first collegiate course in a discipline), intermediate-level courses (those geared toward second- and third-year students following introductory courses), or advanced courses (those for third- and fourth-year students).

  • ○ Type of college: We also logged whether the intervention was implemented at 2-year or 4-year colleges and universities.

• Assessment evidence: We examined whether each published approach included data assessing the efficacy of the assessment. We gathered information on which constructs (increased content knowledge, interest in science, etc.), if any, were assessed; the assessment population (including the class level); and the scope of assessment (i.e., whether the intervention was assessed at one institution or across multiple institutions).

SUMMARY OF APPROACHES

We identified a total of 74 approaches for teaching PSL (Supplemental Table S1). These approaches varied drastically by length of intervention, modality, target student population, and assessment evidence (Figure 1). However, we identify a few themes among these approaches. First, we find that every pedagogical approach in the literature has been conducted with students from 4-year colleges and universities, with only one of the approaches (which was developed at a 4-year college) also describing implementation and providing evidence of effectiveness from 2-year colleges (Figure 1A). This paucity of pedagogical approaches developed for or tested at 2-year colleges is striking, given that about one-third of STEM students in the United States are currently enrolled in 2-year colleges (Varty, 2022), and calls into question the effectiveness of such techniques across a broader range of students and the generalizability of the assessment data provided in these papers. Similarly, only three of the approaches for teaching PSL have been developed for STEM courses for nonmajors (Figure 1B), despite calls that have highlighted the need to train non–STEM majors in scientific literacy (Gormally and Heil, 2022). While we acknowledge that some instructors may find low utility in teaching PSL to non–STEM majors, we highlight the lack of studies that have examined the impact of teaching PSL in nonmajors courses and the possibility that having such students engage with PSL may lead to improvements in critical thinking and scientific literacy. We also identify that most approaches involved an instructor dedicating multiple class sessions to teaching PSL or spanned more than half a term, although approximately one-third of published approaches only required a single class session (Figure 1D).

In addition, we find that many papers that described instructional strategies for teaching PSL did not have any assessment data on student learning or student affect, and those that did have data were limited in several aspects. For instance, we identify that all but one of the approaches were only assessed with students at one institution, with many only assessing a small number of students enrolled in that one class. More assessment data with more diverse populations is needed for these studies to establish the impact of such pedagogical interventions, given the demographic variation among students in different classes and institutions. Similarly, the assessment data included in these studies largely focused on either cognitive (e.g., science process skills or content knowledge) or affective (e.g., interest and motivation in STEM) outcomes. Few studies examined the impact of the specific PSL intervention on both cognitive and affective constructs, suggesting that additional data are needed to fully measure the efficacy of such techniques. We also observe that many of the studies we examined reported instructor perceptions (e.g., instructors indirectly reporting on student perceptions and informal comments) or selected narrative comments from student evaluations of teaching. In our suggested framework for implementing the teaching of PSL (see Suggested framework for instructors to teach PSL section), we encourage instructors to use these types of data to help guide their reflections and iterative improvement of their teaching of PSL. In addition to these types of data, though, there may be opportunities to develop assessment instruments that are reliable and valid and to collect more formalized assessment data that test the efficacy of such approaches across diverse contexts.

Finally, we identify that there are major discrepancies between different STEM disciplines in the number of approaches published (Figure 1C). For instance, we found that the majority of approaches for teaching PSL were situated within biology, while we only identified a small number across engineering, math, or physics. There are several possible reasons for this variation. First, there may be different disciplinary literacies across these fields. For example, Vision and Change highlights the importance of reading PSL in biology (AAAS, 2011), and a recent survey of undergraduate biology program learning outcomes revealed that reading and analyzing PSL was the sixth most common competency among undergraduate biology programs in the United States (Clark and Hsu, 2023). This disciplinary emphasis on reading PSL and its associated skills may explain why most of our identified approaches are situated in biology. In contrast, the Accreditation Board for Engineering and Technology, a coalition formed by engineering societies to provide accreditation for engineering programs, does not reference PSL or any reading of original research in its nationally recognized student outcomes (ABET, 2021). Similarly, the Joint Task Force on Undergraduate Physics Programs, convened by the American Physical Society and the American Association of Physics Teachers, lists recommended learning goals for physics programs, which also do not include any mention of PSL or reading original research (American Physical Society, 2016). These data may indicate that such disciplines may not rely on reading PSL at the undergraduate level, though future work is needed to investigate this further and explore potential reasons. However, such a lack of emphasis on PSL may explain why there are so few published approaches for reading PSL in these disciplines.

Similarly, we note that there are very few published approaches for teaching students the reading of PSL in math. The few approaches we identified were centered more on statistics and applied mathematics (Grindle et al., 2021; Maher, 2005; Rabin and Nutter-Upham, 2010) and not on pure math. A blog post on the American Mathematics Society illuminates why this may be, noting that “the present research frontier in mathematics is generally too far removed from the undergraduate experience to make [reading of PSL] possible” (Barnett et al., 2015). Instead, most of the literature surrounding the reading of primary sources in math has centered on increasing the use of historical mathematical literature and assessing the efficacy of integrating such texts in the undergraduate math classroom (Jahnke et al., 2002; Fenaroli et al., 2014; Jankvist, 2014; Barnett, 2022). However, there remain relatively few pedagogical approaches for reading PSL published in math, though there are several collections of instructional modules centered around different historical mathematical texts that instructors may wish to consult (Barnett et al., 2013; Bezhanishvili et al., 2003). In addition, we note that several of these fields may rely on different modalities to communicate original research. For instance, some STEM disciplines may communicate more through conference proceedings, potentially limiting the amount of PSL available and leading to less emphasis on reading PSL (Lisée et al., 2008). Finally, we note that, while many of these approaches can be broadly applicable for reading PSL across STEM disciplines, there may be significant variation between STEM disciplines about how PSL is presented and how students read and conceptualize the concepts from such articles. There may thus be an opportunity to develop discipline-specific strategies for reading PSL and assess whether such techniques can augment the impact of reading PSL in that discipline.

SUGGESTED FRAMEWORK FOR INSTRUCTORS TO TEACH PSL

Based on our review of approaches, we propose a suggested framework that follows the principles of backward design for instructors wishing to teach PSL in their classes (Allen and Tanner, 2006; Roth, 2007).

1. Determine the intended goals of having students read PSL. Our work identified that different published approaches for teaching PSL had different aims. For instance, some approaches were designed to primarily promote student conceptual knowledge of the discipline when reading PSL, while others included explicit interventions to develop several competencies, such as having students generate hypotheses and make models. Other approaches were also designed to positively influence student affect, such as increasing student motivation and interest in STEM. Therefore, we recommend that instructors first determine what their primary objective of having students read PSL is. For instance, is the instructor more interested in promoting content knowledge in the class, or providing a chance for students to develop specific competencies, such as asking scientific questions and evaluating hypotheses? While many approaches that were only designed for one of these goals may likely have impacts in the other areas as well, instructors should carefully consider which of these goals they want to prioritize by examining both their course learning outcomes and their program learning outcomes and reflecting on any available assessment data from the course, including past iterations. Instructors may also wish to refer to any developed standards for their discipline that relate to teaching PSL. For instance, the BioSkills Guide (Clemmons et al., 2020) is a resource for biology instructors that is aligned with the Vision and Change report (AAAS, 2011). Developed through an iterative process that included a diversity of instructors and other content experts, the guide includes both program learning outcomes and specific learning outcomes for undergraduate biology education, including multiple learning outcomes related to the reading of PSL (Clemmons et al., 2020). Thus, instructors of biology classes may wish to use this framework when considering their intended learning objectives and goals when teaching PSL.

2. Choose PSL to read. Several of the interventions summarized in this work included detailed instructions or techniques for selecting an appropriate PSL article. There are many factors that instructors should consider when choosing PSL, and we build upon the framework previously suggested by Muench (2000) to highlight several of these factors. First, instructors should identify a paper that allows them to accomplish their intended learning outcomes for the course and educational goals of incorporating PSL. For instance, if the instructor primarily wishes to have students gain conceptual understanding of a given technique or practice analyzing results from a set of experiments, the instructor should identify a paper that includes such techniques and analysis. Other considerations include the length and complexity of the paper. For instance, the instructor will wish to think about what level of background information is needed to engage with each specific article being considered, and whether students in the class are likely to have this level of knowledge. In addition, the specific article should be chosen in a way that will best encourage students to accomplish the learning goals. This is particularly true if the instructor chooses an approach that does not scaffold how students approach the paper by dividing it into smaller sections. A lengthy paper, even if not overly complex, may intimidate and overwhelm a novice reader of PSL, so instructors of nonmajors or introductory-level courses may wish to err on the side of less-complex papers that present findings in a more digestible manner. Instructors may also wish to consider both the anticipated level of student interest in the chosen paper and the age of the paper. Students report being more engaged when reading PSL that relates to a topic area they are interested in, and thus will likely be more motivated by such literature (Howard et al., 2021). Similarly, we speculate that students may be more interested in and motivated to read papers that are more recent, as they may have an easier time recognizing the relevance of the work if it is current. Finally, instructors may also wish to consider the accessibility of the article. For instance, different institutions subscribe to different journals, so instructors should ensure that their institutions have access to a selected article and may also wish to rely on freely accessible open access articles (Pence and Losoff, 2011).

3. Design instruments to assess the intended learning goals. Next, instructors should determine what evidence they would need to determine that students have met the learning goals and design instruments aligned with these goals. For instance, if instructors are interested in assessing changes in student affect or mastery of core competencies, they can use established instruments that measure such constructs, whereas if instructors are interested in students’ gain of content knowledge, they may need to write questions themselves based on the chosen PSL.

In addition to considering alignment of the assessment questions based on the intended learning outcome, instructors should also carefully consider alignment of the cognitive level of the questions with those of the intended learning goals. To start, instructors should reflect on whether their intended learning goals involve lower- and/or higher-order cognitive skills, which are differentiated by whether they require deep, conceptual understanding (Zoller, 1993). For example, an instructor who primarily intends the use of PSL to be a mechanism for introducing new vocabulary terms in the discipline may have lower-order learning goals and assessment questions (e.g., questions that rely on recalling a definition or providing a term), while those wishing to use PSL to guide students in generating hypotheses and evaluating data will have higher-order cognitive learning goals and assessment questions. Instructors may wish to use the Blooming Biology Tool, which provides a framework for instructors based on Bloom’s taxonomy, to assist with this reflection and alignment (Crowe et al., 2008). Bloom’s taxonomy provides a hierarchical list of cognitive domains that range from lower-order (remembering and understanding) to higher-order cognitive skills (applying, analyzing, synthesizing, and evaluating; Bloom, 1956; Crowe et al., 2008; Larsen et al., 2022).

Instructors may also examine the choice of instrument for assessment once they identify an approach to use. However, we note that our data reveal that many instructional techniques for teaching PSL either do not have assessment data to demonstrate the technique’s efficacy or have assessment data that are limited in scope. Finally, we note that instructors may also wish to draw upon published instruments that can be used to measure students’ mastery of content or given competencies or provide insight into how students engaged with the PSL. For example, concept inventories are validated tools that can provide key insight into student learning on a concept or a related set of concepts (Evans et al., 2003; Smith and Tanner, 2010; Sands et al., 2018), and there are disciplinary guides available for instructors on how to best use concept inventories to support student learning and assessment (Furrow and Hsu, 2019). We also highlight several specific, validated instruments that may be of interest to instructors wishing to measure student competencies. For instance, the Test of Scientific Literacy Skills measures students’ scientific literacy (Gormally et al., 2012), while there are separate instruments focusing specifically on STEM students’ critical thinking (e.g., White et al., 2011), information processing (Reynders et al., 2020), scientific reasoning (e.g., Moore and Rubbo, 2012), and motivation and self-efficacy when reading PSL (Chatzikyriakidou and McCartney, 2022). Instructors may also wish to rely on published frameworks for evaluating assessments, such as the Three-Dimensional Learning Assessment Protocol (Laverty et al., 2016), when searching for and deciding on appropriate assessments to use. Finally, we highlight the PSL Reading Strategies Assessment, which can be used to provide insight into how students are approaching the assigned PSL readings (Lee et al., 2022).

• 4. Identify appropriate pedagogical method for teaching PSL. Instructors can use our table, wherein we identify over 70 approaches for teaching PSL (Supplemental Table S1), and determine which approach best fits their given class when considering learning goals, amount of time available, class level, and outcomes desired. To facilitate instructors finding an appropriate pedagogical method, this table is organized by the disciplinary context for which each of the given pedagogical approaches was developed, though we note that many of the approaches may be broadly applicable within STEM. In addition, within each discipline, we further sort the entries by length of intervention. We have also included a sortable spreadsheet file of this table (see Supplemental Material) so that instructors can search through the approaches more easily. Instructors may also wish to merge parts of different approaches or consider tailoring approaches to fit their classes. We provide a few suggestions based on themes that emerged from these approaches:

  • Check for campus resources. We note that many of our identified approaches commence with instruction in information literacy (including finding, locating, and evaluating PSL), which often involves sessions led by university librarians or integrating resources from the library. These resources developed by librarians can be invaluable for instructors to help guide students on using campus databases, locating articles appropriate to their discipline and interests, conducting literature searches, and differentiating between research and review articles; past work has identified that effectively structured partnerships between campus librarians and STEM faculty can support student learning and scientific literacy (Scaramozzino, 2010; Schilling and Applegate, 2012). Thus, we urge instructors to check with their STEM librarian on campus before teaching PSL to see what resources may be available.

  • Scaffold how PSL is taught. Scaffolding—the breaking down of a complex concept or task into smaller parts that can be structured with guidance for students—is a common instructional strategy that can improve students’ self-regulation of learning and student learning outcomes in STEM (Belland, 2016; Offerdahl et al., 2017; Sabel et al., 2017; Ewell et al., 2022). Unsurprisingly, we see that nearly all the identified approaches to teaching PSL scaffold how PSL is introduced and taught, often dividing the PSL into smaller sections for students to read. Scaffolding is known to reduce the cognitive load in students; thus, scaffolding of PSL may prevent students from becoming overwhelmed and discouraged by the size and complexity of a PSL article (Rhodes et al., 2020). In addition, multiple approaches further scaffold the reading of PSL by providing a set of guided reading questions for students. By seeking answers to the guided reading questions, students read the PSL article with greater focus and direction. Other scaffolded interventions dictate that students read the article in small groups, complete writing assignments of increasing analytical depth, or interpret data figures one by one. Given that past work has shown that asking students to read PSL without such guidance and structure may lead to negative impacts on student attitudes (Chatzikyriakidou and McCartney, 2022), it is imperative that instructors provide a structured framework for undergraduates to progress toward competency and proficiency when reading PSL.

  • Integrate development of oral and written communication. We note that the reading of PSL provides an ideal platform to integrate opportunities for students to develop both oral and written communication, competencies highlighted as critical for undergraduate education in national reports and disciplinary accreditation bodies across multiple STEM disciplines (AAAS, 2011; ABET, 2021; American Chemical Society, 2023; American Physical Society, 2016). We identified multiple approaches in which students orally communicated their insights from reading PSL through journal clubs, small-group discussions, and presentations. Similarly, many approaches involve students submitting written summaries and analyses of the PSL, providing for the development of written communication skills. We urge instructors to consider how their identified approach and assessment will promote students’ oral and written communication skills.

• 5. Teach, assess, reflect, and iterate. Following these steps, instructors can then implement their planned approaches for teaching PSL, assess the efficacy of their approaches, reflect on these data, and then iterate through the process again by making changes when needed. This reflection on teaching is critical and has been identified as a key part of improving undergraduate STEM courses (Henderson et al., 2011). In addition, given that so many of our identified approaches lack assessment data, we urge instructors to share their assessment data and insight with the STEM education community through publications and presentations, which will benefit future instructors.

We hope that this framework will help instructors mitigate many of the identified barriers for teaching PSL. For instance, one of the main barriers instructors have cited for not teaching PSL is the lack of instructional time (Wagoner, 2016). Our review identified a diversity of approaches for teaching PSL, including some that only focus on certain components of PSL, such as the title or figures (Liao, 2017; Massimelli et al., 2019). These approaches should require smaller amounts of instructional time as compared with reading an entire article, potentially allowing more instructors to incorporate the reading of PSL in their classes. Similarly, many instructors also report having limited time themselves as a barrier for curricular changes (Brownell and Tanner, 2012). We hope that our framework and compiled list of published approaches for teaching PSL serve as a useful and time-saving resource for instructors, guiding instructors to think about the integration of PSL in their classes and allowing instructors to identify and select a published approach for teaching PSL easily.

In addition, many instructors may hesitate to teach PSL, given that many students may find such an endeavor challenging (Lie et al., 2016; Hubbard and Dunbar, 2017; Hubbard et al., 2022). Instructors can take several steps to address this barrier. For instance, instructors may wish to discuss their own experiences with reading PSL and highlight how they may have struggled with reading PSL themselves as students (and indeed, may still struggle reading some PSL now). This use of noncontent Instructor Talk can build students’ relationships with their instructors as well as students’ self-efficacy (Seidel et al., 2015; Harrison et al., 2019). Similarly, instructors may wish to briefly discuss and justify their choices of pedagogical approach for teaching PSL, again potentially building students’ confidence and trust in the activities. Finally, each of the approaches included in our Essay is designed to guide students’ reading of PSL, directly addressing this concern. Instructors may wish to highlight students’ learning and gain in ability to read and process PSL after finishing instruction, again building students’ self-efficacy.

THE CREATE METHOD AS A CASE STUDY

Next, we highlight how several of the published approaches have integrated this framework, including iterative assessment of both cognitive and affective skills. Specifically, we focus on the Consider, Read, Elucidate hypotheses, Analyze and interpret data, and Think of the next Experiment (CREATE) method of reading PSL that was first published in 2008 (Hoskins, 2008) as an exemplar of how one published approach for teaching PSL led to multiple follow-up studies that incorporated assessment from a wide variety of contexts, leading to a more comprehensive understanding of how the CREATE method promotes student reading of PSL.

The initial paper describing CREATE in 2008 was set in the context of an undergraduate neuroscience course, describing how CREATE was used to guide students to learn about developmental neurobiology (Hoskins, 2008). The article provided an overview of the process but did not include any rigorous assessment data, similar to many of the articles we identified in our review. However, the author of this 2008 paper continued refining the approach and teaching using CREATE, and then gathered data to examine the impact of the CREATE method on both student attitudes toward and abilities gained in reading PSL, finding positive gains (Hoskins et al., 2011). Subsequently, the original author and colleagues recognized that the data they gathered were limited, because they came from only one institution and were limited to the specific context of neuroscience. Since then, the authors and others have revised and implemented the CREATE method in different contexts, with many publications discussing modifications of this approach, assessment of students’ cognitive and affective changes, as well as ways to train and support faculty in implementing this technique (Hoskins et al., 2007, 2017; Hoskins, 2008, 2010, 2019; Gottesman and Hoskins, 2013; Stevens and Hoskins, 2014; Hoskins and Krufka, 2015; Hoskins and Gottesman, 2018; Kenyon et al., 2016, 2019; Hsu, 2020; Krufka et al., 2020). For example, this body of literature on CREATE includes research that examines the effectiveness of this approach in 2-year colleges (Kenyon et al., 2016), explores how this approach can be modified to guide students to read and think critically about secondary sources (Hoskins, 2010), investigates the impact of CREATE on students’ attitudes toward science in both introductory and upper-division courses across diverse institutions (Stevens and Hoskins, 2014; Hoskins and Gottesman, 2018), assesses the efficacy of both shortened or elongated versions of the approach (Gottesman and Hoskins, 2013; Krufka et al., 2020) or integrating the approach into lab courses or lab-like settings (Hoskins et al., 2007; Hoskins and Krufka, 2015), and provides assessment data on the impact of workshops to train faculty at both 2- and 4-year colleges on this approach (Hoskins et al., 2017; Kenyon et al., 2019). This body of literature showcases how a single instructor’s approach for teaching PSL, once published, can lead to iterative refinement and assessment in a broad diversity of contexts, furthering our understanding of how students learn to read PSL and the impact of a given approach for teaching PSL. Thus, this case study suggests that there may be opportunities for instructors who implement an approach for teaching PSL to provide valuable assessment data to the scientific education community.

IMPLICATIONS FOR THE SCIENCE EDUCATION RESEARCH COMMUNITY

We identify several areas of interest for the STEM education research community. First, there is an urgent need to expand the assessment of pedagogical approaches for teaching PSL and to use instruments with evidence of validity and reliability. We identify that most approaches for teaching PSL were only tested with a limited audience, with very few techniques developed for or assessed at 2-year colleges. Indeed, most approaches were limited to testing the efficacy within one course or institution; very few approaches were tested within a diverse set of institutions. We identified only one approach—the CREATE method—that has led to a number of publications investigating the effectiveness and impact of the approach across different institutions, including both 2-year and 4-year colleges (Kenyon et al., 2016; Stevens and Hoskins, 2014). This paucity of work done at 2-year colleges is particularly striking, as community college STEM students are more diverse and contain a greater percentage of students historically minoritized in STEM compared with their 4-year counterparts, and these students face unique challenges in STEM (Hoffman et al., 2010; Varty, 2022). This limited study population calls into question the generalizability of the results, and we call on the STEM education research community to implement and test the efficacy of such approaches with a more diverse range of student populations. In addition, there is an urgent need to examine whether 2-year colleges are teaching PSL at different frequencies than 4-year colleges. For instance, past work has suggested that many 2-year colleges may not have the resources to subscribe to many journals in STEM, limiting what PSL instructors can teach in the STEM classroom and preventing students from exploring PSL deeply (Pence and Losoff, 2011). Thus, there is an urgent need to examine how PSL is taught in the context of 2-year institutions and to develop interventions that can mitigate potential barriers.

Second, there is a need to investigate factors that lead to any differences in how students of different demographics approach and read PSL. While past work has focused on expert–novice comparisons as well as student attitudes toward reading PSL (Verkade and Lim, 2016; Nelms and Segura-Totten, 2019), we are not aware of any previous studies that have examined variation within the undergraduate student population in terms of their affect and approaches toward reading PSL. Given that past work has identified differences in scientific literacy between different demographics of students (including by race/ethnicity, first-generation status, and level of religiosity; Nuhfer et al., 2016; McPhetres and Zuckerman, 2018; Shaffer et al., 2019) and adults (Allum et al., 2018) that cannot be fully explained by past educational history and/or socioeconomic status, we speculate that there may be differences in how different student demographics feel about PSL and how they read PSL. Examining these possible differences and exploring the factors that shape such differences can provide powerful insight for developing interventions and refining pedagogical approaches for teaching PSL.

Finally, we identified that the assessment of pedagogical approaches for teaching PSL is also primarily limited to testing changes in student conceptual understanding of the topics introduced by the paper and in a few affective constructs, such as interest. Very few of these published pedagogical approaches investigated changes in students’ approaches to reading PSL, their understanding of the process of science, or their motivations toward reading PSL and science. There is thus a critical need to expand assessment of these approaches to measure the impact on these additional constructs, which will provide additional context into the efficacy of such pedagogical approaches.

LIMITATIONS

We acknowledge several limitations of our work. First, it is limited to examining published strategies for teaching PSL in the context of undergraduate STEM courses. There are other interventions and strategies for teaching the reading of PSL in non–STEM courses that may be translatable to STEM courses that we did not review here. Similarly, we acknowledge that there may exist disciplinary variations between the fields of STEM regarding how PSL is structured. Our work here did not examine these differences, instead focusing on broad-level approaches for teaching PSL. We also do not include any approaches that are designed to guide students on searching, finding, and using PSL (e.g., Klucevsek and Brungard, 2016). Despite these limitations, our work provides the first comprehensive summary of strategies to promote student reading of PSL that we are aware of that incorporates a synthesis of outcomes, assessment data, and target audience.

ACKNOWLEDGMENTS

REFERENCES