Postsecondary biology students’ ways of participating in the critique and discussion of primary scientific literature

INTRODUCTION

Professional standards of practice for both K–12 and postsecondary science education emphasize regular engagement in the knowledge generating practices of science as central components of science learning (American Association for the Advancement of Science [AAAS], 2011; National Research Council [NRC], 2012). When students learn science as a practice—rather than science as a collection of facts—they have the opportunity to both understand science knowledge and also to learn how to participate in science practices that generate science knowledge (Duschl, 2008; Walker et al., 2016; Bolger et al., 2021). Science practices include activities such as asking questions, developing and using models, planning and carrying out investigations, analyzing and interpreting data, constructing explanations, engaging in argument from evidence, and obtaining, evaluating, and communicating information (NRC, 2012). These science practices are also represented by the core competency for undergraduate biology education of applying the process of science (AAAS, 2011; Clemmons et al., 2020). As part of each practice, scientific knowledge is generated through the interplay of idea construction and idea critique (Ford, 2008a). Scientists construct ideas in response to their questions and when they develop models, plan investigations, interpret data, and construct explanations. Of equal importance, scientists critique ideas when they consider multiple hypotheses, refine models, negotiate an appropriate experimental design, scrutinize how data were analyzed and interpreted, and evaluate alternative explanations. In short, science moves forward because newly constructed ideas are refined through critique. Thus, as postsecondary educators create opportunities for students to learn science as practice, it is important to consider the role of critique in science and understand how postsecondary students participate in this role.

One common opportunity for students to learn to participate in critique in university science classrooms is through reading and discussing primary scientific literature. There are many instructional approaches for teaching undergraduate students how to read primary scientific literature for different goals (e.g., Hoskins et al., 2007; Hoskins et al., 2011; Goldey et al., 2012; Murray, 2013; Round & Campbell, 2013; Lacum et al., 2014; Sato et al., 2014; He & Masuda, 2015; Rawlings, 2019). However, fewer published studies have focused on describing how students discuss primary literature with their peers in the classroom, either using critique or using other forms of participation (although see Edwards et al., 2001; Gillen, 2006; Zagallo et al., 2016; Wood, 2020). In a recent review on approaches to teaching primary scientific literature in undergraduate science, technology, engineering, and mathematics (STEM) classrooms, Goudsouzian & Hsu (2023) highlighted the importance of educators identifying clear goals for having students read primary scientific literature and then collecting evidence that students met those goals. If one goal for reading primary scientific literature is for students to discuss and critique the ideas generated in the article, then it is important to be able to recognize the varied ways that students participate in critique in the context of classroom primary literature discussions.

However, there are a diversity of ways of generating knowledge in science and also within subdisciplines of the biological sciences (Longino, 1990; Gray, 2014; Bang et al. 2017). Because of this diversity, it is unreasonable to frame science practices—like critique of primary literature articles—as discrete packages of rules for students to follow and educators to assess. Across subdisciplines of biology, students’ participation in critique will look different depending on the particular focus of the research article and each student’s purposes and intentions for what and how they critique. Our goal is to expand on how we understand the practice of critique and the forms of participation that emerged during primary literature discussions in postsecondary biology education contexts.

Towards this goal, we purposefully designed this work to capture ways of participating at unique cross-sections along the biology program of study at this institution: a lower-division undergraduate course, a journal club embedded in a summer research experience, and a cross-listed upper-division undergraduate and graduate course. In each case of this multi-case study, students were guided on how to prepare for the weekly primary literature discussions. The guidance also highlighted varied ways of participating in primary literature discussions. The structure of the discussions intentionally centered student agency and authority in shaping the weekly article discussion. Thus, these cases were appropriate contexts to investigate how postsecondary biology students participate in the critique and discussion of primary scientific literature. Toward this goal, we asked two specific research questions:

1. What do students attend to when they critique primary literature articles during group discussions?

2. What forms of participation (other than critique) emerged from students reading and discussing primary literature articles across these three contexts?

In what follows, we first review the literature that informed our analysis of the practice of critique, productive disciplinary engagement in science practices, and goals for having students read and discuss primary literature. Second, we present our qualitative, multi-case study to answer our research questions. Third, we consider how similarities and differences in the instructional contexts help explain the patterns of participation that emerged in primary literature discussions. We end by discussing instructional implications and directions for future research.

Critique as a practice of science

Science, as a discipline, is concerned with making claims about the natural world. Moreover, these claims are put forth “through the process of argument– relating the imaginative conjectures of scientists to the evidence available” (Driver et al., 2000, p. 295). It is through this process of inspecting how conjectures are related to evidence that critique is used in conjunction with science practices (e.g., planning and carrying out investigations, analyzing and interpreting data, constructing explanations) to generate knowledge of the natural world (Ford, 2008a; Ford, 2008b; NRC, 2012). This process of constructing and critiquing claims is one that scientists have become accustomed to because, according to Ford (2008b), they have developed a “grasp” of scientific practice. Scientists know that the construction of new knowledge must withstand the critique of the scientific community because they have learned how constructing new knowledge works in tandem with critiquing new knowledge. Ford (2008b) describes how, compared with nonscientists, scientists are critical of the ways that claims are developed because they know that certain information is essential in determining its validity, exemplifying their “grasp” of scientific practice. Fundamentally, learning science is intertwined with understanding that the construction of new scientific knowledge does not occur without critique (Ford, 2008a). Thus, goals for science education emphasize both what we know in science and how we know that information (Duschl, 2008). The how we know component directly relates to making knowledge generation practices visible to students and creating opportunities for students to engage in critique as a disciplinary practice.

However, the shift to emphasize teaching science as practice in classrooms raises the question of how to foster authentic science engagement in the classroom. Engle and Conant (2002) originally put forth four essential components to foster productive disciplinary engagement in science: 1) problematizing, where students work on disciplinary problems that can be addressed through a variety of approaches and have multiple possible solutions; 2) students are positioned with the authority to shape the knowledge generation in the classroom by constructing and critiquing ideas, 3) the ideas that students generated are held accountable (by both students and teacher) to the norms and routines of their classroom and the discipline for handling, developing, critiquing, and using ideas, and 4) adequate resources, such as materials and time, are available to support the disciplinary work. Forman and Ford (2014) used productive disciplinary engagement as a lens to examine how students participated in the construction and critique of ideas to solve a disciplinary problem. They defined critique as:

In our work, we used Engle and Conant’s four essential components to identify postsecondary biology classroom contexts where we expected to observe productive disciplinary engagement in the practice of critique during primary literature discussions. We also used Forman and Ford’s (2014) definition to identify instances of critique.

Also using productive disciplinary engagement as an analytical lens, Ong et al. (2020) characterized the disciplinarity of critique in group discussions. Ong et al. (2020) distinguished between epistemic and nonscientific types of critiques. Nonscientific critiques described things like communication goodness, or how easy it was to understand a particular aspect of the work (e.g., a data table, figure, or writing style). Epistemic critiques described things like how the data were collected, why particular data were appropriate to serve as evidence, or how data were represented and analyzed to answer a research question. In our work, we found Ong’s et al. (2020) distinction between epistemic and nonscientific critiques useful to categorize different ways of participating in critique. However, we view both types of critique as playing necessary roles in shaping how science is done and communicated. Thus, in framing this work, we renamed Ong et al.’s (2020) category of nonscientific to stylistic, given the important role of stylistic critiques in science. The features of epistemic and stylistic critiques are listed in Table 1.

Recognizing varied forms of participation as productive disciplinary engagement

Although the essential components of productive disciplinary engagement provide guidance on how educators can foster participation in the practices of science, these components do not describe how to recognize varied or unexpected forms of authentic participation that may emerge in the classroom. Kapon et al. (2018) defined disciplinary authenticity as a diverse, context-dependent construct comprising the conceptual, epistemological, social-institutional, and affective aspects of science. Disciplinary authenticity involves taking up practices that have been socially negotiated by disciplinary communities in a way that is also personal to one’s own intentions and purposes (Levrini et al., 2015). Thus, the authenticity of ways of participating in science practices is shaped by the norms of the science community within each subdiscipline of science (e.g., evolutionary biology, microbiology, paleobiology; Gray, 2014). What is considered as authentic participation can also be governed by figures of power and authority, which is one reason Engle and Conant (2002) and Forman and Ford (2014) both emphasized the importance of positioning students with authority to shape the knowledge generation in their classroom science community.

The research on disciplinary authenticity highlights our previous point that, because authenticity is context-dependent and tied to individual intentions and purposes, it is unreasonable to frame science practices—like the practice of critique—as universal protocols of rules. The growing body of literature focused on the role of uncertainty in science and science education, highlights how scientists modify and reimagine the ways they engage in their science practice to respond to unexpected challenges and circumstances (e.g., Manz, 2015; Bolger et al., 2021). Thus, productive ways of participating in disciplinary practices develop over time and students have epistemic agency to play a role in shaping the knowledge generating practices of their science classroom communities (Stroupe, 2014; Miller et al., 2018; Stroupe et al., 2018; Stroupe, 2023). Yet, because students have agency to shape how they participate in practices, educators need to become attuned to recognize varied forms of productive disciplinary engagement in critique. Previous research has documented that the ways that students engage with the primary literature are varied, and many students feel overwhelmed, anxious, and intimidated by scientific research articles during their entire postsecondary experience (Round and Campbell, 2013). Such feelings of anxiety and intimidation can arise when certain forms of participating are privileged and other ways of participating go unrecognized or are misinterpreted (Bang et al., 2017). Our goal is to expand how we understand the practice of critique and describe the varied forms of participation that emerged during primary literature discussions. Such descriptions are necessary for biology instructors to design for, recognize, and celebrate varied forms of participation.

Goals for having students read and discuss primary scientific literature

Providing opportunities for students to read and discuss primary literature is a common goal for postsecondary biology educators. Recently, Goudsouzian & Hsu (2023), reviewed published approaches to achieve this goal across STEM disciplines. Approaches described in the literature include strategies to focus students’ attention on understanding and interpreting the figures within articles (Round and Campbell, 2013), methods to teach students the rhetoric of research articles (Lacum et al., 2014), and approaches to “demystify” and humanize the process of science by dissecting articles and planning questions and comments to relay to the authors themselves (Hoskins et al., 2007; Hoskins et al., 2011). The articles reviewed by Goudsouzian & Hsu (2023) (e.g., Goldey et al. 2012; Murray, 2013; Sato et al., 2014; He & Masuda, 2015; Rawlings, 2019) contribute to educators’ and researchers’ understanding of how to engage students with reading and understanding primary scientific literature towards different goals. Our research aims to contribute descriptions of how postsecondary students participate in discussions of the primary literature articles they read. In the previous sections, we reviewed the literature that we used to conceptualize and recognize ways of participating in the practice of critique during primary literature discussions. However, there are ways of participating in primary literature discussions other than critique that are also aligned with educators’ goals for having students read and discuss primary literature. Along with expanding how we understand the practice of critique, we aim to describe the varied forms of participation (other than critique) that emerged during primary literature discussions.

Research on how students socially participate in discussing and making sense of primary scientific literature spans a variety of contexts, approaches, and instructional goals (e.g., Edwards et al., 2001; Gillen, 2006; Wood, 2020). Often, students indicate how they read and understood an article by summarizing aspects of the article (e.g., Shannon & Winterman, 2012; Round & Campbell, 2013; Sato et al., 2014; Nelms & Segura-Totten, 2019; Rawlings, 2019). Zagallo et al. (2016) described how undergraduate students participated in sensemaking in small groups to use target models of biological phenomena to interpret data from primary literature articles. The authors found that students collaboratively generated ideas about data figures and supported their own and their peers’ ideas with direct evidence from the figure. Zagallo et al. (2016) emphasize that participation in collaborative sensemaking commonly emerged in small groups although it was not explicitly prompted by the instructor. Making sense of a biological phenomenon and the methods researchers used to examine it is a typical instructional goal for engaging students with scientific literature. Thus, sensemaking is a productive way of participating during literature discussions. Some instructors integrate scientific literature with the goal of students making connections between ideas in the article and the concepts and techniques they are currently learning about in the course (Jacques-Fricke et al., 2009; Yeong, 2014). Summarizing, sensemaking, and connecting are three goals suggested in the literature for having students read and then discuss primary scientific literature. Our work adds to this literature by describing the nuanced ways postsecondary students participate in summarizing, sensemaking, and connecting during primary literature discussions at three cross-sections along the biology program of study so that educators and researchers can become attuned to recognize and support varied forms of productive disciplinary engagement in discussing scientific literature.

METHODS

University context and case study selection

This research focused on undergraduate biology education at a four-year university in the Intermountain West United States. At the time of this study, this institution had approximately 12,000 students enrolled, both undergraduate and graduate. This department served over 500 undergraduate biology majors and 50 graduate students. The department offered both lecture and laboratory courses and additional undergraduate research opportunities with biology faculty and graduate student mentors.

We conducted a descriptive multiple case study to qualitatively describe the forms of engagement that occurred via critique and otherwise when postsecondary biology students read and discussed primary literature articles in a group setting (Tracy, 2019). This study used purposeful sampling to identify three contexts for case studies based on designated characteristics (Tracy, 2019). Each case needed to provide structured opportunities for productive disciplinary engagement in discussing primary literature articles. In each case, students were positioned with the authority to shape the primary literature discussions and guided in how to hold ideas (both in the article and surfaced through discussion) accountable to the norms and routines of their classroom and the discipline for developing, critiquing, and using ideas. Each aspect of an article was problematized as something that could continue to be critiqued and discussed despite already passing through a peer review process to publication. Finally, students were provided resources for learning both how to read an article in preparation for discussion and how to facilitate a discussion. Students were given sufficient time for meaningful discussion and the instructor was also present as a resource.

Importantly, these three cases were selected because the same guidelines for how to read articles, how to prepare for discussion, and how to participate in article discussions were provided in each context. The 200-level Ecology course and the 400/500-level Organismal Biology course were taught by the same professor (Dr. B, pseudonym), using the same approach to structure primary literature discussions. To explore how students discussed primary literature outside of a course context, we selected an undergraduate summer research program that included a Journal Club. In the Journal Club, the graduate student facilitators also used Dr. B’s guidelines to set expectations for reading and preparing to discuss primary literature. The aspects that were similar and different across the three cases are shown in Table 2. Each case was bounded by one semester. Our analysis for each case was focused on four of the weekly discussions spaced evenly across the semester: one discussion at the beginning, two discussions in the middle, and one discussion at the end. Below, we describe the cases in order of how the experiences were situated within the trajectory of the biology degree program.

The 200-level course: Ecology.

The 16-wk lecture and laboratory course focused on the foundations of how organisms and the environment interact. The professor teaching the lecture (Dr. B) also designed the laboratory portion of the course. The labs were led by a graduate teaching assistant. For 12 weeks, the laboratory portion of the course began with a primary literature discussion of an article that was related to the subsequent laboratory investigation. Each whole-class discussion was about 30 min. The primary literature discussions were structured such that two students were randomly selected at the start of the lab to lead the discussion. The role of the leaders was to start and guide the conversation. The course syllabus set expectations for each student to participate in each discussion by asking questions and expressing their ideas. Participation was a graded component of the lab. Students were provided with the same primary literature discussion guidelines as the other two cases Figure 1. Written primary literature reports were another graded component of the course. Students selected five of the twelve articles to write reports on. Each report was due before the class with the primary literature discussion for that article. Dr. B provided resources for how to read a scientific paper, instructions for what to include in the primary literature report, and a template with sentence starters for how to organize the report. The reports were two pages at the minimum and were expected to include the following: 1) Describe background information, problem statement, and the questions/hypotheses; 2) Briefly describe the experimental methods; 3) Describe what the research found (i.e., results) and whether the hypotheses were supported; 4) Explain why this study is important; 5) Propose next steps and possible future experiments. Dr. B required students to complete three of the five reports before the midpoint of the semester and gave feedback on each report to help students improve for subsequent reports. The written reports and feedback were not analyzed as part of this research study, but it is important to describe these assignments because they were a component that structured how students engaged with the primary literature articles in preparation for the class discussions. All ten students enrolled in the lab section were invited to participate in the research and six (60%) consented. Comments from nonparticipants during the primary literature discussions were not included in the analysis.

Summer undergraduate research program: Journal Club.

This case was the Journal Club portion of the 10-wk Biology Summer Undergraduate Research Program at the University. The program funded two biology graduate students to organize the weekly programming. Each week, undergraduate students conducting summer biology research were invited to meet with their peers to discuss primary literature articles related to their research. Although all undergraduate researchers were invited to participate in this portion of the program, six undergraduates regularly participated in the Journal Club (three 4th years, one 3rd year, and one 2nd year). Of those six, five were involved with the same research group for their summer research. One of the primary mentors of this research group was Dr. B, the same faculty who taught the Ecology and Organismal Biology courses. Undergraduate students in this research group were expected to participate in the Journal Club as part of their paid research experience position. Additional expectations of the paid research experience included a written research proposal and an oral presentation at the department’s summer research symposium. Although the proposal and oral presentation were not analyzed as part of this research study, these components could shape the participants’ purposes for selecting, reading, and discussing articles in the Journal Club.

The Journal Club was designed so that each week a different student selected an article related to their research to discuss with the group. This student was the discussion leader that week. Over the course of the summer, each Journal Club participant selected an article and lead the discussion at least two times. Students in the Journal Club were provided with the same primary literature discussion guidelines as the other two cases Figure 1. In addition, students in the Journal Club were prompted to think about their summer research in preparation for the weekly discussion (e.g., How does this paper relate to your research or the research of your peers in the journal club? Are there any takeaways that can inform your research project as you progress through the summer? Are you inspired to troubleshoot a problem in a new way that you could discuss with your mentor? Are there any figures or tables that you could use to model your own data?). All students that attended Journal Club were invited to participate in the research and six (100%) consented to the study.

The 400/500-level course: Organismal Biology.

This was an upper-division Organismal Biology lecture course with a laboratory component. The course introduced students to the structure, development, taxonomy, and life histories of insects and the ecological, economic, and management considerations. Both undergraduate and graduate students were enrolled in this course, which is why our research questions specify postsecondary students rather than undergraduate students. The professor (Dr. B) was the same professor as the 200-level Ecology course. Dr. B taught both the lecture and laboratory components of the course. The lecture portion of the course met for 50 min on Monday, Wednesday, and Friday. This was a 16-wk course and primary literature discussions took place during 10 of the Friday class meetings for approximately 30–50 min. The structure of the primary literature discussions was the same as the Ecology course. Two students were randomly selected at the start of class to lead the whole-class discussion. The role of the leaders was to start and guide the conversation, not to give a presentation on the article. The course syllabus set expectations for each student to participate in each discussion by asking questions and expressing their ideas. Participation was a graded component of the lab. Students were provided the same primary literature discussion guidelines as the other two cases Figure 1. Written primary literature reports were a graded component of the course and were structured the same as the Ecology course. The written reports and feedback were not analyzed as part of this research study, but it is important to describe these assignments because they structured how students engaged with the primary literature articles in preparation for the class discussions. All students enrolled in the course were invited to participate in the research and eleven (100%) consented to the study.

Data sources and collection

Guidelines for preparing for and participating in primary literature discussions.

The cases in this multi-case study were provided the same explicit guidelines for how to prepare to discuss and then how to participate in primary literature discussions Figure 1. To ensure this consistency, the guidelines were collected for each case.

Primary literature discussion transcripts.

Audio recordings of all primary literature discussions were collected in each case: Ecology (12 discussions), Journal Club (10 discussions), and Organismal Biology (10 discussions). Audio files were transcribed using a transcription service, reviewed for quality control, and deidentified. The first author reviewed every discussion transcript in the three cases and then purposefully selected four focal discussions from each case based on the timing of the discussion within the semester and participant attendance Table 2.

Data analysis

Guidelines for preparing for and participating in primary literature discussions.

We reviewed and compared the guidelines for each case to ensure consistency. The guidelines outlined the role of the discussion leaders and provided questions for students to keep in mind while reading the articles. The guidelines also addressed how to critique constructively. Dr. B adapted these guidelines from documents his biology faculty mentors used when he was in graduate school to guide primary literature discussions in undergraduate courses. Dr. B had designed primary literature discussions into his courses before the semesters this study took place. In the previous semesters, Dr. B refined the guidelines to address elements of the discussions he was not yet satisfied with. For example, Dr. B noticed in previous semesters that his students mostly critiqued weaknesses of the article and did not comment on the strengths of the articles during discussions (Dr. B personal. communication during instructor interviews). This observation led him to add the bullet point of, “CRITIQUES are CONSTRUCTIVE” and the last item of, “Don’t spend the whole time trashing a paper…,” to the guidelines in Figure 1 before the data collection for this study took place. Importantly, the guidelines communicated expectations for varied ways of participating productively in primary literature discussions by emphasizing epistemic critiques, stylistic critiques, and forms of productive engagement that were not critique. These guidelines informed our analysis and the varied ways of participating we anticipated might emerge in the primary literature discussions.

Primary literature discussion transcripts.

We used the qualitative analysis software ATLAS.ti 23 to analyze the focal discussions (12 discussions total: four from each case) at the unit of talk turn to understand 1) what students attended to when they critiqued primary literature articles during group discussions, and 2) what forms of participation other than critique emerged from students reading and discussing primary literature articles. A turn of talk began when a person started speaking and ended when they stopped a continuous turn at speaking. Both researchers independently coded the twelve discussion transcripts, using the coding scheme described in Figure 2. Multiple ways of participating could occur in one turn of talk. We compared our coding and resolved any differences to achieve 100% consensus (see Validity and Reliability section below).

Instances of critique.

We used Forman and Ford’s (2014) definition of critique to distinguish between talk turns that functioned as critique and talk turns that represented ways of participating that were not critique. For each instance of critique, we coded the disciplinarity of the critique, the substance of the critique, and whether the critique evaluated a strength or a weakness of the article (Figure 2). The disciplinarity of each critique was categorized as either epistemic or stylistic (Table 1). The substance of the critique captured the aspect of the article the critique focused on (Table 3). Finally, we coded whether the critique highlighted a strength or a weakness of the article. Coding these three aspects of critique allowed us to answer our research question: what did students attend to when they critiqued primary literature articles during group discussions across the three cases? Based on the provided guidelines (Figure 1), we expected to observe students making both epistemic and stylistic critiques. For example, the guidelines prompted epistemic critiques focused on different aspects of the articles: Did the authors address their research questions? Are the methods appropriate and valid? Closely examine the figures/results. These guidelines also prompted stylistic critiques: Was it well written? Why or why not? Do I like this paper or not, and why? Last, multiple guidelines direct students to attend to both strengths and weaknesses: Point out things that you like first and then things that you think could be changed to improve the paper. Don’t spend the whole time trashing the paper, make a list of good or novel things about the research.

Other ways of participating.

When we determined that a turn of talk was not an instance of critique based on Forman and Ford’s (2014) definition, we coded it as a category of other ways of participating in primary literature discussions. Based on previous literature and the primary literature discussion guidelines (Figure 1), we expected to observe students participating in discussions by summarizing, sensemaking, and making connections. We expected summarizing might occur based on guidelines like, what were the key points? We expected sensemaking might occur based on guidelines like, what did you not understand? We expected students might make connections based on guidelines like, what is a new idea this paper gave you? The goal of this research is to describe varied forms of participation in primary literature discussion. Thus, we left our coding open for ways of participating to emerge as we analyzed the data.

Cross-case analysis.

After we achieved 100% agreement in our code applications, we constructed an analytic summary for each focal discussion and conducted the within case analysis. We used the Code-Document analysis tool and the Query Tool in ATLAS.ti to enumerate the relative frequencies of ways of participating within each case. Then we conducted a cross-case analysis to compare the patterns for each case and to consider how similarities and differences in the instructional context for each case helped explain the patterns of participation in primary literature discussions we observed within each case (Merriam and Tisdell, 2022). The primary literature discussion guidelines described varied ways to participate in critique and discussion of primary scientific literature and our analyses provide insight into how students put these guidelines into practice across the three cases.

Validity and reliability.

For reliability, the two researchers discussed the coding scheme in detail and then independently coded the 833 turns of talk across the twelve discussion transcripts, resulting in 963 code applications. After comparing our coding, we were 90% in agreement, and 99 code applications (10%) required further discussion to come to a consensus. We reviewed the 99 code applications and reached a consensus, resulting in 100% agreement.

For validity, the first author reviewed transcript excerpts and discussed code applications with Dr. B to validate the analysis with a biologist’s perspective on discussing primary scientific literature in biology. The first author conducted two semi-structured interviews with the course instructor. The first interview occurred at the beginning of the research and captured Dr. B’s purposes for implementing primary literature discussions in his courses and in mentoring undergraduate researchers. The interview included questions such as: What would students be missing out on if discussing and critiquing the primary literature was not a feature of this class? Do you have a certain method for guiding and/or supporting students to participate in the discussion? Dr. B’s answers to these questions validated our analyses of how students participated in the practice of critique and how that participation aligned with Dr. B’s intentions. The second interview occurred after data had been collected from all three cases. The first author and Dr. B listened to a primary literature discussion together. Dr. B paused the recording at moments that were notable and offered his expertise for why this was a notable moment. Dr. B’s comments validated how we distinguished an instance of critique from an instance of other forms of participation. This interview triangulated our analysis with an expert’s perspective of how students participated in primary literature discussions.

RESULTS

The purpose of this study was to qualitatively describe how postsecondary biology students participated in structured primary literature discussions. In all three cases, we observed students participating in the practice of critique. However, the majority of talk turns were categorized as forms of participation other than critique. The percentage of talk turns that were critiques ranged from 23–36% (Table 4). In the following sections, we present our findings on what students attended to when they critiqued primary literature articles during group discussions and the forms of participation other than critique that emerged.

Ways of participating in the practice of critique

There were similarities and also unique aspects to how students participated in critique across the three contexts. Table 5 shows the relative frequencies of stylistic and epistemic critiques for each case and what students attended to when they made stylistic critiques and epistemic critiques. Table 5 also summarizes how students attended to strengths or weaknesses when they made stylistic or epistemic critiques.

The 200-level Ecology class.

When students made critiques during primary literature discussions in the Ecology case, they made both stylistic critiques and epistemic critiques in similar frequencies (Table 5). Overall, critiques in the Ecology case tended to highlight strengths as opposed to weaknesses (Table 5). The stylistic critiques in the Ecology case focused on the data tables/visuals and the writing style/organization of the article. Students’ critiques attended to criteria such as conciseness, organization, and clarity. For example, the quote below is a stylistic critique that highlights strengths of the paper’s clarity and structure and also a weakness of the section headings:

The epistemic critiques in the Ecology case focused on the methods (e.g., sampling design), the scientific argument (e.g., recognizing strong claims supported by evidence), and the research rationale (e.g., justifying that a paper served as a foundation for understanding effects on an ecosystem; Table 5). For example, this quote shows a student attending to a methodological strength of using an appropriate random sampling strategy:

Journal Club.

In Journal Club, students made both stylistic critiques and epistemic critiques, but epistemic critiques were slightly more common (Table 5). Overall, critiques in the Journal Club case tended to highlight weaknesses as opposed to strengths (Table 5). Similar to the Ecology case, stylistic critiques in Journal Club focused on the effectiveness of the data tables/visuals and the writing style/organization of the article (Table 5). In Journal Club, the epistemic critiques focused on the methods, the data tables and visuals, and the scientific argument (Table 5). For example, this student makes an epistemic critique of methodological strengths. Notably, the student also mentions that this specific feature helped them with their own independent research design:

The 400/500-level Organismal Biology class.

In Organismal Biology, the majority of critiques were epistemic critiques and critiques highlighted strengths and weaknesses with similar frequencies (Table 5). Similar to the Ecology and Journal Club cases, the few stylistic critiques focused on the data tables/visuals and the writing style/organization of the article (Table 5). The epistemic critiques focused predominantly on the methods (Table 5). This quote illustrates an epistemic critique of a strength in the methods. The student also explains why they thought the large sample size was important because the purpose of this study was to get at a “large scale” and to do that they needed to look at a lot of sites:

In summary, we saw students in all three cases participate in critique of the primary literature in various ways. Across the three cases, stylistic critiques were aimed at the data tables, visuals, writing style, and article organization. In the Ecology case students critiqued both epistemic and stylistic strengths in relatively similar frequencies, with epistemic critiques focused on the research rationale, the methods, and the argument (Table 5). In Journal Club, the students’ critiques leaned more towards epistemic than stylistic critiques, and most critiques highlighted weaknesses. The epistemic critiques focused on weaknesses of the methods, argument, data tables, and visuals (Table 5). In Organismal Biology, students made epistemic critiques the majority of the time that featured both strengths and weaknesses of the methods (Table 5).

Other ways of participating in primary literature discussions

As shown in Table 4, students in all three cases predominantly participated in primary literature discussions in ways other than making critiques. Table 6 shows the relative frequencies of summarizing, making connections, and sensemaking for each case. Although summarizing was observed in all three cases, it was relatively infrequent compared with making connections and sensemaking. Summarizing often occurred when students highlighted key points of the article to start off the discussion. Additionally, students summarized an aspect of the article to focus attention on it before they continued to critique, connect, or sense make around that aspect of the article. Table 6 shows that the relative frequencies of making connections and sensemaking were different in each case. Making connections was the most common way of participating in the Ecology case and sensemaking was the most common way of participating in Organismal Biology. In Journal Club, students participated with similar frequencies in making connections and in sensemaking. An unanticipated form of sensemaking emerged in all three cases (Table 6), which we refer to as thought experiments. Also, how students made connections was unique to each case. We elaborate on these two findings in the following sections.

Thought experiments emerged as an important form of sensemaking in all cases

In all three cases, more than a third of the talk turns focused on sensemaking and thought experiments emerged as an important form of sensemaking (Table 6). However, sensemaking was relatively less common in the Ecology case and most common in Organismal Biology. Sensemaking was typically instigated when a student volunteered an aspect that confused them (e.g., what a word meant, the type of analysis). For example, one exchange in the Ecology case focused on understanding the predator-prey interactions of the study organism. Another example from Journal Club focused on figuring out how the variable “bark furrow” was measured. A last example from the Organismal Biology case focused on making sense of a data analysis method and how it accounted for nonindependence. These ways of participating are closely aligned with Dr. B’s guidelines: if there is something you really want to understand/explore (like particular methods), you can use others' knowledge to help you get there (Figure 1). Sensemaking was also one way that students in all three cases shaped participation in primary literature discussions to figure out and deepen their understanding of the science ideas they wanted to figure out.

Across all three cases, we also observed an unexpected form of sensemaking that we refer to as a thought experiment. Thought experiments were characterized by posing a hypothetical situation for discussion (e.g., what would happen if…). For example, a student might predict what would happen if the researchers manipulated a different factor in the study. During the instructor validation interview, Dr. B also recognized the thought experiment as an important way of participating in primary literature discussions. Thought experiments accounted for 12–17% of the instances of sensemaking we observed (Table 6). Below we provide examples to illustrate how thought experiments emerged across the cases.

In the following example from the Organismal Biology case, the excerpt starts with general sensemaking to figure out a confusing aspect of the paper (how ants influence cation content) and then shifts to a thought experiment (thought experiment is underlined):

In this example, Student 3 poses a thought experiment (underlined) about how cation content could inform how fertilizers are influencing the larger community and asks their peers how they think one could actually do this. This exemplifies a thought experiment because Student 3 introduces a hypothetical situation for her and her peers to think about and discuss.

Here is an example of a thought experiment from the Ecology case:

In this example, the student begins by summarizing some details about the paper. However, the student continues by highlighting what they did not see in the paper and wondering how birds and other animals might also interact with the ants.

Lastly in Journal Club, thought experiments were often characterized by students putting themselves in the place of the researcher and thinking “I wonder how….” Here is an example:

In this excerpt, Student 1 highlights that sagebrush steppe ecosystems are nutrient-dense areas and wonders if this quality makes them susceptible to invasive species. Student 1 continues pondering how the West has both invasive species and wildfires at play. The conversation remains on this subject, with some connecting to biological concepts and summarizing content in the paper, before Student 1 poses a thought experiment (underlined) by imagining himself as the researcher wondering how early in the growing season one would have to get out to the “plots” to ensure that sagebrush outcompeted invasive species in the area.

In summary, we view thought experiments as important ways of participating in primary literature discussions because they are a marker of productive disciplinary engagement when students expand the discussion to hypothetical situations that reflect their interests, curiosities, and goals for engaging with the article. When students think beyond the bounds of the article and use what they know about other scientific concepts, theories, hypotheses, and laws it reflects their productive engagement in making sense of science as practice.

Ways of connecting that emerged as important forms of participation

Based on the primary literature discussion guidelines, we expected making connections would be an important way of participating. We were interested in how students would put Dr. B’s guidelines into practice and how they would make connections. The ways of making connections that emerged in our analysis are described in Table 7.

Across all three cases, 29–53% of talk turns in each case were making connections (Table 6). However, making connections was relatively more common in the Ecology case and less common in the Organismal Biology case. Students connected topics in the article to previous or new ideas in distinct ways across cases (Table 8), which are illustrated by the examples below.

Ecology students connected the articles to the broader issues of science and society such as anthropogenic effects on the environment, climate change, and scientific literacy. Ecology students focused less on specific details of the assigned article and focused more on how to minimize human impact on the environment. For example, this student connects invasive species to broad issues of historical human impacts on ecosystems:

Ecology students commonly connected content in the articles to concerns about scientific literacy. The dialog below shows students brainstorming strategies to educate the public about climate change:

In this excerpt, Students 1 and 2 brainstorm ideas to connect topics they read about in the article to action items related to science and society. Then, Student 3 connects more specifically to their own personal relevance, illustrating another common way of participating in the Ecology case (Table 8).

Connecting to individual research was a defining characteristic of the Journal Club case, as illustrated by the following excerpt:

This excerpt shows a common way connecting happened in Journal Club. One student asked the discussion leader how the article applied to or would help them with their own research. Notably in Journal Club, students also connected papers to another students’ research in the group (e.g., I wish he [student] was here because that kind of goes with his research). Connecting to one’s own individual research was an expected form of participation based on the context of the Journal Club. However, suggesting connections to peers’ research was an unexpected form of participation.

Students in Organismal Biology most often connected details of the primary literature to biological concepts. Below is an example:

In this example, Student 1 elicits ideas from their peers by asking how dragonflies’ sense that there are fish in the ponds. Student 1 and Student 2 then make connections to biological concepts such as the parent-offspring relationship, instincts, and ways of sensing the environment to explore this question.

Making connections was an important way that students participated in primary literature discussions in all three cases, but how students participated in making connections differed (Table 8). Ecology students participated by making connecting to issues of science and society, Journal Club students connected most to their individual research, and Organismal Biology students made connections to biological concepts and future research.

Summary of patterns of participation in primary literature discussions across cases

Table 9 summarizes the qualitative patterns of participation across the three cases. In all three cases, students participated in critique, but the majority of talk turns represented other ways of participating in primary literature discussions.

When students did participate in critique, there were similarities and differences across the three cases. In all cases, students made both epistemic and stylistic critiques, but the relative frequency of stylistic critiques within a case decreased in cases with more upper-division students compared with cases with more lower-division students. In all cases, students’ stylistic critiques were focused on the data tables/visuals and the writing style/article organization. However, the substance of students’ epistemic critiques differed across cases. Lower-division Ecology students evaluated big-picture aspects like the rationale, scientific argument, and some methodological approaches. In the Journal Club case, where students selected articles related to their independent research projects, students focused on critiquing the methods, data visualizations, and arguments. Upper-division Organismal Biology students focused largely on critiquing the methods of the articles. There were also differences in how students framed their critiques across cases. Lower-division Ecology students highlighted strengths more often than weaknesses, while upper-division Organismal Biology students critiqued both strengths and weaknesses in similar frequencies. In the Journal Club, where students selected articles that might be informative to shaping the methods and presentation of their independent research, the critiques more frequently highlighted weaknesses of the articles.

Most of the talk turns in all three cases represented ways of participating in primary literature discussions other than critiques, but these other ways of participating were unique to each case. In the Ecology case students largely participated by making connections. In Journal Club, students made connections and also participated through sensemaking. In Organismal Biology, students spent much of the discussions participating through sensemaking. Thought experiments emerged as a specific form of sensemaking in all three cases. Participating by making connections was particularly nuanced across the three cases. The relative frequency with which students participated by making connections decreased in cases with more upper-division students compared with cases with more lower-division students (Table 6) and how students made connections was different in each case. In the lower-division Ecology context, students connected aspects of the articles to concerns with science and society, to biological concepts they learned in the course, and to topics that were personally relevant. When students made connections in the Journal Club that was a component of a summer research experience, those connections were to their individual research and biological concepts. Finally, in the upper-division Organismal Biology context, students made connections to biological concepts and ideas for future research.

DISCUSSION

This paper reports on how students participated in the critique and discussion of primary scientific literature in three cases that represented different cross-sections along the biology program of study. In what follows, we first discuss how particular features of these cases help us understand the patterns of participation that emerged in our multi-case study. We end with a discussion of the implications of this research for educators.

Substance and framing of critiques reflected the context of the case

Students participated in primary literature discussions by posing critiques—both epistemic and stylistic—in all three cases, although less frequently than other forms of participation. There are many instructional goals for having students read and discuss articles that are described in the literature (e.g., Edwards et al., 2001; Gillen, 2006; Zagallo et al., 2016; Wood, 2020) and that were communicated to students in this study through the primary literature discussion guidelines (Figure 1). Thus, we find it promising that students engaged in the scientific practice of critique, which is just one of many goals, when afforded the opportunity in these instructional contexts. Furthermore, the context where we observed the highest relative frequency of critique was the Journal Club context, where students selected articles related to their independent research projects. Journal Club was not a formal course so the goals for students to learn course content by reading primary scientific literature were not prominent and goals for evaluating aspects of the articles that could be useful for their independent research projects were more prominent in this context. Previous research has described the importance of creating opportunities for students to take up science practices in ways that are authentic to the discipline and also in ways that are authentic to themselves and their own purposes (Levrini et al., 2015; Kapon et al., 2018). In Journal Club, students’ participation in critique may have been shaped by recognizing how critiquing the articles informed their own research. For instance, Journal Club participants often critiqued data visualizations which were also a central component of their final presentations at the end of the summer research program.

The patterns of critique, in terms of what students attended to and how they framed critiques, that we observed in the three cases may reflect the increased expertise and experience of students in the upper-division cases. The topics of the articles selected for the Journal Club and for the Organismal Biology course were more narrow, specialized, and connected to students’ interests based on the elective nature of the summer research experience and the upper-division Organismal Biology elective. The specialized nature of the topics may have allowed students to practice performing critique on a more closely related body of literature each week. This repeated practice could help explain why we observed a greater frequency of epistemic critiques focused on specific methods and data visualizations and a greater frequency of critiques that highlighted weaknesses in the Journal Club and Organismal Biology contexts. In comparison, the Ecology context was a lower-division general survey of ecology and it is a required course for all majors in the biological sciences, who represent diverse interests and career goals. As we have emphasized throughout this article, ways of participating in science practices like critique cannot easily be bundled into a “how-to” guide for students (Gray, 2014; Manz, 2015; Bolger et al., 2021; Stroupe, 2023). The appropriateness of the methods and interpretations for each scientific investigation must be judged based on the research question and study context (Ford, 2008a; Ford, 2008b). Thus, learning what and how to critique about a particular type of study may depend on a combination of both broad exposure to diverse studies and specialized focus within a particular line of scholarly questioning. The weekly articles in the general Ecology course may have allowed lower-division students to practice critiquing many types of studies and the weekly articles in the Journal Club and the Organismal Biology course may have allowed students to specialize and practice critique with a more related body of literature.

Additionally, much of the literature examining students’ participation in the critique of ideas has been conducted in contexts where students also participate in the construction of ideas (Sampson et al., 2011; Forman & Ford, 2014; Walker et al., 2016; Jones et al., 2017; Osborne, 2010; González-Howard & McNeill, 2020). Scientific knowledge is generated through the interplay between both idea construction and idea critique (Ford, 2008a). Forman & Ford (2014) describe how students take up disciplinary authority and accountability—both essential components of Engle & Contant’s (2002) productive disciplinary engagement—by anticipating the critiques one may receive about the ideas that they constructed. Therefore, it is important to consider how the participation in idea critique that we observed in our study—where the ideas in the articles were constructed by outside researchers and had already passed through peer review for publication—could be related to students’ prior experience with idea construction. It is likely that the students who were at an earlier stage of their biology degree program and enrolled in the lower-division Ecology course had less experience with idea construction than the students participating in the Journal Club and the students enrolled in the upper-division Organismal Biology course. The Journal Club students were embedded in a summer research experience where they engaged in science as practice by developing and carrying out an independent research experience with their faculty mentors. Aspects of the structured research experience outside of the journal club may have shaped how students participated in critique because structured research experiences have been shown to positively shape student motivation, sense of belonging, and STEM identity (Hunter et al., 2010; Thiry and Laursen, 2011; Brownell et al., 2012; Jordan et al., 2014; Rodríguez Amaya et al., 2018). In the case of Organismal Biology, students were further along in their postsecondary degree program, the graduate students in the course were conducting graduate research, and many of the undergraduate students in this course also had research experience related to insects. Additionally, the laboratory component of Organismal Biology provided all students with opportunities to both construct and critique ideas during field-based and lab-based argument-driven inquiry investigations (Sampson, 2011; Brooks et al., 2023). These investigations engaged students in fieldwork to collect and identify insects and lab work to develop and justify taxonomic keys to describe their collections. Thus, students in the Journal Club and Organismal Biology cases likely had more experience with constructing ideas than in the Ecology case, which could have shaped how they participated in critique during primary literature discussions.

We have just described how epistemic critiques that attend to both weaknesses and strengths may be associated with the increased experience and expertise of the students in the Journal Club and Organismal Biology contexts. However, to be clear, we are not discounting stylistic critiques or critiques that focus on strengths as less productive ways of participating in critique. In our study, students’ stylistic critiques attended to strengths and weaknesses of how the data tables/visuals and writing style/article organization were successful (or not) at communicating science to the reader. Such critiques are aligned with the science practice of evaluating and communicating information (NRC, 2012) and the core competencies of applying the process of science and communicating with other disciplines (AAAS, 2011; Clemmons et al., 2020). Furthermore, the prevalence of critiques that highlight strengths in the Ecology context is notable because the practice of critique can often be interpreted with a negative framing related to searching for errors. In his previous classes, Dr. B noticed that students focused mostly on weaknesses. In response, Dr. B modified the primary literature discussion guidelines to signal highlighting strengths as a productive way of participating in critique. In the Ecology case, students appeared to attend to that feature of the guidelines and participated in critique by noting when their search for errors concluded by identifying strengths. Our point is that students in the Ecology case and students in the other two cases participated differently in the practice of critique, and students with more experience constructing ideas appeared to participate more frequently by posing epistemic critiques that highlighted both weaknesses and strengths.

Students shaped the primary literature discussions within their science classroom communities

The majority of talk turns during the primary literature discussions in this study reflected forms of participation other than critique. Although we anticipated that students would participate by summarizing, sensemaking, and making connections, the students within each case shaped how they participated in discussing primary literature in unexpected ways that were relevant to their science classroom communities. We view students shaping the forms of participation that were taken up and recognized within their community as an indicator of productive disciplinary engagement (Engle & Conant, 2002). Furthermore, attending to what is relevant to the disciplinary and classroom community is central to discussions of disciplinary authenticity (Kapon et al., 2018).

We found that students shaped primary literature discussions in a few different ways in this study. First, students participated in sensemaking in all three cases, but it was more common in the Journal Club and Organismal Biology cases. Similar to patterns in the practice of critique, this finding could be explained by differences in experience and expertise of these contexts which prepared them to draw on their peers to make sense of ideas in the articles. However, thought experiments emerged as an unexpected form of sensemaking in all three cases, which indicates that students across cases felt they could participate via thought experiments regardless of whether they were in a lower- or upper-division context. The work by Watkins et al. (2018) on public displays of uncertainty—positioning as not-understanding—may be useful for understanding why sensemaking through thought experiments was an approachable form of participation that emerged in all three cases. When students pose thought experiments of What would happen if… or I wonder how… they open up space for low-stakes sensemaking, where brainstorming many ideas becomes a more celebrated outcome for the group than arriving at one correct answer.

In contrast to how the discussions in Journal Club and Organismal Biology centered on sensemaking, students in the Ecology case shaped their discussions by making connections. The ways of participating by making connections that emerged in each case were relevant to the context of each case. All three cases participated by making connections to biological concepts that were relevant to the course or research context of the case. Although students in the lower-division Ecology course may have less prior knowledge from their coursework to make sense of ideas in the articles compared with Journal Club and Organismal Biology, they do have relevant prior knowledge from their experiences to make connections with ideas in the articles. Making connections between the articles and relevant issues for science and society as well as connecting to aspects of personal relevance became important ways of participating in the Ecology case. Additionally, Ecology students’ connections to issues of science and society might have been shaped by the focus of the course on topics such as climate change, anthropogenic effects on the environment, and scientific literacy.

We aimed to capture how students shaped their participation in primary literature discussions in unexpected ways in each case so that educators and researchers can learn to attune themselves to recognize varied forms of participation as productive disciplinary engagement. Such attunement is important because, when students participate in unexpected ways, an initial reaction may be to dismiss or misinterpret such participation (Bang et al., 2017). For example, connecting to science and society or to personal relevance were important ways of participating in the Ecology case. However, this type of connecting could be interpreted as off-topic or as an attempt to avoid deep analysis of the content of the article. Similarly, it is possible that a student’s thought experiment could be misinterpreted as a distraction from the goals of discussing primary literature. Yet, we put forward that each way of participating—via critique or otherwise—that we described in this study is a form of productive disciplinary engagement in primary literature discussions that is aligned with goals described in the literature and, perhaps more importantly, aligned with Dr. B’s goals. As we learn to attune ourselves to diverse repertoires of participation in science practices, we suggest returning to Engle and Conant’s (2002) productive disciplinary engagement component of accountability. Rather than dismissing unexpected forms of participation, invite students to explain more about how making that connection or posing that thought experiment helps them to make meaning of the article during the primary literature discussion.

Limitations and future research

There are limitations to the findings we reported in this study. First, this multi-case study explored three contexts at one institution, limiting the generalizability of the results. Our analysis focused only on the primary literature discussions and cannot speak to how students participated in critique through writing or what their perceptions were about their participation in these discussions. Additionally, our descriptive, multiple case study describes what happened in these cases and suggests factors that may be important to understand what we observed, but it does not describe cause and effect relationships. Also, cases were selected at different cross-sections of education, and we did not follow students across the trajectory of their postsecondary education. The contrasts between the lower-division and upper-division courses are informative for how we might expect students to participate at different points of their postsecondary trajectory, but we did not observe these changes in the same individual over time. The limitations of this study leave room for future research to explore these lingering questions.

CONCLUSION AND IMPLICATIONS

Our study described a variety of ways postsecondary students participate in primary literature discussions. Our findings contribute to expanding what forms of participation are anticipated and recognized by educators, students, and scientists as productive disciplinary engagement in the practice of critique and participation in primary literature discussions. Additionally, our findings suggest that structuring primary literature discussions in ways that both elevate and connect students’ agency and personal relevance is important for fostering varied forms of productive disciplinary engagement within a science classroom community.

This study has implications for how educators can anticipate the ways students may participate in primary literature discussions in their own contexts. The instructional contexts in this study did not use a published approach to teach how to engage with primary scientific literature. However, we think that a critical feature of these contexts was Dr. B’s guidelines document and how it made explicit a variety of productive ways of participating in primary literature discussions. Our study describes how students in the three cases attended to different pieces of guidance and put the guidelines into practice in different ways. Such descriptions can help educators plan goals for participation, anticipate how students may participate based on the context, and also plan how to respond if students are participating in some ways but not others. For example, Dr. B responded when students predominantly critiqued the weaknesses of articles by modifying the guidelines for future semesters and he also brought the point up directly with the class during the semester. This example illustrates how educators can plan goals for diverse forms of participation, monitor how students participate, and then guide students to explore different ways of participating during the semester.

However, with this suggestion, we also urge educators to use caution and reflect on what aspects of student participation they are dissatisfied with and why. Our point should not be taken as a suggestion to constrain or narrow forms of participation. Rather, we are suggesting that educators be mindful of forms of participation that may limit who feels they can participate (Bang et al., 2017; Stroupe, 2023). In the previous example, Dr. B was dissatisfied that the practice of critique was taken up in a narrow form of “trashing” the paper. This narrow way of participating that dominated the science classroom community could limit who feels they can participate in the practice of critique when critique was portrayed and recognized as only highlighting the weaknesses of the paper. By positioning students with the authority of epistemic agents and celebrating a variety of ways of participating in critique, educators can create space for students to shape the science practice of their classroom communities.

ACKNOWLEDGMENTS

REFERENCES