Investigating Student Perceptions of Instructor Talk: Alignment with Researchers’ Categorizations and Analysis of Remembered Language

INTRODUCTION

To what extent do instructors use language unrelated to course concepts during class time? How might this noncontent language set the tone of the classroom learning environment? And how do students perceive and interpret noncontent language used by instructors? Increasingly, the language used by instructors in undergraduate science courses is considered key to the student experience and to student learning. Accordingly, such language could be the explanatory variable shaping student outcomes of active-learning experiences and driving variations in studies on classroom teaching. This variation in student outcomes may be observed not only across courses and instructors but also with the same instructor over time with additional pedagogical training. While some researchers have described the nature of instructor language related to science content and conceptual learning (Coley and Tanner, 2015; Gouvea and Simon, 2018; Betz et al., 2019; Kranzfelder et al., 2019; Brazeal et al., 2021), others have investigated instructors’ noncontent language, referred to as Instructor Talk (Seidel et al., 2015; Harrison et al., 2019; Lane et al., 2021). Instructor Talk—noncontent language spoken by instructors and directed to the whole class—has been identified in every biology course studied to date. Previous research has quantified the frequency of Instructor Talk instances in each course session across a semester of introductory biology lectures (Seidel et al., 2015), in lecture courses at community colleges and primarily undergraduate institutions (Harrison et al., 2019), online (Seah et al., 2021), at large research universities (Brazeal et al., 2021; Lane et al., 2021), and even in laboratory classes led by graduate student teaching assistants.

Instructor Talk was purposefully characterized in a single undergraduate biology classroom taught by a pair of biology instructors trained in inclusive pedagogical practices (Seidel et al., 2015). Analysis from this initial investigation of noncontent instructor language revealed multiple categories of Instructor Talk that were encouraging and positive, resulting in the initial Instructor Talk framework (Seidel et al., 2015; see Table 1). Subsequent studies investigated 61 courses of 53 instructors from 14 community colleges and an urban university, which produced a sampling method for quantifying Instructor Talk in a large number of courses (Harrison et al., 2019). Instructor Talk instances in the first 15 minutes of class were a representative and enriched sample of Instructor Talk instances per hour. As a result of this sampling strategy, new categories of Instructor Talk emerged that were not codable with the original framework. This newly described Instructor Talk appeared to be discouraging and negative in nature. As a result, Harrison et al. (2019) developed the Negatively Phrased Instructor Talk framework that mirrored the initial one, now referred to as the Positively Phrased Instructor Talk framework (Seidel et al., 2015; see Table 1). Despite the widespread prevalence of Instructor Talk in undergraduate biology classrooms, how students perceive and remember different kinds of Instructor Talk has yet to be studied systematically.

Student perceptions of how instructor language influences their learning environment in science classrooms may not align with instructors’ intentions. For example, instructors might feel like they are keeping the class on track by asking students to hold their questions until the end of class, but students might perceive this as instructors distancing themselves from students. Additionally, instructors could be trying a new active-learning strategy and warning students that it might not go well, which might inadvertently cultivate student resistance to the learning experience. Further, instructors might say something that they perceive as encouraging, such as “This is going to be a tough exam, so study hard.” However, some students could interpret the phrase to be intimidating, especially for students experiencing stereotype threat in the subject area. Given the importance of factoring in students’ interpretations of instructor language, we explore multiple theoretical and phenomenological frameworks relevant to Instructor Talk through the lens of student perceptions: instructor immediacy, student resistance, stereotype threat, and sense of belonging. Consideration of these frameworks suggest that students likely attend to and remember instructors’ noncontent language in college classrooms.

The first theoretical framework that drives interest in the study of student perceptions of Instructor Talk is instructor immediacy, or communication that reduces the perceived psychological distance between instructors and students (Witt et al., 2004). Previous research in communication studies and social psychology has explored how instructor communication consists of verbal and nonverbal immediacy cues. Verbal immediacy cues include using student names, humor, and self-disclosure. Nonverbal immediacy cues include gesticulating, smiling, and making eye contact. Through both verbal and nonverbal cues, instructor immediacy has been shown to have a substantial effect on students’ learning, attitudes, and perceptions of the instructor (Gorham, 1988; Witt et al., 2004). As such, systematic analysis of instructors’ noncontent language may yield insight into how instructors do or do not create instructor immediacy in college classrooms. Specifically, the Instructor Talk frameworks are a tool that could be used to categorize and quantify instructor language unrelated to course content. One could assess the prevalence of verbal immediacy cues with categories of the Positively Phrased Instructor Talk framework, such as Building the Instructor/Student Relationship and Sharing Personal Experiences. Further, one could investigate the impacts of verbal cues that diminish instructor immediacy by also considering parallel categories of the Negatively Phrased Instructor Talk framework: Dismantling the Instructor/Student Relationship and Sharing Personal Judgment. However, there are yet to be empirical studies investigating how undergraduate students perceive examples of such language, which we hypothesize could increase (or decrease) instructor immediacy and positively (or negatively) influence students’ perceptions of instructors. One may wonder to what extent students even remember noncontent instructor language that relates to the instructor–student relationship.

Additionally, a phenomenological framework that suggests the importance of Instructor Talk and student perceptions of this language is student resistance. While there have been international calls to integrate more active-learning strategies into science, technology, engineering, and math (STEM) classrooms (Woodin et al., 2010), faculty concerns about student resistance to novel strategies may discourage instructors from integrating these beneficial practices into their classrooms. Although many instructors are concerned about student resistance to new instructional approaches, students seem more resistant to being patronized by instructors (Kearney et al., 1991; Seidel and Tanner, 2013). As such, students may be more resistant to the instructor’s language than to the new instructional practices themselves. In fact, when instructors explain to students what the components of an activity and its purpose are (i.e., explanation strategies), both students and instructors perceive that student resistance is reduced (Finelli et al., 2018; Tharayil et al., 2018). This aligns with a key component of the Positively Phrased Instructor Talk framework, the Explaining Pedagogical Choices category, in which instructors explain their reasoning behind instructional decisions to their students (Seidel et al., 2015). Indeed, recordings of Instructor Talk in biology courses demonstrate that instructors frequently explain the reasoning behind instructional decisions with their students (Seidel et al., 2015; Harrison et al., 2019; Lane et al., 2021). Taken together, these studies suggest that students may appreciate Instructor Talk that explains the rationales behind instructors’ pedagogical choices, and as a result, such language might minimize resistance and have an overall positive impact on the learning environment. However, we currently have limited, if any, insights into how students perceive instructor language about instructional practices in relation to researchers’ evaluation of this language.

Finally, multiple theoretical frameworks from social psychology, in particular stereotype threat and sense of belonging, suggest that students likely remember and are influenced by Instructor Talk. Stereotype threat is a phenomenon in which capable individuals underperform when they feel at risk of conforming to negative stereotypes affiliated with an aspect of their identities, potentially compromising students’ performance in high-stakes assessment situations (Steele and Aronson, 1995; Steele, 1997). Notably, stereotype threat is hypothesized to disproportionately reduce the academic performance of underserved students in science (Spencer et al., 1999) and influence these students to leave STEM altogether (Beasley and Fischer, 2012). On the other hand, students who experience a high sense of belonging—when individuals feel accepted and supported by a social group, setting, or institution—in science classrooms can have higher academic performance and persistence in STEM disciplines (Stout et al., 2013; Trujillo and Tanner, 2014; Fink et al., 2020). Thus, understanding students’ perceptions of language in science classrooms may be key to mitigating their negative experiences in these classrooms. We hypothesize that Instructor Talk could be used to mitigate stereotype threat by making it clear that all students have a place in science, which may in turn influence their sense of belonging. Such language is represented in two categories of the Positively Phrased Instructor Talk framework: Establishing Class Culture, in which instructors build a biology community among students, and Unmasking Science, in which instructor language promotes diversity in science (Seidel et al., 2015). Further, we predict that science students can remember examples of Instructor Talk used to promote positive learning experiences that mitigate stereotype threat and foster belonging in science classrooms. Finally, we hypothesize that students from historically excluded groups in STEM, such as students of color, first-generation college students, and women, may remember more Instructor Talk that promotes a negative learning environment from their perspective. Such language would be represented in two categories of the Negatively Phrased Instructor Talk framework: Disestablishing Class Culture, in which instructors discourage community among students, and Masking Science, in which instructor language intimidates students from science (Harrison et al., 2019). Despite the potential for noncontent instructor language to influence students’ experience of stereotype threat and sense of belonging, initial studies exploring how students across demographic groups remember and interpret Instructor Talk are still needed.

While Instructor Talk has been well documented in biology classrooms across institutions and classroom types (Seidel et al., 2015; Harrison et al., 2019; Lane et al., 2021), it remains unclear how aligned student perceptions of this language are with researchers’ previous categorizations and to what extent students even remember Instructor Talk. Further, a detailed understanding of student perceptions of Instructor Talk would provide additional validity evidence for the Instructor Talk frameworks. Because Instructor Talk may be a mediating variable for a range of student outcomes, detailed investigations of student perceptions are a key step in characterizing this novel classroom variable, which could inform the interpretation of many studies. To investigate these ideas, we addressed the following research questions: 1) To what extent do students’ perceptions of noncontent language align with researchers’ interpretations as synthesized in the Instructor Talk framework? 2) To what extent do students remember examples of noncontent language used by instructors?

METHODS

This study was determined exempt by the Institutional Review Board at San Francisco State University under protocol number E19-129.

Design and Approach

We designed two studies to address these research aims: 1) study 1 evaluated the extent to which students’ and researchers’ perceptions of Instructor Talk were in alignment (hereafter, student–researcher alignment), and 2) study 2 assessed the proportion of students who remembered Instructor Talk from their biology courses and the nature of that language. Like interviews and focus groups that ask students to recall and self-report their experiences, our approach systematically collected student perceptions of Instructor Talk and their memories of noncontent language on a larger scale with a written survey consisting of multiple activities and assessment prompts. Table 2 summarizes the student activities and assessment prompts for both studies. We structured the assessment prompts for both studies so that students became familiar with examples of Instructor Talk before they were prompted to recall instructor language that they remembered as promoting a positive or a negative learning environment.

Identifying Participants for the Studies

Both study 1 and study 2 were conducted at a public, urban, master’s-granting university on the West Coast. The study site is designated as a Hispanic-serving institution and Asian American and Native American Pacific-Islander–serving institution.

To evaluate both the extent to which biology undergraduates’ perceptions of instructor language are in alignment with researchers’ categorizations and the extent to which they remember noncontent instructor language, we opted for study participants who were upper-division students majoring in biology. During the semester our studies were conducted, the demographics of upper-division biology students included 70% women and 52% underrepresented minority status as defined by the institution, which includes Black, Latinx, and Native American students (San Francisco State University Institutional Research, 2019).

Because they had experience across several biology courses to draw from for the studies, all participants were recruited from the last course in a series of three core courses required for the biology major. Participants completed the assessments for both studies during a single in-person class session. They received participation points toward their course grade for completing the assessment prompts, and students who were not in class that day were excluded from the study. All data were collected toward the end of the semester.

Characterizing Participant Demographics

To represent the biology students participating in these studies, we asked each participant to complete an optional demographic form after finishing the activities for study 1 and study 2. The demographic form collected self-identified data on study participants’ class standing (junior, senior, etc.), community college transfer status, and first-generation college-going status, along with responses to open-ended questions concerning gender identity, sexual identity, and race and ethnic identity.

To evaluate differences in responses across participant characteristics, we disaggregated some of the resulting data by self-identified binary gender, first-generation college status, as well as race and ethnic identity in two ways: 1) persons of color (i.e., non-white students) and 2) persons excluded because of their ethnicity or race (i.e., PEER: participants who self-identified as Latinx/a/o, Black or African-American, and persons of mixed race who included any of these designations; Asai, 2020).

Study 1: Evaluating Student–Researcher Alignment in Their Perceptions of Whether Specific Instructor Talk Quotes Would Promote a Positive or Negative Learning Environment

The two Instructor Talk frameworks—Positively Phrased and Negatively Phrased—were constructed by researchers and to date have not been correlated with student perceptions. To evaluate the extent to which students’ perceptions of Instructor Talk were in alignment with researchers’ perceptions as promoting a positive or a negative learning environment, study participants engaged in two activities for study 1 (Table 2). In the first activity, referred to as a choice activity, students were given an Instructor Talk stimulus set of 20 quotes printed on strips of paper in an envelope. For this choice activity, students were first prompted to choose up to three specific quotes that they perceived as promoting the most positive learning environment and then were prompted to choose up to three specific quotes that they perceived as promoting the most negative learning environment. In the second activity, referred to as an evaluation activity, students evaluated all 20 Instructor Talk quotes in the stimulus set as promoting either a positive or negative learning environment. In the following sections, we describe how we identified instances of Instructor Talk for these two activities and how we evaluated the degree of student–researcher alignment in their perceptions of Instructor Talk using statistical approaches.

Constructing an Instructor Talk Stimulus Set for the Study.

To gauge students’ perceptions of noncontent instructor language, we developed a random, stratified sampling strategy to select Instructor Talk quotes—10 Positively Phrased and 10 Negatively Phrased—from Harrison et al. (2019; see Table 3 for the stimulus set of 20 Instructor Talk quotes). Harrison et al. (2019) sampled and analyzed transcripts of the first 15 minutes of two class sessions in 61 courses for 53 instructors. We randomly selected quotes from this study, because the quotes were recorded from instructors at a range of institutions, including the one in the present study. In the 61 courses recorded in Harrison et al. (2019), a total of 565 instances of Instructor Talk were identified, 545 of which were categorized as Positively Phrased and 55 of which were categorized as Negatively Phrased. We reviewed this database of Instructor Talk instances from Harrison et al. (2019) and excluded instances of Instructor Talk that: 1) included language noted as “unintelligible” in the transcript, 2) contained specialized language (e.g., idioms, jargon, discipline-specific language) that may not be understood by a large variety of students, 3) lacked clarity out of context, 4) were duplicates, 5) were lengthy and contained more than 120 words, and/or 6) were directed at non-biology majors. After these exclusion criteria were applied, 59% of the Positively Phrased (n = 321) and 52% of the Negatively Phrased (n = 29) instances of Instructor Talk were excluded. The Instructor Talk instances that met the inclusion criteria represented all five Positively Phrased categories and all five Negatively Phrased categories of the Instructor Talk frameworks. Further, all 18 subcategories of the Positively Phrased and nine of the 15 subcategories of the Negatively Phrased Instructor Talk frameworks were represented by the remaining instances.

After these selection criteria were applied, we used a random, stratified sampling strategy to ensure that two instances of Instructor Talk were randomly chosen within each of the five categories for both the Positively Phrased and the Negatively Phrased Instructor Talk frameworks (see Table 1). To ensure the Instructor Talk instances presented to students were representative of widespread use in biology classrooms, we selected the first two random samples generated for each category that 1) included at least one sample from the most prevalent subcategory of the Instructor Talk frameworks and 2) would make the most sense for students based on clarity and context. This random stratified sampling strategy resulted in a stimulus set of 20 specific Instructor Talk quotes that were used for the study (n = 10 each for Positively and Negatively Phrased Instructor Talk; see Table 3 for quotes).

Choice Activity: Assessing Student Perceptions of Instructor Talk That Promoted the Most Positive or Most Negative Learning Environment.

Although researchers have described instances of Instructor Talk as Positively Phrased or Negatively Phrased using the existing frameworks, student perceptions of how Instructor Talk impacts the learning environment have not been assessed. Using the random, stratified Instructor Talk stimulus set of 20 quotes described earlier (see Table 3), we asked students to respond to two assessment prompts to explore their perceptions of how Instructor Talk may influence the learning environment in biology classrooms.

First, students were prompted to choose up to three specific Instructor Talk quotes from the Instructor Talk stimulus set that they perceived as promoting the most positive learning environment. Next, students were prompted to choose up to three specific Instructor Talk quotes from the same Instructor Talk stimulus set that they perceived as promoting the most negative learning environment. For both of these prompts, students were invited to provide rationales for their choices of the Instructor Talk quotes.

Based on a pilot study with students, the assessment for study 1 prompts students to select Instructor Talk quotes that they perceive as promoting the most positive or negative learning environment. This language compelled pilot study participants to read all the Instructor Talk quotes more carefully and to evaluate the entire stimulus set for the most impactful quotes, rather than just selecting the first few quotes they encountered that resonated as promoting a positive or negative learning environment.

Evaluation Activity: Assessing Student–Researcher Alignment in Perceptions of Instructor Talk as Promoting a Positive or Negative Learning Environment.

To evaluate student–researcher alignment in their perceptions of Instructor Talk across the entire stimulus set, students were asked to judge each of the 20 Instructor Talk quotes in the stimulus set (Table 3) as promoting a positive learning environment or a negative learning environment, or students could select “neither/unsure.”

Analyses of Student–Researcher Alignment in Their Perceptions of Instructor Talk Promoting a Positive or Negative Learning Environment.

We used a three-pronged approach for data analyses. In the following sections, we describe these approaches in detail. All data analyses and figure production were done in R (R Development Core Team, 2017) using the ggplot and ggtheme packages (Wickham and Wickham, 2009; Arnold, 2017).

Choice Activity Data Analyses: Quantifying Student Choices of Instructor Talk That Promoted the Most Positive or Most Negative Learning Environment.

For the first analysis, we quantified the proportion of students who chose two or three specific Instructor Talk quotes for each prompt in alignment with researchers (i.e., choosing Positively Phrased Instructor Talk quotes as promoting a positive learning environment and Negatively Phrased Instructor Talk quotes as promoting a negative one). For both assessment prompts, we determined 1) the extent to which students chose specific Instructor Talk quotes as promoting the most positive or most negative learning environment and 2) the extent to which students’ choices aligned with how Instructor Talk quotes were previously coded by researchers.

Evaluation Activity Analyses: Statistical Approach for Evaluating Overall Student–Researcher Alignment in Their Perceptions of Instructor Talk.

Additionally, we used an overall summative statistical approach for the evaluation activity. We assigned values of either 1, 0, or −1 to students’ evaluation of each of the 20 Instructor Talk quotes as either positive, unsure, or negative, respectively. For each student, we summed the total values of their evaluations across the 10 Positively Phrased Instructor Talk quotes and then separately summed total values of their evaluations across the 10 Negatively Phrased Instructor Talk quotes. Thus, each student’s values ranged between +10 and −10 for the Positively Phrased quote group and the Negatively Phrased quote group. If students assigned all 10 Positively Phrased Instructor Talk quotes as promoting a positive learning environment, the sum of their assignments would be +10 and in complete alignment with researchers. If students assigned all 10 Negatively Phrased Instructor Talk quotes as promoting a negative learning environment, then the sum of their assignments would be −10 and in complete alignment with researchers. Further, if students were unsure or chose randomly, then the sum value for either the group evaluations of the 10 Positively Phrased quotes or the 10 Negatively Phrased quotes would be expected to approach 0.

This summative approach allowed us to statistically evaluate the extent to which students’ perceptions of Instructor Talk were in alignment with researchers by using two, two-sided Wilcoxon signed-rank tests: one for the 10 Instructor Talk quotes that researchers had previously categorized as Positively Phrased and another for the 10 Instructor Talk quotes that researchers had previously categorized as Negatively Phrased. We opted for this nonparametric statistical approach because the data did not have a Gaussian distribution.

Evaluation Activity Analyses: Quantifying Student Evaluations of Instructor Talk That Promoted a Positive or Negative Learning Environment at the Category and Individual Quote Level.

Finally, to assess student–researcher alignment at the category level of the Instructor Talk framework, we quantified the proportion of students who assigned each of the 20 Instructor Talk quotes as promoting a positive or negative learning environment, or whether students were unsure. Similarly, we also quantified the proportion of student evaluations (positive, negative, or unsure) at the individual quote level for the 20 Instructor Talk quotes.

Study 2: Assessing Student Memories of Instructor Talk

Although Instructor Talk has been recorded and categorized in undergraduate biology classrooms in previous studies (Seidel et al., 2015; Harrison et al., 2019), the extent to which students themselves can remember noncontent instructor language is unknown. Furthermore, the types of noncontent instructor language that students may remember and perceive as promoting a positive or negative learning environment is also not currently known. After students had explored some examples of Instructor Talk (study 1; see Table 3), we invited students to provide up to three examples of language that their biology instructors had used that they felt promoted a positive learning environment. Next, we invited students to provide up to three examples of language their biology instructors had used that they felt promoted a negative learning environment. For each activity, students were asked to provide rationales for why they felt their memories of noncontent instructor language promoted either a positive or negative learning environment. We opted to solicit student memories of instructor language after students evaluated the stimulus set of 20 Instructor Talk quotes to support students’ understanding of the concept of noncontent instructor language and increase the validity of the memory prompt.

Determining Whether Student Memories of Instructor Language Could Be Categorized Using the Instructor Talk Frameworks.

Given that undergraduate students’ memories of noncontent instructor language have not been studied in biology classrooms to date, we first determined whether the existing Instructor Talk frameworks (Table 1) could be used to categorize and analyze student memories of noncontent instructor language. According to the pre-existing criteria defined by Seidel et al. (2015, p. 1), Instructor Talk was considered to be “any language that 1) was spoken by an instructor, 2) was addressed to the class as a whole, 3) was not specific to course content, 4) was not an analogy for course content” and 5) excludes “course logistics or agenda items at the start or end of class.” Based on these pre-existing criteria of Instructor Talk, student memories that described instructor behavior (e.g., “Puts smiley face on my exam”), logistical language (e.g., “It’s not on the syllabus, but I drop your lowest test score.”), or language used outside of the classroom (e.g., over email) were coded as “Not Instructor Talk.” In addition to the pre-existing criteria for Instructor Talk, we also coded student memories written in third person as Not Instructor Talk, because it was not clear whether the student memory of instructor language was a direct quote or paraphrased (e.g., “I had a prof tell us one time that he learns the same amount from us as we do him.”). Therefore, student memories that were written as a direct quote from an instructor and conceivably spoken in a classroom, directed to the class as a whole, and neither logistical nor content metaphors were coded as “student memories of Instructor Talk.”

Analyzing Student Memories of Instructor Talk with the Instructor Talk Frameworks.

Using the Positively Phrased and Negatively Phrased Instructor Talk frameworks, student memories of instructor language were coded independently by each researcher and then discussed as a team (D.O, M.M.R., J.V.L., K.T., and K.D.T.). The composition of our team allowed us to discuss undergraduate student memories informed by the perspectives of undergraduate researchers. To consider and resolve the different perspectives of the research team, we discussed how each researcher had independently categorized the student memories of noncontent instructor language and then collectively agreed upon the final category using the Instructor Talk frameworks (see Table 1). To reach consensus, we developed an approach to discuss student memories of noncontent instructor language. First, each student memory was coded as Instructor Talk or Not Instructor Talk. For student memories of Instructor Talk, we then determined whether the student memory was Positively Phrased or Negatively Phrased Instructor Talk. Third, we coded the student memory of Instructor Talk into one of the five categories of the corresponding framework. Finally, the student memory was assigned to a subcategory within the corresponding category. Occasionally, a student memory could be coded with more than one category/subcategory, and the senior author (K.D.T.) was consulted to reach consensus for these instances. Only a few student memories could be double coded into multiple Instructor Talk subcategories. Given that we only asked students to provide three memories per prompt, we opted to exclude the two splits that emerged during data analysis when calculating the proportion of students who recalled memories of Instructor Talk.

In summary, the established approach for coding each student memory with the Positively Phrased and Negatively Phrased Instructor Talk frameworks could be described with four questions of increasing specificity:

1. Does the student memory meet the criteria for Instructor Talk?

2. Does the student memory of Instructor Talk align with the Positively Phrased or Negatively Phrased Instructor Talk framework?

3. Which category does the student memory align with in the corresponding Instructor Talk framework?

4. Which subcategory does the student memory align with in the corresponding Instructor Talk category?

Data Analyses of Student Memories of Instructor Talk.

After categorizing student memories of Instructor Talk, we conducted multiple quantitative analyses of the resulting codes, as described in the following sections.

Analyses of Student Memories of Instructor Talk Disaggregated by Students’ Self-Identified Characteristics.

First, we quantified the proportion of students who provided at least one memory codable as Positively Phrased Instructor Talk perceived as promoting a positive learning environment and at least one memory codable as Negatively Phrased Instructor Talk perceived as promoting a negative learning environment. Next, to assess whether students of different identity groups could recall instances of Instructor Talk at proportionally higher rates, we disaggregated these results by self-identified characteristics. Statistical comparisons were conducted using Pearson chi-square tests to determine the probability of independence, with Bonferroni corrections to minimize the risk of type I error due to multiple statistical comparisons (level of significance was set at 0.00625). We conducted eight chi-squares total, four for student memories that promoted a positive learning environment and four for student memories that promoted a negative learning environment. The four pairwise comparisons included: gender (women, men), first-generation college-going status (first-gen, non–first gen), PEER (Latinx/a/o, Black or African-American, mixed race; non-PEER), and POC (people of color).

Quantification of the Most Prevalent Instructor Talk Categories for Student Memories of Instructor Language Promoting a Positive or Negative Learning Environment.

We analyzed student memories of Instructor Talk in two ways. First, to ensure students who provided multiple memories from the same category were not overrepresented, we determined the proportion of students who recalled memories of Instructor Talk in each of the categories of the Positively Phrased and Negatively Phrased Instructor Talk frameworks. Second, given that students could provide up to three memories, we also quantified the percentage of student memories coded with the two Instructor Talk frameworks. These two analyses allow us to consider both the proportion of students who can recall certain types of Instructor Talk and the percentage of student memories overall.

Comparing Prevalence of Categories for Student Memories of Instructor Talk with Results from a Previous Study Investigating the Prevalence of Instructor Talk in Undergraduate Biology Classrooms.

To assess whether student memories reflected what was said most frequently in classrooms, we compared our quantification of student memories at the category level with the prevalence of Instructor Talk recorded in 61 undergraduate biology courses (Harrison et al., 2019). This comparison determined whether the categories of student memories of Instructor Talk mirrored the prevalence of previously recorded Instructor Talk in biology classrooms. Notably, the comparison study included Instructor Talk from the university at which we conducted the present study.

RESULTS

We investigated how students evaluated previously recorded Instructor Talk quotes and whether students could remember noncontent instructor language. The results are divided into two sections for the aforementioned studies. In study 1, evaluating student–researcher alignment in perceptions of Instructor Talk, students were prompted to 1) choose up to three Instructor Talk quotes that promoted the most positive learning environment, 2) choose up to three Instructor Talk quotes that promoted the most negative learning environment, and 3) evaluate each of the 20 Instructor Talk quotes as either promoting a positive or negative learning environment (see Table 3 for the Instructor Talk stimulus set). The aim of study 1 was to assess the extent to which there was student–researcher alignment in perceptions of noncontent instructor language. In study 2, student memories of Instructor Talk, students were prompted to 1) recall three memories of noncontent instructor language that promoted a positive learning environment and 2) recall three memories of noncontent instructor language that promoted a negative learning environment. The aim of study 2 was to quantify the type of Instructor Talk, if any, that students could remember as influencing their learning environments.

Demographics of Participants in Study 1 and Study 2

A total of 90 undergraduates participated in both studies out of 146 students who were enrolled in the upper-division biology course from which we recruited study participants. The exact participation rate is unknown, as attendance was not recorded by the instructor for the day the assessment was given. As shown in Table 4, most participants self-identified as women (70%, n = 62) and were first-generation college-going (56%, n = 50), while a smaller proportion were transfer students (26%, n = 23). Participants who self-identified as Latinx/a/o, Black or African-American, or persons of mixed race who included any of these designations were designated as PEER (40%, n = 36). POC included all non-white students (76%, n = 68). The percentage of students by self-identified ethnicity can be found in Supplemental Table S1. Based on institutional data described in the Methods, the demographics of participants were representative of upper-division biology students at the time the studies were conducted.

Study 1: Evaluating Student–Researcher Alignment in Perceptions of Instructor Talk

To evaluate the degree of student–researcher alignment in their perceptions of whether Instructor Talk promoted a positive learning environment or a negative learning environment, we invited study participants to engage with two activities. The first activity was a choice activity in which students reviewed 20 Instructor Talk quotes and then chose up to three quotes that they felt promoted the most positive learning environment. Students then chose up to three quotes that they felt promoted the most negative learning environment from the same stimulus set. The second activity was an evaluation activity in which students evaluated each of the 20 Instructor Talk quotes in the stimulus set as promoting a positive learning environment or a negative learning environment or said that they were unsure.

Choice Activity: Assessing Student Perceptions of Instructor Talk That Promoted the Most Positive Learning Environment.

When prompted to choose up to three Instructor Talk quotes perceived as promoting the most positive learning environment, 99% students (n = 83/84) chose at least two quotes that researchers had also coded as Positively Phrased Instructor Talk and thus were in high alignment with researchers. Nearly 64% of students (n = 54/84) were fully aligned with researchers in their perceptions of the Instructor Talk quotes, choosing all three quotes that they perceived as promoting the most positive learning environment that researchers had also previously coded as Positively Phrased Instructor Talk. The most misalignment could be attributed to students choosing the Negatively Phrased Instructor Talk stimuli quotes within either the Focusing on the Grade/Short Term (10%, n = 8/84) and Expecting Students to Know What to Do (10%, n = 8/84) categories as promoting a positive learning environment.

The proportion of students who chose each Instructor Talk quote from the stimulus set that they perceived as promoting the most positive learning environment is detailed in Table 5. All 10 Instructor Talk quotes previously coded by researchers as Positively Phrased were chosen in this activity by at least two students as promoting the most positive learning environment. Of the 20 Instructor Talk quotes provided in the stimulus set, three specific quotes were chosen by more than 40% of students as promoting the most positive learning environment. Students most often chose the quote within the Building the Instructor/Student Relationship category (44% students, n = 40/90), followed by quotes within the Unmasking Science (42% students, n = 38/90) and Establishing Classroom Culture (41% students, n = 37/90) categories. Only one participant chose one of these three quotes as promoting the most negative learning environment. Overall, only 18% of students (n = 16/90) chose an Instructor Talk quote that they perceived as promoting the most positive learning environment that researchers coded as Negatively Phrased Instructor Talk (see Table 5).

^aQuotes are sorted into Positively Phrased and Negatively Phrased Instructor Talk and presented in descending order, based on the proportion of students who chose that particular Instructor Talk quote. Blue font (top % and number) is the proportion of students who chose the Instructor Talk quote as promoting the most positive learning environment. Red font (bottom % and number) is the proportion of students who chose the Instructor Talk quote as promoting the most negative learning environment. Percentages sum to greater than 100%, because students could select up to three quotes.

Choice Activity: Assessing Student Perceptions of Instructor Talk That Promoted the Most Negative Learning Environment.

When prompted to choose up to three Instructor Talk quotes from the stimulus set that they perceived as promoting the most negative learning environment, 96% of students (n = 81/84) chose at least two quotes that researchers also coded as Negatively Phrased Instructor Talk and thus were in high alignment with researchers. Only 45% of students (n = 38/84) were in full alignment with researchers and chose all three Instructor Talk quotes as promoting the most negative learning environment, which were previously categorized as Negatively Phrased Instructor Talk by researchers. It is worth noting that 38% of students (n = 32/84) chose the Positively Phrased Instructor Talk stimuli quote within the Pre-framing Classroom Activities category as promoting a negative learning environment.

The proportion of students who chose each Instructor Talk quote from the stimulus set that they perceived as promoting the most negative learning environment is described in Table 6. All 10 Instructor Talk quotes previously coded by researchers as Negatively Phrased were chosen in this activity by at least five students as promoting the most negative learning environment. Of the 20 Instructor Talk quotes provided in the stimulus set, two specific quotes were chosen by nearly half of students as promoting the most negative learning environment. Within the 10 Negatively Phrased Instructor Talk quotes, students chose the quote within the Compromising Pedagogical Choices category most often (53% students, n = 48/90), while nearly half of students (48%, n = 43/90) chose the Instructor Talk quote within the Dismantling the Instructor/Student Relationship category as promoting the most negative learning environment. Only one participant chose one of these two quotes as promoting the most positive learning environment. Over a third of students (36%, n = 32/90) were misaligned with researchers and chose a quote within the Positively Phrased Instructor Talk framework in the Establishing Classroom Culture category as promoting the most negative learning environment. Overall, two quotes coded by researchers as Positively Phrased Instructor Talk were chosen by some students as promoting the most negative learning environment (see Table 6).

^aQuotes are sorted into Negatively Phrased and Positively Phrased Instructor Talk and presented in descending order, based on the proportion of students who chose that particular Instructor Talk quote. Blue font (bottom % and number) is the proportion of students who chose the Instructor Talk quote as promoting the most positive learning environment. Red font (top % and number) is the proportion of students who chose the Instructor Talk quote as promoting the most negative learning environment. Percentages sum to greater than 100%, because students could select up to three quotes.

Evaluation Activity: Assessing Student–Researcher Alignment in Perceptions of Instructor Talk That Promoted a Positive or Negative Learning Environment.

When prompted to categorize each of the 20 Instructor Talk quotes in the stimulus set as promoting a positive or negative learning environment, students’ evaluations of Instructor Talk quotes were in alignment with researchers (two-sided Wilcoxon signed-rank test; p < 0.0001 for both tests; Figure 1). The median of student evaluations of the 10 Positively Phrased Instructor Talk quotes was 5.5 (95% confidence interval = 5.0, 6.0), while the median of student evaluations of the 10 Negatively Phrased Instructor Talk quotes was −4.5 (95% confidence interval = −5.0, −4.0). It is worth noting that only 4% of students (n = 4/90) evaluated all 10 Positively Phrased quotes as promoting a positive learning environment, and 1% of students (n = 1/90) evaluated all 10 Negatively Phrased quotes as promoting a negative learning environment. To ensure there were no discrepancies across demographics, the summative analysis for the evaluation activity was disaggregated by self-identified gender (women and men) and race (POC and non-POC; see Supplemental Figure S1).

Category-Level Analysis: Instructor Talk Categories That Promoted a Positive Learning Environment.

There was generally high agreement between students and researchers across the five categories of the Positively Phrased Instructor Talk framework (Figure 2A). On average, students agreed with researchers 70% ± 8% (mean ± SE) of the time across the five categories. Yet only 17% ± 7% of students evaluated Positively Phrased Instructor Talk as promoting a negative learning environment. Student–researcher alignment varied across the five categories of the Positively Phrased Instructor Talk framework (Figure 2A). Alignment was the strongest for the Unmasking Science category (91% agreement, n = 162/178). The student–researcher alignment in the remaining four categories of the Positively Phrased Instructor Talk framework are as follows, in order of decreasing prevalence: Building the Instructor/Student Relationship (79% agreement, n = 141/179), Explaining Pedagogical Choices (73% agreement, n = 130/177), Sharing Personal Experiences (53% agreement, n = 94/178), and Establishing Class Culture (50% agreement, n = 88/177). Nearly 30% (n = 56/178) of students were unsure whether the two quotes for Sharing Personal Experiences category promoted a positive or negative learning environment.

Category-Level Analysis: Instructor Talk Categories That Promoted a Negative Learning Environment.

Overall, student agreement with researchers was relatively high across the five categories of the Negatively Phrased Instructor Talk framework (Figure 2B). On average, 61% ± 3% of students evaluated the Negatively Phrased Instructor Talk as promoting a negative learning environment. Only 19% ± 2% of students on average evaluated Negatively Phrased Instructor Talk as promoting a positive learning environment. Alignment between students and researchers was highest for the Dismantling the Instructor/Student Relationship and Disestablishing Class Culture categories (66% of agreement for both, n = 119/179 and n = 116/175, respectively). Student–researcher alignment was 60% for the Compromising Pedagogical Choices (n = 108/178) and Sharing Personal Judgment (n = 107/179) categories. The Masking Science category had the least student–researcher alignment at 51% agreement (n = 91/179). For each of the categories, approximately 20% of students on average were unsure whether the Instructor Talk quotes promoted a positive or negative learning environment.

Individual Quote Analysis: Instructor Talk Quotes That Promoted a Positive Learning Environment.

Because the two different example quotes in some categories were perceived quite differently from one another by students, we also analyzed overall student evaluations for each of the 20 individual Instructor Talk quotes in the stimulus set (Figure 3A). All but one quote had more than 75% of student–researcher agreement as indicated by the dotted line in Figure 3A. Alignment was highest for the Positively Phrased Instructor Talk quote for the Fostering Wonder in Science subcategory (+US2; 96% agreement, n = 85/96). Following this quote, the other individual Instructor Talk quotes with the highest alignment, in decreasing order of alignment, were: Demonstrating Respect for Students (+BISR1; 88% agreement, n = 79/90), Building a Biology Community among Students (+ECC2; 88% agreement, n = 78/89), Discussing How People Learn (+EPC2; 74% agreement, n = 66/89), Connecting Biology to the Real World and Career (+EPC1; 73% agreement, n = 64/88), Boosting Self-Efficacy (+BSIR2; 70% agreement, n = 64/88), Relating to Student Experiences (+SPE2; 53% agreement, n = 47/89), and Recounting Personal Information or Anecdotes (+SPE1; 53% agreement, n = 47/89). The subcategory with the lowest student–researcher alignment was Pre-framing Classroom Activities (+ECC1; 11% agreement, n = 10/88). Examples of student rationales for evaluating Positively Phrased Instructor Talk quotes in alignment and misalignment with researchers can be found in Supplemental Table S2.

Individual Quote Analysis: Instructor Talk Quotes That Promoted a Negative Learning Environment.

We also evaluated overall student evaluations for each of the 20 individual Instructor Talk quotes in the stimulus set (Figure 3B). There were only three individual Instructor Talk quotes where less than 75% of students agreed with researcher categorizations as indicated by the dotted line in Figure 3B. Student–researcher alignment was highest for the Instructor Talk quote for the Using Convenience to Drive Teaching Choices subcategory (−CPC1; 94%, n = 85/90). This was the only Negatively Phrased Instructor Talk quote that students did not categorize as promoting a positive learning environment. Following this individual quote, the other Instructor Talk quotes with the highest alignment, in decreasing order of alignment, were: Making Public Judgments about Students (−DISR2; 79% agreement, n = 71/90), Expecting Students to Know What to Do (−DCC1; 72% agreement, n = 64/88), Sharing Self-Judgment/Self-Pity (−SPJ2; 71% agreement, n = 63/89), Expecting Students to Know What to Do (−DCC2; 60% agreement, n = 52/87), Ignoring Student Challenges (−DISR1; 54% agreement, n = 48/89), and Sharing Self-Judgment/Self-Pity (−SPJ1; 49% agreement, n = 44/90). The Instructor Talk categorized by researchers within the “Focusing on the Grade/Short Term” subcategory had the lowest student–researcher agreement (-CPC2; 26% agreement, n = 23/88). Examples of student rationales for evaluating Negatively Phrased Instructor Talk quotes in alignment and misalignment with researchers can be found in Supplemental Table S3.

Study 2: Assessing Student Memories of Instructor Talk

To investigate the extent to which students could recall noncontent instructor language that 1) promoted a positive learning environment or 2) promoted a negative learning environment, we coded and quantified student responses to each of these prompts (n = 216 student memories of instructor language promoting a positive learning environment and n = 194 student memories of instructor language promoting a negative learning environment). Because students could provide up to three memories and not all students did, we analyzed the proportion of students who could recall instructor language of particular types and the percentage of total memories provided by all students. In the following sections, we present five analyses of these data.

Student Memories of Instructor Talk Overall and Disaggregated by Self-Identified Demographics.

Overall, the majority of students could recall at least one memory of instructor language that was codable as Instructor Talk (see Figure 4A). Of the 90 participants, 99% were compliant and provided at least one memory (n = 89). The majority of students could recall at least one memory codable as Positively Phrased Instructor Talk that promoted a positive learning environment (78%, n = 70) and at least one memory codable as Negatively Phrased Instructor Talk that promoted a negative learning environment (74%, n = 67). The remaining students either left the form blank (n = 6 for positive, n = 9 for negative), provided only examples that were misaligned with the frameworks (n = 1 for positive, n = 1 for negative), and/or only provided memories that were coded as Not Instructor Talk (n = 14 for positive, n = 14 for negative).

To determine whether higher proportions of certain student groups recalled Instructor Talk, we disaggregated the data by students’ self-identified characteristics after quantifying the total proportion of students who could recall at least one instance of Instructor Talk. Figure 4B and C shows the proportion of undergraduate students who recalled at least one memory of Instructor Talk in a biology class, disaggregated by self-identified binary gender, first-generation college status, PEER, and POC. No significant differences were found in the proportion of students from different demographics who provided more than one instance of Instructor Talk (Pearson’s chi-square test with Bonferroni corrections, p > 0.085 for all tests; Supplemental Table S4), though this may be due to our moderate sample size.

Promoting a Positive Learning Environment: Student Memories of Instructor Talk.

To investigate which categories of Instructor Talk were the most salient for students, we quantified the proportion of students who remembered instructor language in each framework category that promoted a positive learning environment. Given that students could provide up to three memories, we also analyzed the percent of all student memories of Instructor Talk that were coded into each category of the Instructor Talk frameworks.

Analyzing the Proportion of Individual Students Who Provided Memories of Instructor Talk Perceived as Promoting a Positive Learning Environment.

To determine the extent to which students could remember noncontent language as promoting a positive learning environment and in which categories these memories could be coded, we calculated the proportion of students who wrote down a memory that was codable as Instructor Talk and the categories of these memories (see Figure 5 and Table 7). The vast majority of students wrote down a memory of instructor language promoting a positive learning environment that was coded as Building the Instructor/Student Relationship (76%, n = 53/70). The proportion of students who wrote down memories coded in the remaining four categories of the Positively Phrased Instructor Talk framework are as follows, in order of decreasing prevalence: Establishing Classroom Culture (27%, n = 19/70), Explaining Pedagogical Choices (24%, n = 17/70), Unmasking Science (21%, n = 15/70), and Sharing Personal Experiences (14%, n = 10/70). Additionally, a subset of students recalled a memory of instructor language that they felt promoted a positive learning environment but was coded as Negatively Phrased Instructor Talk by researchers (16%, n = 11/70).

Analyzing the Percentage of All Student Memories of Instructor Talk Perceived as Promoting a Positive Learning Environment.

Each student could provide up to three memories of Instructor Talk that they perceived as promoting a positive learning environment. We explored the prevalence of student memories of Instructor Talk overall by analyzing the percent of all student memories coded in each category. Table 8 summarizes the percentage of student memories of Instructor Talk that students felt promoted a positive learning environment (n = 166 total), sorted by category and with examples of student memories for each subcategory. Overall, students’ memories of Instructor Talk perceived as promoting a positive learning environment could be coded in all five categories and nine of the 10 subcategories of the Positively Phrased Instructor Talk framework, with the exception that no student memories were coded in the subcategory Giving Credit to Colleagues. Over half of all student memories of Instructor Talk that promoted a positive learning environment were coded in the Building the Instructor/Student Relationship category (52%, n = 87/166). The second most frequent category of student memories of Instructor Talk that promoted a positive learning environment was Establishing Class Culture (14%, n = 23/166). Students’ memories of Instructor Talk also consisted of memories coded as Explaining Pedagogical Choices (11%, n = 19 /166) and Unmasking Science (9%, n = 15/166). Surprisingly, the least frequent category of student memories of Instructor Talk that fostered a positive learning environment was Sharing Personal Experiences (6%, n = 10/166). Approximately 7% (n = 12/166) of all student memories of Instructor Talk perceived as promoting a positive learning environment could be coded with the Negatively Phrased Instructor Talk framework, the majority of which were coded in the subcategory Focusing on the Grade/Short Term (5%, n = 9/166). Other subcategories of student memories coded as Negatively Phrased Instructor Talk that students perceived as promoting a positive learning environment included Assuming Poor Behaviors from Students (n = 2/166) and Criticizing Colleagues (n = 1/166), as shown at the bottom of Table 7. The percent of student memories of Instructor Talk at the subcategory level of the Positively Phrased Instructor Talk framework is detailed in Supplemental Table S5.

Promoting a Negative Learning Environment: Student Memories of Instructor Talk.

Additionally, we quantified the proportion of students who remembered instructor language that promoted a negative learning environment and in which categories of the Instructor Talk frameworks these student memories of Instructor Talk were coded. As with the previous analyses, because students could provide up to three memories, we also analyzed the percent of all student memories of Instructor Talk coded into each category of the Instructor Talk frameworks.

Analyzing the Proportion of Individual Students Who Provided Memories of Instructor Talk Perceived as Promoting a Negative Learning Environment.

To determine the extent to which students could remember noncontent language as promoting a negative learning environment and in which categories these memories could be coded, we calculated the proportion of students who wrote down a memory that was codable as Instructor Talk and the categories of these memories (see Figure 6 and Table 9). The majority of students provided a memory of instructor language that promoted a negative learning environment that was coded as Dismantling the Instructor/Student Relationship (61%, n = 41/67). The proportion of students who wrote down memories coded in the remaining four categories of the Negatively Phrased Instructor Talk framework are as follows, in order of decreasing prevalence: Masking Science (40%, n = 27/67), Dismantling Classroom Culture (34%, n = 23/67), Compromising Pedagogical Choices (27%, n = 18/67), and Sharing Personal Judgment (3%, n = 2/67). Some students recalled a memory of instructor language that they felt promoted a negative learning environment but was coded as Positively Phrased Instructor Talk by researchers (15%, n = 10/70).

Analyzing the Percentage of All Student Memories of Instructor Talk Perceived as Promoting a Negative Learning Environment.

Given that each student could provide up to three memories of Instructor Talk perceived as promoting a negative learning environment, we analyzed the percent of all student memories coded in each category of the Instructor Talk frameworks. Table 10 summarizes the percentage of student memories of Instructor Talk that students felt promoted a negative learning environment (n = 141 total) and Table 10 has examples of student memories for each subcategory. Overall, student memories of Instructor Talk perceived as promoting a negative learning environment could be coded in all five of the categories and seven of the 10 subcategories of the Negatively Phrased Instructor Talk framework. The most frequent category of student memories of Instructor Talk that promoted a negative learning environment were coded in the Dismantling the Instructor/Student Relationship (40%, n = 57/141). The second most frequent category of student memories of Instructor Talk that promoted a negative learning environment was Masking Science (20%, n = 28/141). Student memories of Instructor Talk also consisted of memories coded as Disestablishing Class Culture (17%, n = 24/141) and Compromising Pedagogical Choices (14%, n = 19 /141). The least frequent category of student memories of Instructor Talk that promoted a negative learning environment was Sharing Personal Judgment (1%, n = 2/141). Approximately 7% (n = 10/141) of all student memories of Instructor Talk perceived as promoting a negative learning environment could be coded with the Positively Phrased Instructor Talk framework, the majority of which represented the subcategories Pre-framing Classroom Activities (3%, n = 4/141) and Recounting Personal Information or Anecdotes (2%, n = 3/141). The percent of student memories of Instructor Talk at the subcategory level of the Negatively Phrased Instructor Talk framework is detailed in Supplemental Table S6.

Comparing Prevalence of Categories of Student Memories of Instructor Talk to a Previous Study of the Prevalence of Categories of Instructor Talk in Undergraduate Biology Classrooms.

To establish whether student memories of Instructor Talk in biology classrooms were a reflection of what instructors say the most often, we compared the frequency of student memories of Instructor Talk in our study to the prevalence of Instructor Talk categories recorded previously in biology classrooms (Harrison et al., 2019). Figure 7 shows the percent of total student memories for each Instructor Talk category (n = 307 total memories) and compares this with reprinted data from a previous study on Instructor Talk, for which researchers recorded and quantified Instructor Talk instances (n = 567 total instances) across 61 undergraduate biology courses (Harrison et al., 2019). Both student memories and previously recorded instances of Instructor Talk are displayed at the category level of the Positively Phrased and Negatively Phrased Instructor Talk frameworks.

As shown in Figure 7A, the most prevalent categories of student memories of Instructor Talk perceived as promoting a positive learning environment were Building the Instructor/Student Relationship and Establishing Class Culture, which were also the most prevalent categories of Positively Phrased Instructor Talk instances in classrooms. We observed discrepancies in the remaining categories, with fewer student memories than might be expected. There were far fewer student memories of Instructor Talk categorized as Sharing Personal Experiences and Explaining Pedagogical Choices than what might be expected based on the prevalence of recorded Instructor Talk instances categorized as such. Additionally, the rarest category of recorded Instructor Talk instances was Unmasking Science, but the prevalence of student memories of this category of Instructor Talk was much higher than what might be predicted.

As shown in Figure 7B for Negatively Phrased Instructor Talk categories, we found that with the exception of Disestablishing Class Culture, there were discrepancies between the prevalence of categories for student memories of Instructor Talk perceived as promoting a negative learning environment and Instructor Talk instances recorded previously in classrooms. There were far more student memories of Instructor Talk categorized as Dismantling the Instructor/Student Relationship than what might be expected based on the prevalence of this category for Instructor Talk instances recorded in classrooms. Student memories of Instructor Talk were similarly spread across three categories: Disestablishing Class Culture, Masking Science, and Compromising Pedagogical Choices. However, there were fewer student memories of Instructor Talk categorized as Compromising Pedagogical Choices and far more categorized as Masking Science than what might be expected relative to the prevalence of Instructor Talk instances categorized as such. Finally, the least prevalent category of student memories of Instructor Talk perceived as promoting a negative learning environment was Sharing Personal Judgment/Self-Pity, which was found much less than what might be predicted based on the prevalence of these Instructor Talk instances recorded in classrooms.

DISCUSSION

Instructor Talk has been observed in every biology classroom studied to date (Seidel et al., 2015; Harrison et al., 2019; Lane et al., 2021), but a systematic analysis of how students remember and perceive Instructor Talk has yet to be conducted. Our results show the kinds of Instructor Talk that students remembered from biology courses and how students perceived examples of Instructor Talk as influencing the learning environment. In our studies, we were able to characterize the vast majority of student memories of noncontent instructor language using the existing Instructor Talk frameworks. Further, we found significant alignment between researchers’ categorizations and students’ perceptions of Instructor Talk quotes—namely, if the quote was perceived as promoting a positive or negative learning environment. In the following sections, we discuss six key findings from our studies of student memories of Instructor Talk and student–researcher alignment in their perceptions of Instructor Talk quotes. We contextualize each finding with theoretical frameworks related to instructor immediacy, student resistance, and stereotype threat. These findings serve as an impetus for future studies on noncontent instructor language, and we hypothesize that Instructor Talk may be a key variable mediating outcomes in biology education research.

Student Memories of Instructor Talk Predominantly Related to the Instructor–Student Relationship

In our investigations, students perceived instructor language as affecting their learning environments, but what kinds of Instructor Talk did students remember the most? It could have been that most students remembered how their instructors described active-learning techniques. There is extensive research on how such strategies support equitable student performance in science classrooms (Freeman et al., 2014; Theobald et al., 2020), so one might predict that students remember instructor language for implementing these pedagogical strategies. Or perhaps, students would remember the language of instructors who curve exams and offer extra credit. Academic grades are often used as a proxy for predicting future success, despite evidence to the contrary (Roth et al., 1996; Arum and Roksa, 2011), so one could imagine that students pay close attention to instructors’ references to grades. Another possibility might be that students would remember the personal stories that instructors had shared about themselves. There is evidence that relevant self-disclosure by instructors can have a positive impact on learning outcomes (Kromka and Goodboy, 2021). However, we found that instructor language related to pre-framing classroom activities, focusing on the grade, or sharing personal stories was not the most prominent kind of Instructor Talk that students remembered in the present study.

To our surprise, the majority of students remembered language that related to the instructor–student relationship (i.e., the Instructor Talk framework categories of Building the Instructor/Student Relationship and Dismantling the Instructor/Student Relationship). Previous research on instructor immediacy has studied certain aspects of instructor language, including grammatical subtleties (e.g., verb tense and adjective use) and cultural subtleties, such as instructors being called by their first name or using humor (Gorham, 1988; Stuart and Rosenfeld, 1994). However, these verbal cues were described by researchers as promoting instructor immediacy without considering the perceptions and memories of students. In the present study, we observed that students remembered more than grammatical or cultural subtleties in instructor language. Notably, students could provide specific examples of what instructors said about the Instructor/Student relationship, whether promoting a positive learning environment (e.g., “When you need help, feel free to reach out. I am always here for you.”) or a negative learning environment (e.g., “Please do not waste my time.”).

One wonders whether student memories of instructor language related to the instructor–student relationship would be as prevalent at different time points or in other contexts. Our studies solicited upper-division biology students’ memories of instructor language. Students were encouraged to consider all the biology courses they had taken. Would students remember different kinds of Instructor Talk right after a class session? How might targeted, short-term memories of instructor language compare with the overarching, long-term memories we solicited from students? Additionally, the present studies were conducted in a department with a high rate of faculty participation in professional development for scientific teaching (Owens et al., 2018). Would student memories of instructor language related to the instructor–student relationship be as prevalent at an institution with less pedagogical training on inclusive practices in the sciences? How might instructor language that students perceive as promoting instructor immediacy affect students’ sense of belonging in science? Future investigations could explore the relationship between these variables and student memories of instructor language.

Student Memories of Noncontent Instructor Language Are Extensive and Codable with the Instructor Talk Frameworks

From study 2, it is clear that the vast majority of students can remember examples of what instructors say that is unrelated to course content. To distinguish and quantify student memories of instructor language, we first needed to determine how to categorize the kinds of memories that students provided. While there was no guarantee that the Instructor Talk frameworks would encapsulate the instructor language that students remembered, we discovered that the vast majority of student memories could be categorized within the existing frameworks. In fact, we observed that all 10 categories of the Instructor Talk frameworks were represented in our sample of student memories of instructor language. Further, we saw evidence of student memories of instructor language for all but two of the 33 subcategories of the existing Instructor Talk frameworks. The two subcategories of Instructor Talk for which we have not yet observed student memories were Giving Credit to Colleagues (Positively Phrased) and Sharing Self-Judgment/Self-Pity (Negatively Phrased). One may speculate that instructor language related these two subcategories were not salient for students, because such language is more instructor centered than student centered. Interestingly, two of the 20 quotes of our Instructor Talk stimulus set were examples of Sharing Self-Judgment/Self-Pity, which students saw before being prompted for memories of instructor language. Even so, none of the students recalled a memory that could be categorized as Sharing Self-Judgment/Self-Pity. Despite these exceptions, it was striking that we observed almost every subcategory of the existing Instructor Talk frameworks was represented in our sample of student memories of instructor language.

Although the vast majority of student responses offered as memories of instructor language could be coded with the Instructor Talk frameworks, a small portion of student memories could not. Characteristics of student memories that were not codable as Instructor Talk included nonverbal cues (e.g., “Smiling”), references to inclusive practices (e.g., “Utilizing non-gendered cues”), examples of class assignments (e.g., “Doing a poster session in biology lab”), logistical language (e.g., “Exams and problem sets are take-home, you can use anything that is at your disposal, and it will be collected in one week.”), or descriptions of what was said rather than an actual quote (e.g., “Personal struggle on how to remember certain things/topics”). The rarity of these kinds of student memories, as well the variation in formats and themes, did not enable systematic quantitative analysis. These offerings by students suggest that nonverbal cues and actions by instructors are a ripe area for study. One wonders what biology students would remember about nonverbal cues, inclusive practices, and assignments that promote what they perceive to be a positive learning environment.

Students and Researchers Similarly Classify Positively Phrased and Negatively Phrased Instructor Talk

While the Positively Phrased and Negatively Phrased Instructor Talk frameworks were developed by researchers, it had not yet been determined whether students would similarly perceive examples of noncontent instructor language as positive or negative. We found that the vast majority of students’ perceptions of Instructor Talk quotes—as promoting either a positive or negative learning environment—were in alignment with researchers’ categorizations. The proportion of students who were unsure about how to evaluate examples of Instructor Talk was relatively low, which suggests that most students can confidently evaluate instructor language as promoting a positive or negative learning environment.

Our findings contribute to validating the Instructor Talk frameworks—students’ perceptions of our sample quotes mostly align with the researchers who previously categorized them. Given the overall alignment between students and researchers in their perceptions of Instructor Talk, how might biology students’ sense of belonging correspond to the frequency of different kinds of Instructor Talk in their classes? For measuring sense of belonging, several items have been developed and applied in STEM settings (Trujillo and Tanner, 2014). Items to assess students’ sense of belonging could be adapted to invite students to reflect on the classroom learning environment and share what kinds of noncontent instructor language affect their sense of belonging. Future studies could explore students’ perceptions of their actual learning environments as correlated with their evaluations of Instructor Talk quotes recorded from their biology classrooms.

Issues of Grading Influence Student–Researcher Disagreements about Perceptions of Instructor Talk

Despite the overall high student–researcher alignment, most students disagreed with researchers’ categorization of the Instructor Talk quote that discussed grades as Negatively Phrased. Why might students perceive Instructor Talk that focuses on the grade as promoting a positive learning environment? Enrolling in subsequent courses and postgraduate pursuits often requires students to meet a minimum grade point average (GPA) to be considered, and an even higher GPA to be competitive, for the opportunity. Thus, students are conditioned to consider a high grade to be a metric for academic success, because it is required to enroll in subsequent courses, to finish their degrees, and to gain access to opportunities in pursuit of their career goals. Researchers considered this quote as Negatively Phrased Instructor Talk and categorized it within the Focusing on the Grade/Short Term category, because grades are not a predictor of future success (Roth et al., 1996; Arum and Roksa, 2011) and the quote does not emphasize learning biology for the long term (Schinske and Tanner, 2014). Consider the following Instructor Talk stimulus set quote from the perspective of a student:

Given the disagreement between students and researchers in their perceptions of this quote, we explored student rationales for why they considered this language as promoting a positive learning environment. Students shared that they appreciated that the instructor was transparent about how to earn a high grade in the course and that this language would motivate them to work hard in the class. Additionally, student memories about grades comprised the few instances of positive student memories that researchers categorized as Negatively Phrased Instructor Talk.

Why might students and researchers perceive language about grades differently? Reaching a benchmark grade is required for students to enroll in subsequent courses and pursue academic opportunities. Further, grades are used by students to make sense of their academic standing in a course. Although students perceived the quoted instructor language about grades as motivating their academic success in the course, grading does not accurately assess student learning (Schinske and Tanner, 2014). In fact, grades can increase students’ fear of failure, thus motivating them with fear to earn higher grades rather than achieve authentic learning (Butler and Nisan, 1986; Butler, 1988; Crooks, 1988; Pulfrey et al., 2011). Moreover, grading systems arose from a need for schools to communicate uniformly about student performance and academic readiness (Schinske and Tanner, 2014) not as a need to measure student learning. Thus, there is a breakdown in how students perceive grades (i.e., furthering their academic trajectories) and how researchers perceive grades (i.e., motivation through fear of failure).

Can Instructor Talk that focuses on metacognition and the scientific process shift students’ perceptions of what being academically successful means, away from a focus on grades? Given that students consider instructor language concerning grades as promoting a positive learning environment, future research may focus on using Instructor Talk to shift student perceptions of what constitutes academic success.

Nuances of Instructor Language May Affect Students’ Interpretation of That Language

Previous investigations of Instructor Talk acknowledged both the complexities of language and the challenges of coding instructor language as either Positively Phrased or Negatively Phrased Instructor Talk (Harrison et al. 2019). Although we observed a high degree of alignment between students’ and researchers’ perceptions of Instructor Talk, the language used by biology instructors in classrooms is varied and complex and can be interpreted differently than intended. These nuances in language include the choice of a particular word or expression to communicate an idea. Given the intricacies of language, we explored the two individual Instructor Talk quotes with the lowest agreement between students and researchers that were not focused on grading. We highlight these two misaligned examples and propose future directions to investigate the nuances of instructor language more empirically.

First, we present an example of an Instructor Talk quote that researchers categorized as Positively Phrased, because it prepares students for upcoming in-class activities (category: Explaining Pedagogical Choices; subcategory: Pre-framing Classroom Activities). Yet nearly three-quarters of students considered this quote as promoting a negative learning environment and thus were misaligned with researchers. Consider the following quote:

What reasons did students offer for evaluating this quote as promoting a negative learning environment? In their rationales, many students shared that they do not like being “picked on,” because it can be embarrassing to give an incorrect answer and that being put on the spot makes them nervous. How would changing the language from “I’m going to pick on people” to “I’m hearing lots of great ideas, so I’m going to ask for a few volunteers to share” influence how this language is perceived by students? The former requires students to share when picked on, but the latter invites students to voluntarily share their ideas with the class, which may change how students perceive this instructor language. Such language may help build trust between the instructor and students and thus be advantageous to circumvent student resistance when facilitating active-learning strategies in the classroom, like think–pair–share (Seidel and Tanner, 2013; Cooper et al., 2021). Future studies could analyze instructor language in the classroom in association with the Classroom Observation Protocol for Undergraduate STEM (COPUS; Smith et al., 2013) to correlate the use of active-learning strategies with Instructor Talk. Interestingly, instructors’ COPUS profiles and Instructor Talk have been studied for the first day of class for introductory STEM courses (Lane et al., 2021), offering a model for future work in this area. Additionally, there are several categories of Instructor Talk that could affect student resistance to active learning. For example, previous research has highlighted a misalignment between introductory biology students’ perceptions of why instructors implement active-learning strategies and instructors’ rationales (Brigati et al., 2019). Future studies might consider how instructors explain or compromise pedagogical choices; student memories of such instances of Instructor Talk; and students’ enthusiasm, apathy, or resistance to active learning.

Second, we present an Instructor Talk quote that researchers categorized as Negatively Phrased, because it may discourage students from taking the course and thus from persisting in biology as a discipline (category: Masking Science; subcategory: Intimidating Students from Science). In contrast with researcher categorizations, approximately 40% of students considered this quote as promoting a positive learning environment:

Why did some students evaluate this quote as promoting a positive learning environment? In their rationales, many students shared that this quote would ease nerves by acknowledging the challenges and scariness of taking a biology course and that some humor can help break the ice. While some students appreciated this transparency and authenticity from an instructor, implying that a class is scary may also intimidate other students. Instructor language may influence students to do poorly on assessments due to stereotype threat (i.e., an individual’s fear of conforming to a negative stereotype about an aspect of his or her identity; Steele and Aronson, 1995). Beyond immediate impacts in the course, instructor language that promotes a negative or unwelcoming learning environment may disproportionately influence students who are historically marginalized in science to disenroll from the course or even leave the discipline altogether (Seymour and Hewitt, 1997; Seymour and Hunter, 2019). Yet how would adding language at the end of the above quote that states “I am here to support you through all of the material, and I want you to be successful” influence students’ perceptions? Would more students feel supported and perhaps less intimidated? How would this change across student demographics?

Given the complexity of language and the varied reactions that individuals can have to the same language, empirical laboratory investigations could explore how the specifics of language used in biology classrooms might influence individual students’ experiences. Future studies could also address the nuances of language by engaging students in choice experiments in which they are given two similar Instructor Talk quotes with small changes in language. These choice experiments could allow us to better understand how the complexities of instructor language may promote either a positive or a negative learning environment.

Across Personal Characteristics, Students Remember Details about Biology Instructors’ Noncontent Language

From our investigations, it was striking that all students, across multiple personal characteristics, could similarly remember instructor language that they perceived as promoting either a positive or a negative learning environment. Given previous research on stereotype threat (Steele and Aronson, 1995; Steele, 1997; Beasley and Fischer, 2012), we predicted that more students from groups that are historically and presently excluded from the sciences might remember more Negatively Phrased Instructor Talk. We also had predicted that we would find more student memories of instructor language that might be considered to be microaggressions in biology classrooms (Harrison and Tanner, 2018). However, at our institution, we found that a similar proportion of students from different demographic groups could remember a range of examples of instructor language.

Interestingly, we found a similar proportion of students recalled Positively Phrased and Negatively Phrased Instructor Talk, regardless of their backgrounds. However, we observed a trend where a higher proportion of women, students of color, and first-generation college-going students could recall examples of Negatively Phrased Instructor Talk relative to their dominant culture counterparts. Although the trend was not statistically significant, one wonders whether students who were affected by Negatively Phrased Instructor Talk left the major before reaching the upper-division course from which we recruited participants. This pattern may be emerging as significant if we were to conduct a future study with students at an earlier stage of the academic pathway. Such a study could include students who might be considering the possibility of switching majors and who may remember more examples of discouraging instructor language (Seymour and Hunter, 2019). Future investigations could invite students from lower-division biology courses to share their memories of noncontent instructor language that promoted positive or negative learning environments.

Limitations of the Study and Future Directions: To What Extent Is Instructor Talk a Key Variable and Mechanism for Successful Evidence-Based Teaching?

Instructors Talk, or instructors’ noncontent language used in classrooms, is remembered and perceived by students as promoting positive and negative learning environments. However, there are limitations to our studies that are worth considering. First, in our assessment, we opted to give students a stimulus set of Instructor Talk quotes to evaluate before we asked for student memories of instructor language. Although one might suspect that this primed students to remember certain kinds of instructor language, we found that the categories from the stimulus set were not equally represented in student memories of Instructor Talk. Second, we compared student memories of Instructor Talk at one university to the prevalence of Instructor Talk across multiple institution types, including the university of the present study. Future studies could record Instructor Talk and solicit student memories of instructor language to investigate to what extent the prevalence of Instructor Talk relates to student memories at a single institution. Third, the present study was conducted in a department with a high rate of faculty participation in professional development on inclusive teaching practices. Future studies might investigate students’ memories of Instructor Talk at institutions with little to no faculty professional development on inclusive practices. We hypothesize that Instructor Talk is less prevalent in the classrooms of instructors with less professional development on inclusive pedagogy. Subsequently, fewer students in these contexts may even remember examples of Instructor Talk. Fourth, student perspectives of the present studies were limited to only upper-division biology students. In future studies, one could consider the perspectives of nonmajors or students enrolled in lower-division prerequisite courses. We predict that there may be more Negatively Phrased Instructor Talk remembered by students who chose not to major in the sciences or by students who might be in the process of switching majors. Fifth, we only offered students a random selection of previously recorded Instructor Talk quotes in written form. This approach to soliciting student perceptions of Instructor Talk does not reflect the many contextual variables that inevitably shape students’ interpretations in the classroom. Each instance of Instructor Talk comes from a particular instructor with a unique set of personal characteristics, pattern of nonverbal immediacy cues, and vocal attributes like tone and pacing. Future studies could explore how aspects of instructor identity and delivery may affect student perceptions of Instructor Talk. Finally, given that active learning can result in various student outcomes in STEM courses (Freeman et al., 2014), we hypothesize that Instructor Talk may be the driving mechanism that explains why some instructors observe shifts in student outcomes with these instructional strategies, while others do not, or why the same instructor may observe variations in student outcomes over time. Instructor Talk is yet to be observed across STEM disciplines. Given our findings, future research might also investigate how students’ perceptions of instructor immediacy, student resistance, sense of belonging, and stereotype threat relate to their perceptions of different kinds of Instructor Talk used in science classrooms.

CONCLUSION

Given the pervasiveness of Instructor Talk across science classrooms and institutions, we investigated and discovered that students remember what instructors say in their biology classes. Further, students overall perceive Instructor Talk as promoting positive or negative learning environments, which has implications for how Instructor Talk can be strategically implemented to foster inclusion. Importantly, students’ perceptions of examples of Instructor Talk overall align with researchers’ categorizations. Thus, student memories offer a proxy to study the prevalence and types of Instructor Talk that students experience in biology classrooms. Additionally, we have developed methods for soliciting student memories of Instructor Talk, coding these student memories, and assessing students’ evaluations of examples of Instructor Talk. Finally, the prevalence of student memories of Instructor Talk across personal characteristics suggests that students’ perceptions of Instructor Talk represent a fruitful area for future research.

ACKNOWLEDGMENTS

REFERENCES