Beyond the Biology: A Systematic Investigation of Noncontent Instructor Talk in an Introductory Biology Course
Abstract
Instructors create classroom environments that have the potential to impact learning by affecting student motivation, resistance, and self-efficacy. However, despite the critical importance of the learning environment in increasing conceptual understanding, little research has investigated what instructors say and do to create learning environments in college biology classrooms. We systematically investigated the language used by instructors that does not directly relate to course content and defined the construct of Instructor Talk. Transcripts were generated from a semester-long, cotaught introductory biology course (n = 270 students). Transcripts were analyzed using a grounded theory approach to identify emergent categories of Instructor Talk. The five emergent categories from analysis of more than 600 quotes were, in order of prevalence, 1) Building the Instructor/Student Relationship, 2) Establishing Classroom Culture, 3) Explaining Pedagogical Choices, 4) Sharing Personal Experiences, and 5) Unmasking Science. Instances of Instructor Talk were present in every class session analyzed and ranged from six to 68 quotes per session. The Instructor Talk framework is a novel research variable that could yield insights into instructor effectiveness, origins of student resistance, and methods for overcoming stereotype threat. Additionally, it holds promise in professional development settings to assist instructors in reflecting on the learning environments they create.
INTRODUCTION
What do you say when you teach students? What proportion of what you say is about the concepts you want them to learn? What proportion is about other things? To what extent do you say things to build community among your students? To what extent do you give students a motivational speech leading up to an exam? Or express to your students why you teach the way you do? When during a course do you say noncontent-related things? On the first day of class? Right before or after an exam? To what extent do you plan what you will say to students before you walk into the classroom to teach?
In this initial research study, we define the construct of Instructor Talk and introduce methods to characterize it. We define Instructor Talk as any language used by an instructor that is not directly related to the concepts under study but instead focuses on creating the learning environment. For example, Instructor Talk may include language involved in giving directions, sharing personal stories, or building community among students. Before this research, we hypothesized that the majority of Instructor Talk would likely be focused on explaining why an instructor chooses particular teaching methodologies. However, we know of no research that has systematically recorded, transcribed, and analyzed the talk that happens in a college biology course. This is surprising, given that there are multiple lines of evidence suggesting that what an instructor says in a classroom that is not concept related—Instructor Talk—may be important for student engagement, success, and learning. The most relevant research literatures that suggest that Instructor Talk warrants study focus on three broad areas: 1) student resistance, 2) stereotype threat, and 3) instructor immediacy. We explore here evidence from social psychology and communications researchers and discipline-based education research that suggests Instructor Talk may significantly affect learning environments.
The first research literature that suggests that measuring Instructor Talk may be important is literature on student resistance to student-centered learning environments. Whether warranted or not, student resistance is a great concern for many instructors when they develop student-centered learning environments (Seidel and Tanner, 2013). Despite this fear, the majority of students do not report being resistant, and the majority of instructors do not report facing student resistance. Lea et al. (2003) characterized students’ perceptions and attitudes toward student-centered learning environments and found that, while many students held positive views toward these teaching methods, ∼40% of students reported having more cynical views in a previous class (e.g., the instructor does not want to teach us and is making us do all the work). Similarly, a study of college physics instructors found that as few as 19% of instructors saw decreases in end-of-semester student evaluations—attributed to student resistance—while nearly 50% saw increases in evaluations after implementing research-driven teaching methods (Dancy et al., 2014). If it is the case that a small subset of instructors face student resistance while the majority do not, then what is different about the learning environments in these classrooms? Several instructors who have successfully implemented research-based teaching methods have suggested that the things they say to their students about why they teach the way they do are critically important to mitigating student resistance (Silverthorn, 2006; Science Education Initiative, 2013). However, what instructors say and how this relates to student resistance has not been systematically studied.
The second area of research that indicates that characterizing and attending to Instructor Talk may be important comes from social psychologists who have studied “stereotype threat.” Stereotype threat is a phenomenon that has been demonstrated to occur when people who identify with a particular group (e.g., gender, ethnicity) are negatively stereotyped and worries related to the stereotype hinder their performance in stereotype-evoking high-stakes testing situations (Steele and Aronson, 1995). The result is disproportionately lower scores on the given performance test and the incorrect assumption that people from the stereotyped group are lower performing. Steele and Aronson (1995), in their initial report of stereotype threat, demonstrated that simply telling participants at the beginning of a test that it was “diagnostic of intellectual ability” was enough to induce stereotype threat and the associated underperformance. Many other studies have replicated their findings for a variety of demographic groups (e.g., Croizet and Claire, 1998). Encouragingly, it has been shown that verbal interventions that describe tests as “non-diagnostic of intellectual ability” or use other such verbal phrases that signal safety in the learning environment and minimize stereotype threat can decrease its impact (Croizet and Claire, 1998; Spencer et al., 1999).These studies have repeatedly demonstrated the importance of context and verbal behaviors in inducing or overcoming stereotype threat. Additionally, studies of values affirmations have produced preliminary demonstrations of the potential for small-scale interventions with large impacts on student performance in science, technology, engineering, and mathematics (STEM) courses (Cohen et al., 2006; Miyake et al., 2010). If as little as a single sentence uttered by an instructor or a 15-min writing assignment can have an impact on student performance on an exam, studying Instructor Talk may be critically important for understanding and overcoming stereotype threat and achievement gaps that have been identified for some student populations in STEM courses (Lauer et al., 2013; Madsen et al., 2013; Eddy and Hogan, 2014). Future studies may wish to include Instructor Talk as a variable when measuring potential achievement gaps in science courses.
The third area of research that implies an important role for Instructor Talk and draws a potential link to student learning relates to a phenomenon termed “instructor immediacy.” Social psychologists and communications researchers describe instructor immediacy as instructor behaviors that effectively decrease the social and emotional distance between the instructor and students (Mehrabian, 1971). Instructor immediacy includes both nonverbal behaviors (e.g., facial expressions, gestures, movement around the classroom) and verbal behaviors (e.g., calling students by name, using humor, vocal tone variation), including the noncontent language used by instructors. Interestingly, instructor immediacy has been linked to student learning. A meta-analysis of studies that have looked at the correlation between instructor immediacy and aspects of student learning showed that there is a strong correlation between both perceived and affective learning, although the link to cognitive learning was somewhat less robust (Witt et al., 2004). While Instructor Talk may be a component of the larger concept of instructor immediacy, the nonconceptual language an instructor uses in the classroom may be particularly important for increasing student learning.
Despite these multiple lines of evidence indicating that instructors’ verbal language may impact the learning environment, little has been done to systematically analyze what we are calling Instructor Talk. To begin to analyze Instructor Talk in relation to student outcomes like student resistance, stereotype threat, and student learning, we must first characterize the types of Instructor Talk that exist in undergraduate science classrooms. In this initial analysis of Instructor Talk, we systematically describe Instructor Talk with the aim of addressing the following four research questions: 1) To what extent is Instructor Talk present in an introductory college biology course? 2) What types of Instructor Talk exist in a selected introductory college biology course? 3) When throughout a semester does Instructor Talk occur? 4) To what extent do two instructors differ in the types and quantity of Instructor Talk they appear to use? We share here results of capturing Instructor Talk from a semester-long course with two coinstructors, an emergent framework for analyzing Instructor Talk, and strategies for future use of the Instructor Talk framework in research and faculty professional development.
METHODS
This research aimed to characterize previously unstudied forms of Instructor Talk by systematically identifying instructor statements not explicitly related to course content in a semester-long introductory biology course. In this section, we describe the methods used to develop our framework of Instructor Talk, including: 1) identifying a course for analysis, 2) defining instances of Instructor Talk, 3) developing categories of Instructor Talk, 4) measuring interrater reliability of the framework, and 5) analyzing quantitative data. This study was approved by the institutional review board at San Francisco State University under protocol number E14-141.
Identifying a Course for Analysis
The goals of this project were to develop a method for identifying forms of Instructor Talk, characterizing its prevalence, and quantifying variation between different instructors’ use of Instructor Talk in their classrooms. To maximize the likelihood of capturing a rich Instructor Talk data set, we purposefully sampled in a course in which we hypothesized that we might find non content-specific language. Specifically, we chose a course with three key characteristics: 1) the instructors used a wide variety of innovative, student-centered teaching methods; 2) the instructors reported little to no student resistance; and 3) the course was taught by two instructors. We identified a course that was cotaught by two instructors with varied biological and pedagogical training, career paths, and classroom presentation styles. We hypothesized instructor differences would add to the variety of Instructor Talk identified, providing a more diverse data set for building an initial Instructor Talk framework. Additionally, because analysis of Instructor Talk may be useful in future research to measure correlations with a variety of student outcomes, we chose a course that used a student-centered learning environment with little student resistance as self-reported by the instructors.
The course chosen for analysis was an introductory biology course that was composed of both biology majors and non–biology majors in other science disciplines. Nonmajors typically were required to take the course for other fields of study, including biochemistry, environmental studies, and kinesiology. The majority of enrolled students were aspiring healthcare professionals, including premedical, predental, and prepharmacy students. The course was taught at a large public university and enrolled ∼270 students. Class meetings occurred in a tiered lecture hall three times a week for 50 min per class session, with a total of 43 class meetings over the course of the semester. A graduate teaching assistant videotaped class meetings for the instructors’ own analysis and self-improvement of teaching methods. Videos were filmed in 2011, before development of our Instructor Talk hypothesis, therefore neither instructor was aware of the potential for this type of analysis at the time of instruction. Both instructors agreed to participate anonymously and to give us access to the video recordings from a semester of their course.
Lecture was used in the course; however, both instructors interspersed minilectures with multiple clicker questions or activities in each class session. While the course would not be defined as a traditional lecture course, it was also not a cooperative-learning environment, flipped classroom, or a peer-led teaching and learning classroom. There were also multiple pair discussions among students within each class session. Students were required to complete preparatory homework assignments outside class at least twice per week, and these assignments typically required students to write 300–400 words per assignment and to submit assignments to an online learning system. In addition, the course regularly used equity and diversity strategies to engage all students, including use of name placards, multiple responses to open-ended questions, and the use of culturally diverse examples. Course assessment included not only formative homework assignments and low-stakes clicker questions but also four course exams, which were cumulative and a mix of multiple-choice and essay questions.
In the course studied, there were two instructors, one of whom taught the first and third quarter and another who taught the second and fourth quarter. Both instructors attended each class session and met weekly to plan together and reflect on student work. Both were also committed to scientific teaching and strove to include active learning, equity and diversity, and assessment strategies in each class session. The two instructors involved in this study were both tenured faculty with federal research grants and active laboratories that included multiple graduate students and postdoctoral-level scientists. Instructor A was an associate professor with 6 yr of college teaching experience who had received specialized pedagogical training at both the graduate and postdoctoral level and who was teaching this course for the second time. Instructor B was a professor with 14 yr of college teaching experience who had had no formal pedagogical training at the graduate or postdoctoral level and who was teaching this course for the fifth time. Author S.B.S. had the opportunity to observe both instructors extensively through video and found their teaching styles similar in approach. In sharing information about the instructor demographics, some information has been omitted to comply with IRB requirements to preserve instructor anonymity in this study.
In total, 29 class sessions of more than 35 min were recorded. Those class sessions that were not recorded or included shortened recordings were attributed to: 1) in-class exam days in which recordings were not made, 2) classroom activity days in which a large portion of the class involved student talk/group work, 3) unexpected absence of the graduate teaching assistant who was doing the recording, or 4) recording equipment failure.
Defining Instances of Instructor Talk
The first step in analyzing Instructor Talk was to define inclusion/exclusion criteria and what would qualify as an instance. We began by reading transcripts and highlighting any language that 1) was spoken by an instructor, 2) was addressed to the class as a whole, 3) was not specific to course content, and 4) was not an analogy for course content. We later added a fifth criteria excluding course logistics or agenda items at the start or end of class, because these items did not contribute to the goals of this study (e.g., “Your homework is due Thursday at midnight” or “Today we will be talking about …”). However, in rare instances, we identified examples of Instructor Talk that occurred while describing a homework due date or agenda item. This was included in our analysis because additional information was given (e.g., “Your homework is due Thursday at midnight, because we wanted to give you a whole week to complete the assignment”).
Identifying Emergent Categories of Instructor Talk
Categories were developed using a grounded theory approach to identify emergent themes (Strauss and Corbin, 1990). The goal was to broadly identify Instructor Talk that was not biology-content specific and characterize the types and prevalence of this language. Transcripts from a subset of videotaped class sessions were analyzed, and quotes were chosen independently by two authors (S.B.S. and K.D.T.). Major themes were discussed, and initial categories were developed collaboratively (SEPAL). Additional transcripts were analyzed by one of the two coders (S.B.S.) to further refine categories and were discussed between the two coders. Once categories were relatively stable, a third coder (A.L.R.) analyzed all class session transcripts independently and identified quotes from each of the 29 class sessions and final codes were assigned (S.B.S.) using the developed categories and subcategories. In this process, two additional subcategories were developed, and all quotes were reanalyzed to determine their fit within the new subcategories.
To the extent it was possible, we cut quotes to include a single type of language that fit a specific category and subcategory of the Instructor Talk framework. While some quotes may fit into multiple subcategories, we were generally able to identify a single category that best fit the spirit of the quote. Instances in which it was extremely difficult to choose a single category and the quote could not be cut into two parts represent <10% of the data set.
Measuring Interrater Reliability
To determine the clarity of the categories and subcategories, we asked an additional coder (J.N.S.)—who was familiar with the project but had not participated in the initial coding or category development—to code 10% of the data set. This coder was given a brief explanation of the categories and subcategories and was then given a list of 66 quotes chosen by a random number generator. At the category level, the two raters showed 85% agreement, and at the subcategory level, 75% agreement. This includes agreement on five quotes for which the second coder identified two codes for a single quote. To determine the statistical reliability of agreement, we measured Cohen’s kappa at the category level (0.80) and the subcategory level (0.74), with results that indicated a high level of agreement between coders.
Performing Comparative Statistical Analysis
For instructor comparisons of category and subcategory usage, Pearson’s chi-square analysis was completed. For comparisons of instructors’ average number of quotes per class session, two-tailed t tests with a Mann-Whitney posttest were used. Statistical analysis was performed using JMP 11 software. To normalize for differences in the number of class sessions for Instructor A and Instructor B, we present all variances as SEMs.
RESULTS
This analysis of Instructor Talk was designed to identify the language used by college biology instructors within their classes, beyond the language concerning the biology they teach. Below we present results that yield both qualitative and quantitative data that characterize 1) the emergent categories and subcategories of Instructor Talk, 2) the prevalence of Instructor Talk throughout the semester, and 3) the similarities and differences in Instructor Talk used by two different instructors. The figures and tables presented here are organized to describe the categories and subcategories of Instructor Talk and to demonstrate the relative prevalence of these categories and subcategories by class session as well as by instructor.
Emergence of Five Major Categories of Instructor Talk
Table 1 shows the five categories of Instructor Talk identified in a semester-long introductory biology course. Within each of the five categories, between two and five subcategories of Instructor Talk emerged. The relative quantities of Instructor Talk instances in each category (Figure 1A) and subcategory (Figure 1B) are shown as a percentage of total quotes for the semester. In total, 666 quotes were identified over the course of 29 recorded class sessions. Below, we detail the categories, subcategories, and overall quantitative prevalence of each.
| Category | Subcategory |
|---|---|
| Building the Instructor/Student Relationship |
|
| Establishing Classroom Culture |
|
| Explaining Pedagogical Choices |
|
| Sharing Personal Experiences |
|
| Unmasking Science |
|
Figure 1. Quantitation of categories and subcategories of Instructor Talk. The percent of total quotes for the semester that fell into each of the (A) five categories and (B) 17 subcategories of Instructor Talk are shown. Bar color/pattern differentiates the categories of Instructor Talk. Building the Instructor/Student Relationship (black bars), Establishing Classroom Culture (diagonal bars), Explaining Pedagogical Choices (dark gray bars), Sharing Personal Experiences (vertical striped bars), and Unmasking Science (light gray bars). n = 666 total quotes over 29 class sessions.
Building the Instructor/Student Relationship: An Emergent Category of Instructor Talk
The most prevalent category of Instructor Talk that emerged was Building the Instructor/Student Relationship. In total, 236 quotes were classified as Building the Instructor/Student Relationship, representing 35.4% of the total quotes for the semester. Example quotes from each of the three subcategories of this category are shown in Table 2. The most prevalent subcategory identified was Demonstrating Respect for Students, a subcategory that broadly included quotes in which the instructors explicitly described the ways in which the course or their intentions as instructors were aimed at establishing a respectful and collegial relationship between themselves and the students. A second subcategory of this category was Revealing Secrets to Success, which included quotes focused on making clear to students the actions that might help them to achieve success in the course (e.g., studying with a group or taking notes during class) or in college more generally. The third subcategory of Building the Instructor/Student Relationship was Boosting Self-Efficacy. Quotes in this subcategory included instructors’ compliments for student work or effort and comments bolstering student confidence before exams and important class assignments.
| Subcategory | Example quotes | % Quotesa (n = 236) | ||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Demonstrating Respect for Students |
| 53 (n = 121) | ||||||||||||||||||||||||
| Revealing Secrets to Success |
| 34 (n = 76) | ||||||||||||||||||||||||
| Boosting Self-Efficacy |
| 13 (n = 29) | ||||||||||||||||||||||||
Establishing Classroom Culture: An Emergent Category of Instructor Talk
The second most prevalent category of Instructor Talk was Establishing Classroom Culture. In total, 224 quotes were classified as Establishing Classroom Culture, representing 33.6% of the total quotes for the semester. Example quotes from each of the five subcategories of this category are shown in Table 3. Generally, quotes identified as Establishing Classroom Culture were involved in setting the tone for the course as a whole or for specific activities within the course. The most prevalent subcategory under Establishing Classroom Culture, and also the most prevalent subcategory from any category, was Preframing Classroom Activities. The other four subcategories of Establishing Classroom Culture were: Practicing Scientific Habits of Mind, Building a Biology Community among Students, Giving Credit to Colleagues (both other faculty on campus as well as colleagues within the field of science as a whole), and Indicating That It Is Okay to Be Wrong or Disagree.
| Subcategory | Example quotes | % Quotesa (n = 224) | ||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Preframing Classroom Activities |
| 56 (n = 126) | ||||||||||||||||||||||||
| Practicing Scientific Habits of Mind |
| 18 (n = 40) | ||||||||||||||||||||||||
| Building a Biology Community among Students |
| 11 (n = 24) | ||||||||||||||||||||||||
| Giving Credit to Colleagues |
| 8 (n = 19) | ||||||||||||||||||||||||
| Indicating That It Is Okay to Be Wrong or Disagree |
| 7 (n = 15) | ||||||||||||||||||||||||
Explaining Pedagogical Choices: An Emergent Category of Instructor Talk
The third most prevalent emergent category of Instructor Talk was Explaining Pedagogical Choices, with a total of 123 quotes representing 18% of the analyzed quotes. Example quotes for each of the subcategories of Explaining Pedagogical Choices can be seen in Table 4. Quotes categorized as Explaining Pedagogical Choices generally focused on clarifying for students why both the content and the course structure were relevant to students’ lives and student learning. Quotes within this category fell into one of five subcategories: Supporting Learning through Teaching Choices, Using Student Work to Drive Teaching Choices, Connecting Biology to the Real World and Career, Discussing How People Learn, and Fostering Learning for the Long Term. Please note that the most prevalent subcategory, Supporting Learning through Teaching Choices, included a more general subset of quotes from this category that did not specifically fit into any of the other subcategories.
| Subcategory | Example quotes | % Quotesa (n = 123) | ||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Supporting Learning through Teaching Choices |
| 28 (n = 34) | ||||||||||||||||||||||||
| Using Student Work to Drive Teaching Choices |
| 25 (n = 31) | ||||||||||||||||||||||||
| Connecting Biology to the Real World and Career |
| 22 (n = 27) | ||||||||||||||||||||||||
| Discussing How People Learn |
| 18 (n = 22) | ||||||||||||||||||||||||
| Fostering Learning for the Long Term |
| 7 (n = 9) | ||||||||||||||||||||||||
Sharing Personal Experiences: An Emergent Category of Instructor Talk
The fourth most prevalent category of Instructor Talk that emerged in analysis was Sharing Personal Experiences. In total, 52 quotes were classified as Sharing Personal Experiences, representing 8% of the total quotes during the semester. Example quotes from this category are shown in Table 5. Statements that fit into this category fit one of two subcategories. More general personal statements that seemed to be used by the instructors to share information about their lives with students were coded as Recounting Personal Information/Anecdotes. Quotes in which the instructors shared information specifically from their own experiences as students were subcategorized as Relating to Student Experiences.
| Subcategory | Example quotes | % Quotesa (n = 52) | ||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Recounting Personal Information/Anecdotes |
| 56 (n = 29) | ||||||||||||||||||||||||
| Relating to Student Experiences |
| 44 (n = 23) | ||||||||||||||||||||||||
Unmasking Science: An Emergent Category of Instructor Talk
The final emergent category of Instructor Talk was Unmasking Science. Forty-one quotes were categorized as Unmasking Science, representing ∼6% of the total quotes during the semester. Example quotes from this category are shown in Table 6. Two major subcategories of quotes emerged from quotes in the category Unmasking Science. As a whole, quotes classified as belonging in this category focused on revealing to students what science is about, the kinds of questions science can answer, and the way science is done. The predominant subcategory within this category was Being Explicit about the Nature of Science. The second subcategory, Promoting Diversity in Science, was less prevalent; however, it fit a distinct set of quotes that focused on highlighting the importance of diverse perspectives within the scientific community and promoting to students that all different types of people can and should participate in the doing of science. While there is some similarity in the content of quotes that fit the category Establishing Classroom Culture subcategory Practicing Scientific Habits of Mind and category Unmasking Science subcategory Being Explicit about the Nature of Science, the latter were focused not on what students were being asked to do within the classroom but rather what scientists do more generally.
| Subcategory | Example quotes | % Quotesa (n = 41) | ||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Being Explicit about the Nature of Science |
| 78 (n = 32) | ||||||||||||||||||||||||
| Promoting Diversity in Science |
| 22 (n = 9) | ||||||||||||||||||||||||
Analysis of Instructor Talk Prevalence throughout a Semester-Long Course
Prevalence of Instructor Talk was measured both by the number of quotes identified in each class session and the representation of each of the five categories of Instructor Talk within each class session. Figure 2A shows the prevalence of quotes per class session. The number of quotes ranged from a low of six to a high of 68 within a single class session. On average, there were 26 quotes per class session, with an SEM = 2.9. Figure 2B shows the prevalence of each of the five categories of Instructor Talk as they were represented by one or more quotes in each class session. In 97% of class sessions, quotes were present representing at least three categories of Instructor Talk, while quotes from all five categories were found in 15 of 29 class sessions. Establishing Classroom Culture was the only category for which at least one quote was found in every single class session analyzed.
Figure 2. Distribution of Instructor Talk by class session. (A) The distribution of total number of quotes per class session. ND = no data and signifies that no video of sufficient length for analysis was recorded. (B) The number of categories of Instructor Talk represented in each class session. Building the Instructor/Student Relationship (black bars), Establishing Classroom Culture (diagonal bars), Explaining Pedagogical Choices (dark gray bars), Sharing Personal Experiences (horizontal striped bars), and Unmasking Science (light gray bars).
Analysis of Instructor Talk from Two Instructors
For analysis of instructor-specific differences in the use of Instructor Talk, class sessions in which the two instructors cotaught (n = 4) were excluded from comparative analysis. Therefore, this analysis focuses on the 25 class sessions in which Instructor A (n = 17) or Instructor B (n = 8) taught individually.
Analysis of Category Usage by Instructor
Figure 3A shows the percentage of class sessions taught by a single instructor that included quotes from each of the five categories. Both instructors used all five emergent categories of Instructor Talk at some point during the semester. Examples of each category were found in more than one-third of all class sessions taught by either Instructor A or B. A Pearson’s chi-square test was used to measure differences between Instructor A and Instructor B. These differences in usage by instructor were limited to two of the five categories: Explaining Pedagogical Choices (p ≤ 0.05) and Sharing Personal Experiences (p ≤ 0.001). For all other categories, Instructors A and B showed no significant differences.
Figure 3. Comparison of category use by two instructors. (A) The percent of class sessions in which Instructor A (dark gray bars) or Instructor B (light gray bars) used each of the five categories of Instructor Talk. (B) The percent of class sessions in which each instructor used each of the 17 subcategories of Instructor Talk. Pearson’s chi-square tests were used to compare differences between Instructors A and B. *, <0.05; **, <0.01; ***, <0.001.
Figure 3B shows the percentage of class sessions in which each subcategory was used by Instructor A and Instructor B. Both instructors used 13 of the 17 emergent subcategories, while we identified four subcategories that were unique to Instructor A. These four subcategories were: Building a Biology Community among Students, Discussing How People Learn, Relating to Student Experiences, and Promoting Diversity in Science. The most prevalent subcategory, Preframing Classroom Activities, was used by both instructors in every class session taught. Differences between category usage by instructor were measured by Pearson’s chi-square test (p < 0.05), showing significant differences between the two instructors for seven of the 17 subcategories.
Analysis of Quantity of Instructor Talk by Instructor
Despite similarity at the level of category usage for three of the five categories of Instructors Talk, Instructors A and B differed significantly in the quantity of Instructor Talk used per class session. On average, Instructor A used an average of 28 (SE = 2.6) quotes per class session (n = 17 class sessions), while Instructor B used an average of 10 (SE = 0.7) quotes per class session (n = 8 class sessions), showing a significant difference in the overall quantity of Instructor Talk (p < 0.001). For three of the five categories of Instructor Talk, significantly more quotes were identified for Instructor A than for Instructor B per class session (Figure 4A; all p < 0.05); however, there was no difference in the two instructors’ use of the categories Establishing Classroom Culture or Unmasking Science.
Figure 4. Comparison of quantity of Instructor Talk use by two instructors. Comparison of the amount of Instructor Talk per class session for Instructor A (dark gray bars) and Instructor B (light gray bars). The average number of quotes per class session for each (A) category and (B) subcategory of Instructor Talk. Error bars represent SEM. Mann-Whitney t tests were used to compare differences between Instructors A and B. *, <0.05; **, <0.01; ***, <0.001.
The comparative analysis of the use of each subcategory by Instructors A and B is shown in Figure 4B. There were significant differences in the quantity of quotes used by Instructors A and B for nine of the 17 subcategories (all, p ≤ 0.05). This result shows distinct profiles of Instructor Talk for each of the instructors measured.
DISCUSSION
The critical role of the teacher in student learning has been repeatedly demonstrated in education research (Darling-Hammond and Youngs, 2002). While there is widespread agreement that instructors’ verbal cues play a key role in student learning, systematic analysis of the noncontent language used by college biology instructors has surprisingly gone unexamined. Our results demonstrate the richness of Instructor Talk in an undergraduate biology class and illuminate what two biology instructors said beyond the disciplinary content of their course. In addition, we present an initial framework for future analysis of Instructor Talk and the role it may play in fostering or impeding student resistance, stereotype threat, and student learning. A combination of qualitative and quantitative approaches was used to identify categories of Instructor Talk, characterize the prevalence of these categories throughout a semester, and identify similarities and differences between multiple instructors’ use of this non content-directed classroom talk. Below we will discuss seven main findings from our analyses that put this work into context with other studies of instructor verbal behavior, draw conclusions from the findings, and consider future directions for measuring Instructor Talk.
Instructor Talk Was Present in Every Class Session Examined, Regardless of Instructor
While we chose a course in which we anticipated that Instructor Talk would be richly represented, we did not predict that we would find as many different types of this language as we found, nor did we anticipate that we would find multiple categories of this noncontent language in every class session. There were at least six instances of Instructor Talk identified in every class session analyzed (Figure 2A), with at least two distinct categories of Instructor Talk represented (Figure 2B). This indicates that Instructor Talk is quite prevalent, both within a single class session and throughout a semester. Previously, it has been suggested that it is important to “frame” active learning on the first day of class (Science Education Initiative, 2013) or throughout the term (Silverthorn, 2006); however, the Instructor Talk described here goes far beyond these ideas to describe the timing and types of talk used throughout a course. Further characterization of Instructor Talk will be required to determine the specific impacts of different levels, types, and timing of Instructor Talk.
While Amount of Instructor Talk Differed by Instructor, Both Instructors Used Similar Types of Instructor Talk
For many categories and subcategories of Instructor Talk, the absolute number of quotes differed for Instructor A and Instructor B. However, there was a surprising similarity in the categories and subcategories used by these two instructors during the course. While you might imagine a scenario in which the noncontent biology talk of two instructors would be mutually exclusive, we found a strong overlap in the categories and subcategories used by the two different instructors in our study. Both instructors used all five categories of Instructor Talk, and 14 of the 17 subcategories were commonly used, as well. It is possible that these similarities are due to common messaging or mimicking of the two instructors within a common class. Despite the similarities in category usage, there were interesting differences between the two instructors in their quantity of Instructor Talk. There were nearly three times as many instances of Instructor Talk in class sessions taught by Instructor A as compared with Instructor B. Differences in the quantity of Instructor Talk between different instructors may be important in correlating Instructor Talk with student outcomes like student resistance, stereotype threat, and instructor immediacy. In fact, we hypothesize that subcategories like Preframing Classroom Activities, Building a Biology Community among Students, and Indicating That It Is Okay to Be Wrong or Disagree might be important for overcoming student resistance. Subcategories such as Boosting Self-Efficacy, Revealing Secrets to Success, and Promoting Diversity in Science may play a role in overcoming stereotype threat. Additionally, subcategories including Relating to Student Experiences, Recounting Personal Information/Anecdotes, and Demonstrating Respect for Students might contribute to increased instructor immediacy. Future studies of Instructor Talk with larger numbers of instructors may be able to identify specific footprints of Instructor Talk that characterize specific instructors or correlate specific subcategories of Instructor Talk with a variety of outcome measures.
Some Aspects of Instructor Talk May Be Key to Minimizing Student Resistance to Innovative Teaching
While the Instructor Talk framework broadly characterizes the different types of noncontent language used in the course under study, several specific subcategories may be important for mitigating student resistance to active learning. The most prevalent subcategory of Instructor Talk used by these two instructors was Preframing Classroom Activities, which includes language defining for students exactly what they are expected to be doing and for how long. Research into the phenomenon of social loafing—a term used to describe unfair group interactions in which some students do more work than others—would suggest that it is critically important for students to know what they are expected to do during group activities (Pfaff and Huddleston, 2003). For some students, there may be additional barriers to embracing active learning and participating in group activities, such as fear of interacting with classmates or concern about not knowing the correct answer. A second subcategory, which may play an important role in developing a classroom culture of participation, is Building a Biology Community among Students. Indeed, a pilot study of instructors who self-reported techniques for mitigating student resistance or promoting student buy-in to active learning found that more than 80% reported framing the culture/dynamics of the course (Science Education Initiative, 2013). A third subcategory of Instructor Talk that may influence student participation is Indicating That It Is Okay to Be Wrong or Disagree. Understandably, many students may not want to share their ideas, because they fear being wrong personally or getting the wrong information from another student. Language in this subcategory may signal to students that they are safe taking risks, discussing challenging conceptual ideas, and struggling with difficult ideas together. A recent report from the National Research Council describes common student concerns surrounding the sharing of ideas, including shyness and anxiety about having the wrong answer and talking with someone else who is incorrect (Kober, 2015). While the interviews that revealed these concerns are informative, tools to systematically measure student resistance across large numbers of courses are still needed to uncover the correlations between Instructor Talk and the presence of student resistance in a course. This may be a key to determining what types of Instructor Talk might successfully attenuate student resistance.
Specific Subcategories of Instructor Talk Have Important Links to the Literature on Overcoming Stereotype Threat
Within the broad framework of Instructor Talk presented here, we hypothesize that specific subcategories may play a critical role in constructing inclusive learning environments that have the potential to overcome stereotype threat. Both instructors in this study used language that fit the subcategories Boosting Self-Efficacy and Revealing Secrets to Success, while only one of the two instructors used Promoting Diversity in Science. All three of these subcategories of Instructor Talk may be linked to issues of stereotype threat, because quotes in these subcategories focus on creating an inclusive learning environment that signals safety for all students in the classroom. Research into stereotype threat and methods for mitigating it have demonstrated the importance of even a few critical words when administering a high-stakes assessment. Spencer et al. (1999) demonstrate the impact of stereotype threat on women’s math performance and then use language within the instructions to either induce or overcome the threat, resulting in significant differences in women’s performance on difficult math tests. Future studies with additional instructors could measure correlations between these types of Instructor Talk and stereotype threat. It may also be the case that future studies of Instructor Talk identify stereotype threat–inducing language that unintentionally triggers stereotype threat situations. While this type of Instructor Talk was not detected in the present study, one wonders whether or not the language used in a classroom by instructors could actually be driving differences in the ability to detect an achievement gap in particular science courses (Lauer et al., 2013; Madsen et al., 2013; Eddy and Hogan, 2014). Understanding both stereotype threat–mitigating and stereotype threat–inducing Instructor Talk could be a key missing piece of faculty development for thoughtful instructors with aspirations toward more inclusive teaching (Tanner, 2011).
Emergent Categories of Instructor Talk May Promote Student Learning through Increased Instructor Immediacy
Several Instructor Talk subcategories that emerged from our analysis may impact student learning through increases in instructor immediacy. First characterized by social psychologists, instructor immediacy–perceived social connection between instructor and students—has been linked to a variety of factors, including gains in student motivation, confidence, and learning. The second most prevalent subcategory identified in our analysis was Demonstrating Respect for Students, with quotes from both instructors emerging from nearly every class session. One might imagine that explicitly demonstrating respect for students would positively impact students’ views of the instructor. Indeed, studies of effective college teachers have identified specific teacher behaviors that contribute to student motivation, including respect/concern for student interests (Gorham and Christophel, 1992). Other subcategories of Instructor Talk may be important for both instructor immediacy and student learning. Both of the subcategories from the category Sharing Personal Experiences may help students feel more socially connected to their instructors by helping students get to know their instructors’ experiences as students (Relating to Student Experiences) and their lives more generally (Revealing Personal Information/Anecdotes). The connections between Instructor Talk and instructor immediacy are relevant because of the demonstrated link between instructor immediacy and student learning (Witt et al., 2004). Despite this link, aspects of instructor immediacy such as Instructor Talk have not yet been studied or included in faculty professional development. Therefore, future studies could measure correlations between prevalence of Instructor Talk and either instructor immediacy or student learning.
The Instructor Talk Framework Will Evolve and Change with Analysis of Additional Courses/Instructors
The Instructor Talk framework presented here is based on the analysis of a single class taught by two instructors and will no doubt change as additional instructors are included and quotes that fall outside the established categories are identified. This analysis identified five categories and 17 subcategories of Instructor Talk with striking similarity in category usage between two different instructors. Further analysis of Instructor Talk from other instructors and courses may yield additional categories and/or subcategories of Instructor Talk that build and enrich the rubric for analysis. In addition to identifying potential new categories of Instructor Talk, analysis of additional instructors would allow for correlation studies to begin to identify the types of Instructor Talk most beneficial for minimizing student resistance, enhancing instructor immediacy, and improving student learning for all students. We also anticipate that there will be additional categories of Instructor Talk that work in opposition to these goals. While we did not identify instances of negative Instructor Talk in this class, we hypothesize that such examples will likely be identified in future studies. The emergence of negative Instructor Talk would also be an important to characterize and would likely play a role in the correlation with instructor immediacy, student resistance, and student learning, albeit in the opposite direction.
The Instructor Talk Framework May Be a Critical Missing Piece of Professional Development
Instructor Talk may be a key factor in effectively training faculty to use student-centered teaching approaches. However, it is unclear to what extent current professional development activities for faculty engage them in reflection on the noncontent language used in their courses. To what extent do instructors say things to build community among students? To motivate students leading up to an exam? Or express to students why they teach the way they do? To what extent do instructors plan and/or monitor what they will say to students before they walk into the classroom to teach? While this is the first analysis that has broadly characterized this type of language, the initial framework for Instructor Talk presented here is based on the identification and characterization of more than 600 quotes used by instructors in an introductory biology course. Providing opportunities for instructors to reflect on their teaching practice has been extensively used to train K–12 teachers and could be useful in engaging college and university instructors in examining the role of noncontent language in their teaching, as well (Calderhead, 1989; Hatton and Smith, 1995). The results presented here may be useful in beginning conversations with instructors—especially those who are struggling or experiencing student resistance—about the potential role of noncontent language in successful implementation of student-centered teaching.
Limitations of the Instructor Talk Framework
While this research constitutes an initial description of the Instructor Talk framework, there are limitations related to the nature of qualitative coding and the limited scope of the study. First, a limitation with any type of coding of qualitative data is interpretation and category assignment. We have tried to capture here the broad emergent categories of language; however, we may have missed particular categories of language, and we made purposeful decisions to exclude practical instructional language such as assignment due dates and comments about course logistics. Additionally, while our interrater reliability is quite high, instances of Instructor Talk can occasionally be perceived differently by different coders. Second, there are potentials for error in data collection. For example, there were some times within the course when the recorder was not turned on. Though nearly 70% of class sessions were recorded, it is possible that we missed some critical or unique types of Instructor Talk, especially on exam days. Third, this initial description of Instructor Talk was by necessity limited to a single course with only two instructors. Future studies of Instructor Talk will no doubt uncover additional categories of Instructor Talk, those that both promote and inhibit desired student outcomes. For example, in this particular course, we saw no examples of what we might consider diminishing or threat-inducing Instructor Talk containing microaggressions or other such negative Instructor Talk. Finally, we must note that, while it is necessary to share examples of the raw data, we caution readers that the Instructor Talk examples are highly integrated into a specific course. The tables and quotes presented are not intended to imply that instructors should go out and use the exact language presented here in other contexts.
CONCLUSIONS
To conclude, this research is an initial description of Instructor Talk, a potentially key classroom variable that has yet to be systematically studied. Our findings provide a framework that may be used to investigate noncontent instructor language in a variety of class types, a framework based on qualitative and quantitative analysis of more than 600 quotes from a semester-long introductory biology course. This initial description of Instructor Talk enables multiple lines of research to investigate the relationships between levels and types of Instructor Talk and a variety of classroom phenomena, such as student resistance, stereotype threat, and the role of instructor immediacy in student learning. For example, to what extent does use of certain categories of Instructor Talk correlate with low or high levels of student resistance in the context of reformed teaching environments? Are achievement gaps and indicators of stereotype threat correlated with minimal Instructor Talk or as yet undiscovered negative forms of Instructor Talk? To what extent do high levels of Instructor Talk increase students’ sense of instructor immediacy and, in turn, promote learning gains? And can professional development with instructors and manipulation of their Instructor Talk dramatically shift their success in implementing innovative teaching strategies?
ACKNOWLEDGMENTS
This project was supported with funds from the National Science Foundation (grant numbers 1226361 and 1226344) and the Howard Hughes Medical Institute (grant number 52007556). The authors thank all members of the Science Education Partnership and Assessment Laboratory (SEPAL) for helpful suggestions and comments on the manuscript.
