Scientific Teaching: Defining a Taxonomy of Observable Practices

INTRODUCTION

Recognizing the importance of undergraduate science ­education, national organizations have issued dozens of reports over the past several decades calling for dramatic alterations to undergraduate curricula and teaching methods. Written by scientists, educators, and policy leaders, these reports have three recurrent themes. First, they propose that students should learn about the nature of science and engage in scientific practices (American Association for the Advancement of Science [AAAS], 1989, 2011; National Research Council [NRC], 1999). Second, they stress the need to incorporate learning principles from the cognitive sciences and student performance data in the ongoing development of teaching methods (NRC, 2000, 2003b, 2012). Finally, they call attention to the persistent achievement gap for members of historically underrepresented groups and recommend teaching practices that promote the success and persistence of all students (NRC, 2011; President’s Council of Advisors on Science and Technology [PCAST], 2012).

These calls have had broad impacts within the life sciences community, serving as the impetus for local and national transformation efforts. In 2003, the National Research Council’s BIO2010 report initiated an important movement within biology education by specifically calling for a professional development workshop to help faculty members cultivate their teaching skills (NRC, 2003a). In 2004, this call was answered through the founding of the Summer Institutes on Undergraduate Education in Biology (SI) with support from the Howard Hughes Medical Institute (HHMI) and the National Academy of Sciences (NAS; Pfund et al., 2009). Initially focused on biology faculty members at research institutions and held only once per year, the SI has since expanded to seven regional sites, and more than 1000 faculty members have attended the 5-d workshop as of 2014. These faculty members are primarily—but not exclusively—biologists, and they represent more than 200 institutions from across the country, including 2- and 4-yr colleges and almost all of the nation’s research-intensive universities. SI participants are trained to develop, implement, and disseminate innovative teaching practices at their home institutions, leading to an extensive network of faculty members who have been influenced by the SI program.

The SI curriculum promotes a pedagogical framework called Scientific Teaching (ST) (Handelsman et al., 2004), an approach described in a book by the same name (Handelsman et al., 2007). Reflecting the national calls from which the SI emerged, ST builds on the foundational idea that the way science is taught should reflect the way science is practiced (AAAS, 1990). ST aims to capture the spirit of scientific research by immersing students in the scientific discovery process and using evidence, either local or published, to justify the selection of teaching methods (Cross and Steadman, 1996; Angelo, 1998; Hutchings and Shulman, 1999; Handelsman et al., 2002).

ST encompasses three central tenets: active learning, assessment, and inclusivity. Active learning refers to exercises in which students do something (e.g., writing, discussing, solving, or reflecting), rather than passively listening to a lecture (Crouch and Mazur, 2001; Prince, 2004; Michael, 2006; Wood, 2009; Osborne, 2010). Assessment can be used during a learning event (formative assessment) or at the completion of a unit (summative assessment), in each case providing information to students and instructors regarding student progress (Black and Wiliam, 1998; Tanner and Allen, 2004). Inclusivity embodies the idea that undergraduate science courses contain students of diverse backgrounds and that conscious efforts are required to achieve course environments that minimize potential biases and promote the success of all students (Milem, 2001; Tanner and Allen, 2007).¹ In the past decade, ST has spread throughout the biology education community, providing an overarching framework for biology education research projects and serving as the basis for a number of professional development workshops (Ebert-May and Hodder, 2008; Miller et al., 2008).

The increasing prominence of the ST approach has created a specific need to identify and define its core elements and supporting practices. ST represents a specific articulation of key educational principles pertaining to undergraduate science instruction. It is consistent with broader consensus reports, but it is distilled and packaged in a manner suitable for biology faculty with little pedagogical training. ST implementation involves complex human behaviors modified by social interactions, classroom environments, and task characteristics, such as the content and cognitive demand of a given activity (Hora and Ferrare, 2013). ST can be embodied to different degrees, with one practitioner incorporating a short classroom activity, another revamping an entire course according to the ST paradigm, and both self-reporting as engaging in ST. Some ST practices are readily apparent in the classroom environment, while other important elements are less visible, occurring behind the scenes as an instructor makes plans and adjustments throughout the semester. Several observation protocols have been developed to document classroom practices, but the degree to which they align with ST has not been defined (e.g., Piburn and Sawada, 2000; Hora and Ferrare, 2013; Smith et al., 2013). Furthermore, students can engage in course-related activities either during class or outside class through homework, projects, online forums, or other exercises. For these reasons, future efforts to study ST implementation and associated student outcomes will require systematic definition of ST in a way that accounts for its diverse applications.

Taxonomy development has been used as a research methodology in many disciplines to clarify and elaborate overarching processes, structures, and goals. For example, within the medical education community, taxonomies have been used to better describe medical errors as well as to specify desired competencies for medical residents (Zhang et al., 2004; Graham et al., 2009). Often adopting a hierarchical organization, taxonomies use explicit criteria to systematically identify, classify, and define elements that fit within a broader structure. Specifying a given domain through taxonomy development is recommended as preparation for curriculum building, program evaluation, and instrument construction (Chatterji, 2003). In addition to informing these activities, taxonomies can also guide future research efforts by summarizing current understandings and providing a defined reference point for future studies (Bordage, 2009).

In this article, we describe the development of a taxonomy that operationalizes ST principles through the specification of observable teaching practices associated with ST. We detail the process underlying its initial construction and iterative revision, and we report on classroom observations demonstrating the applicability of the taxonomy within the course context. We also provide data from faculty surveys supporting the comprehensiveness of the taxonomy. The resulting taxonomy identifies core pedagogical goals of ST and articulates specific practices aligned with each goal. By defining observable indicators of ST practice, this taxonomy provides a common framework for researchers studying the processes and outcomes of ST-based educational transformations as well as an important resource for faculty members engaged in course transformations. This taxonomy can also serve as the basis for formal instruments designed to document the implementation of ST within a course.

METHODS

Project Scope

The overall goal of this project was to make explicit the pedagogical goals of ST and to develop a list of observable practices supporting these goals (Figure 1). Using the book Scientific Teaching as the primary guide (Handelsman et al., 2007), we sought to define ST elements that apply within the undergraduate course context, acknowledging from the outset that the resulting product would not include all valuable teaching practices or address other important considerations surrounding higher education. For example, though firmly rooted in the principles of teaching inclusively and embracing diversity, ST does not extensively address issues of affordable access or student disabilities. These issues fall outside the purview of a typical instructor, being governed largely by institutional policies and student support offices. There are also general behaviors associated with quality teaching (e.g., speaking clearly, being organized, or maintaining professionalism) that are not explicitly emphasized in ST.

Identification of Pedagogical Goals

The first part of the taxonomy development process involved identifying ST’s core pedagogical goals. Here, pedagogical goals are defined as learning processes, course structures, and classroom environments that are desired by the instructor. We began by deconstructing the book Scientific Teaching into a comprehensive list of its recommended teaching practices, using other education literature to elaborate certain topics (e.g., see references included in Table 1). We next identified the central intentions and features of each teaching recommendation. For example, the specific suggestion to employ group problem solving in the classroom could be generalized into two components: having students work together and having students solve problems. These broad components were further consolidated based on related features into a list of core pedagogical goals, which were subsequently refined and translated into student-centered terms.

Elaboration of Supporting Practices

To operationalize ST in an explicit manner, we further described each pedagogical goal in terms of specific practices that support its achievement. For example, one goal is for students to “learn to think metacognitively,” a process that can manifest itself in many ways and be elicited by different kinds of activities. To elaborate each goal, we articulated a general approach to encapsulate the different ways that the goal could be achieved and then compiled a series of supporting practices that exemplify each general approach. Many supporting practices were found within the Scientific Teaching book, and we drew from our collective experiences as students and instructors to supplement these practices. Once a draft list had been developed, we conducted informal classroom observations of ST-trained instructors to identify additional practices that had been potentially overlooked. Several rounds of iterative revisions ultimately led to the production of a complete draft taxonomy.

Testing the Applicability of the Draft Taxonomy

Like many educational approaches, the nature of ST precludes definition in absolute or authoritative terms. The growing community of practitioners who implement and disseminate ST shapes its features in an ongoing manner. In light of these qualities, we conducted classroom observations to determine whether the supporting practices listed were in fact detectable within classroom environments. The intent of this exercise was not to use the taxonomy as a measurement tool but rather to ensure that the supporting practices we had compiled were being implemented in practice. Ten faculty members from biology and other science disciplines at the University of Colorado were recruited by email and observed for one class meeting each (Table 2). These faculty members were primarily, but not exclusively, former SI participants and/or had a reputation for utilizing transformed teaching practices. For each observation, two investigators recorded field notes regarding student and instructor activities. After class, each investigator separately determined which of the initial taxonomy’s 38 supporting practices occurred at least once during the class. Initial interrater agreement was 84%, and consensus was reached on the remaining items through discussion. Importantly, 89% of the supporting practices (34 of 38) were scored at least once during this series of classroom observations. These observations suggest that the supporting practices in the draft taxonomy represent a reasonable list of behaviors that are both used and observable.

As a supplement to classroom observations, we also conducted an online survey to determine the extent to which a sample of faculty members would recapitulate our list of supporting practices. Fourteen additional faculty members, including 11 from other institutions, who had completed SI or other similar training were recruited by email to complete a 30-min online survey administered via Qualtrics (Table 2). In the survey, participants were presented with a series of pedagogical goals along with corresponding general approaches and were asked to describe practices they use to achieve these goals in their classes. Participants were given five text-entry boxes for each goal as well as an additional box for general comments. For prevention of survey fatigue, each participant was presented with roughly half of the 17 different goals from the draft taxonomy. From this survey, we collected a total of 288 reported practices, which were subsequently reviewed to determine alignment with the existing supporting practices.

The faculty surveys allowed us to address two principle objectives: to determine whether the supporting practices on the draft taxonomy were similar to those generated by an independent group of faculty members and to identify additional supporting practices reported by faculty members. Two investigators worked together to determine the degree of alignment between the so-called faculty-generated (FG) practices and the existing practices by determining whether each FG response qualified as a general restatement or a specific example of an existing practice based on related keywords, themes, and characteristics. For example, when presented with the goal of students “using science process skills,” the FG response of having “students generate hypotheses and make predictions” was judged to be analogous to the existing practice of having “students identify, construct, or evaluate hypotheses and make predictions based on their hypotheses.” One investigator initially aligned the FG practices to the existing supporting practices, a second investigator reviewed all of the assignments, and then the two investigators discussed any disagreements. Again, 89% of the existing practices (34 of 38) were aligned with one or more FG responses, providing confirmation that the existing practices could largely be corroborated by faculty practitioners.² Faculty members employ similar practices to achieve certain pedagogical goals, and many supporting practices were therefore corroborated by several FG responses. While the majority of the FG practices could be paired with an existing practice, roughly 10% (28 of 288) did not align with an existing practice for the pedagogical goal under which they were originally submitted. Among these, some aligned with other pedagogical goals, while others were deemed to be outside the scope of ST. The few remaining FG practices were added during a final round of taxonomy revisions.

Final Taxonomy Revisions

While the draft taxonomy (with 38 practices) showed considerable alignment with observed and reported teaching practices, our efforts revealed a few areas for further revision. Language throughout the taxonomy was clarified to be more parsimonious, including the merging of two pairs of pedagogical goals based on overlap within faculty responses (e.g., faculty members did not make distinctions between collaborative and cooperative learning approaches). Four supporting practices were removed because they were redundant with other items, occurred outside the observable course context, or were not reported by faculty members. Three new supporting practices were added to reflect previously unlisted FG practices. As an example, for the goal of “affirming students of diverse backgrounds,” multiple respondents mentioned “employing mechanisms to enhance diversity within student groups.” This practice was added to the taxonomy. After final taxonomy revisions, only one supporting practice remained that had not been observed in the classroom or mentioned on faculty surveys. This was having “students analyze data using appropriate methods.” Because this practice had been identified in national reports as an important component of developing science process skills (NRC, 2003a; AAAS, 2011), it was retained in the final taxonomy (37 practices altogether).

RESULTS

Taxonomy Structure

The final ST taxonomy consists of a series of 15 pedagogical goals, 15 general approaches, and 37 supporting practices arranged in a hierarchical manner (Table 3). The taxonomy operationalizes ST by identifying its core elements and elaborating explicit behaviors, artifacts, and features associated with each element. Several aspects of the taxonomy reflect the ongoing evolution of ST since its original publication. For example, while ST was developed within the context of biology education, the taxonomy maintains its applicability throughout the sciences by utilizing interdisciplinary language. Furthermore, the SI curriculum and Scientific Teaching book are geared toward instructors, and they fittingly describe actions that instructors can take to build productive learning environments for their students. In contrast, the taxonomy is phrased with an explicit focus on student actions and perceptions. This student-centered language is not intended to diminish the importance of the instructor role but rather to emphasize that ST is ultimately about what students do and perceive.

While ST is traditionally defined according to the general tenets of active learning, assessment, and inclusivity, the taxonomy is divided more specifically into four sections pertaining to course alignment, science practices, student participation, and cognitive processes. The course alignment section focuses on the interrelation of three different curricular components: learning goals, instructional activities, and summative assessments. Learning goals define what students should know and be able to do upon course completion, instructional activities provide students with opportunities to accomplish this learning, and summative assessments gauge the degree to which students achieved the original goals. ST advocates that instructors explicitly communicate their learning goals to ensure that students are aware of the conceptual understandings, content knowledge, and process skills they are expected to master. ST also stresses the requirement that activities and assessments must align with course learning goals to provide suitable avenues for intended learning and accurate measures of student achievement. This alignment creates an important feedback loop that enables instructors to use student data to help make decisions on how to spend valuable class time and improve their teaching methods.

The science practices section elaborates the idea that undergraduate science courses should capture the spirit of scientific discovery by engaging students in scientific processes. This differentiates ST from more general pedagogical approaches (e.g., student-centered learning or team-based learning) that do not explicitly focus on incorporating scientific reasoning. By articulating that students should explore the relationship between science and society, engage in experimental design and interpretation, and participate in formal scientific discourse, this section addresses the dual intentions of preparing scientifically literate citizens as well as training future scientists.

The student participation section of the taxonomy contains pedagogical goals related to how students participate within a course. These goals embody a constructivist approach by acknowledging the importance of student knowledge and the value of enabling students to build their own mental models through active engagement and peer feedback. One pedagogical goal describes involving students at the whole-class level through the use of classroom response systems such as clickers or the reporting out of group work. Another goal in this section serves to help students feel affirmed as members of the class and the larger scientific community, irrespective of their backgrounds or future career aspirations. Instructors are challenged to demonstrate “cultural competence” by adopting teaching materials and practices that help students from diverse backgrounds self-identify as science practitioners (Tanner and Allen, 2007).

The final section of the taxonomy focuses on different types of cognitive processes that should be cultivated. For students to be innovative and productive members of the future workforce, they must be able to apply, analyze, synthesize, and evaluate disparate pieces of information and to remember and comprehend essential definitions and processes (Bloom et al., 1956; Anderson and Krathwohl, 2001). The increasingly interdisciplinary nature of modern research fields has created demand for the ability to integrate concepts across disciplinary boundaries. Furthermore, students need to be encouraged to develop metacognitive habits that allow them to self-reflect in order to optimize their problem-solving and study skills (Tanner, 2012).

To visualize the relationship between the taxonomy and ST’s traditional tenets, we categorized each supporting practice as supporting active learning, assessment, or inclusivity (Figure 2). Not surprisingly, we found that there was extensive overlap between these tenets, with most supporting practices addressing multiple tenets. Scientific Teaching itself suggests that active learning and assessment are inextricably linked, in that active learning incorporates assessment of student understanding, and assessment necessarily elicits active student engagement. This is borne out in the taxonomy, as there are 27 practices that address both active learning and assessment. ST is intended to be inclusive, and there is growing evidence that implementation of interactive teaching practices and the development of science process skills can have beneficial outcomes for members of traditionally underrepresented groups (Dirks and Cunningham, 2006; Haak et al., 2011; NRC, 2012). Within the taxonomy, we found that 21 supporting practices were related to inclusive teaching. While these designations are not intended to be absolute, they do serve to illustrate the relationships between the traditional ST tenets.

ST consolidates research-based practices from across the science education literature, but it contains particular terminology and emphases that make it unique. After constructing the final taxonomy, we sought to align the ST taxonomy to the broader education literature by collecting reviews and research articles describing and justifying ST approaches. Table 1 contains a listing of several references related to each pedagogical goal. These citations support the consistency of the ST pedagogical goals with broader educational dialogue. Most articles apply to more than one pedagogical goal, and so the designation of an article under a particular goal does not negate its applicability to other parts of the taxonomy. For example, in addition to an overall focus on helping students develop higher-order cognitive skills, the Blooming Biology Tool described by Crowe et al. (2008) has applications for course alignment, activity development, assessment, and student metacognition. We hope that this table will provide a helpful starting point for practitioners wanting to learn more about ST’s research foundations.

DISCUSSION

In response to ongoing national calls, ST was formulated as a way to help scientists bring their expertise into the classroom in more authentic and productive ways to improve student learning. With numerous copies of Scientific Teaching in circulation and hundreds of educators being trained in ST-based programs each year, ST has achieved significant influence within the education community. Future efforts to monitor ST’s use and impact will depend on having ways to identify its application. In this study, we have operationalized ST practices through the development of a taxonomy that identifies the core goals of ST and defines observable practices associated with each goal, thus providing faculty members with a concise, inclusive reference guide representing key ST elements.

To achieve the goal of defining ST in explicit terms, we took the viewpoint of an objective observer with course access similar to that of a student (i.e., someone who can attend class, download course materials, take exams, etc.). While many aspects of ST remain hidden from such an observer, we attempted to identify how these hidden elements are manifested. For example, ST advocates the use of a curriculum design process called backward design (Wiggins and McTighe, 2005). During the backward design process, an instructor first drafts course learning goals, then considers evidence that would indicate student mastery of the learning goals, and finally designs instructional activities that allow students to reach the desired performance level. While this design process provides a valuable guide, it is not possible to objectively determine whether an instructor’s method was in fact “backward” without being present throughout course construction. Nonetheless, the desired outcome of a rational course design can be judged through the provision of written learning goals to students and alignment of subsequent activities and assessments to these goals.

The ST taxonomy is designed to have broad relevance within the context of undergraduate science courses. ST can be used in many different parts of a course, including lecture, lab, recitation, homework, and online forums. The taxonomy describes student actions and experiences that can occur within any of these different settings and includes practices that are found across a large range of course sizes. While some ST practices are easier to implement in smaller classes, all of the practices listed in the taxonomy are feasible in a large-enrollment class, particularly with the aid of instructional technologies or additional personnel (Caldwell, 2007; Otero et al., 2010). Finally, the taxonomy is intended to be applicable across the sciences, and efforts were made during the validation process to include perspectives from non–biology disciplines. However, given ST’s historical roots and current prominence within biology, it remains possible that the taxonomy does not fully account for teaching practices unique to other disciplines. Understanding these pedagogical differences remains an important research question, particularly in light of ongoing efforts to develop interdisciplinary courses and learning environments (Meredith and Redish, 2013; O’Shea et al., 2013; Thompson et al., 2013).

The taxonomy addresses two important issues related to conducting research on the use of transformed teaching practices. First, previous efforts to gauge teaching practices have been criticized for an overreliance on self-reported data, which may not accurately reflect actual classroom practices (Kane et al., 2002; Ebert-May et al., 2011). Second, many previous studies on the implementation of research-based instructional practices have focused on the use of specific strategies (e.g., clicker questions) that each contain a number of different components (e.g., question content, peer discussion, group sharing, etc.). Instructors tend to adapt these strategies according to their own classroom needs, resulting in a wide range of different practices that recapitulate the original design with varying degrees of fidelity (Turpen and Finkelstein, 2009; Dancy and Henderson, 2010). By specifying a comprehensive list of practices indicative of ST, the taxonomy begins to address each of these issues by laying the groundwork for the development of observation-based rubrics that separately identify the different layers present within any given course exercise.

While the items on the taxonomy are written in explicit terms, the taxonomy is not intended as a formal observation protocol for classroom evaluation. We anticipate that the taxonomy will serve as the basis for the development of such instruments, which will require additional delineation of scoring mechanisms, response scales, and measurement criteria. The taxonomy is consistent with the frameworks underlying several existing observation protocols, but also identifies elements unique to ST. Developed for K–12 classrooms, the Reformed Teaching Observation Protocol, or RTOP, captures the student-centeredness of a classroom, and most of its items are consistent with the goals and practices listed in the ST taxonomy (Piburn and Sawada, 2000). The Teaching Dimensions Observation Protocol, or TDOP, is based on systems-of-practice theory and accounts for different dimensions of the classroom environment, including teaching methods, pedagogical strategies, student–teacher interactions, cognitive engagement, and instructional technology (Hora and Ferrare, 2013). The TDOP thus includes codes that capture elements of ST as well as practices beyond the scope of ST. Developed under the TDOP framework, the Classroom Observation Protocol for Undergraduate STEM, or COPUS, focuses on faculty and student behaviors, but does not include codes to capture course alignment, science practices, or cognitive processes (Smith et al., 2013).

In its current form, the taxonomy has several applications for instructors and departments working to improve their educational programs. First, individual faculty members can use the taxonomy to 1) self-assess their own teaching and identify ways to diversify their courses; 2) report and justify the use of transformed teaching practices to promotion and tenure committees; and 3) communicate the rationale behind instructional decisions to students (e.g., on the first day of class in discussing course goals and format). Second, as a professional development tool, the taxonomy can be used to informally document classroom practices and to facilitate dialogue with course instructors (see Supplemental Material 1 and 2). Third, the taxonomy can provide a basis for pedagogy-related conversations at the departmental level: 1) departments engaged in curricular reviews can identify when, where, and how their students are addressing each pedagogical goal within their program; 2) faculty members can use the taxonomy as a common frame of reference drawn from the education literature to guide formative peer feedback (Gormally et al., 2014), rather than invoking outdated or subjective ideas of what constitutes “good” teaching; and 3) since the content of the taxonomy complements the NSF Pulse Fellows Vision and Change rubrics, it can be used as part of departmental efforts to self-evaluate awareness, acceptance, and use of transformed teaching practices (Aguirre et al., 2013).

While the ST taxonomy has numerous applications for researchers and instructors with an ST background, we propose that the taxonomy is also useful for individuals engaged in other reform initiatives. The language of the taxonomy is general in nature and captures many key ideas related to educational reform. Thus, we foresee the taxonomy serving as a theoretical underpinning for researchers studying the outcomes of general professional development workshops or other efforts of a more comprehensive nature. We also expect that the taxonomy will be useful for introducing instructors to different aspects of course transformation and providing a concise representation of research-based educational practices. The instructional uses described above would apply equally well for faculty members who have received ST training, non-ST training, or no educational training.

The taxonomy presented in this article identifies and defines observable practices associated with ST. It is not expected that all the goals listed on the taxonomy would be addressed in a single class period or assignment, as the suitability of these practices depends on the scope and goals of a course. Finally, the taxonomy represents an articulation of ST in its current state. Staying true to ST principles, the taxonomy should evolve as future research efforts lead to a deeper understanding of how different teaching practices affect student outcomes. Please contact the corresponding 
author (B.A.C.) if you would like a copy of the taxonomy that has been formatted as a one-page handout.

FOOTNOTES

ACKNOWLEDGMENTS

REFERENCES