Increased Learning Observed in Redesigned Introductory Biology Course that Employed Web-enhanced, Interactive Pedagogy

INTRODUCTION

When most departments at our university decided to require a biology course in their curricula, we projected that annual enrollment in Introduction to Biology (BIOL 1010) would go from 100 to perhaps 1000 in several years. The biology department responded to this challenge by designing a one-semester course appropriate for all students, including life science majors. We decided to replace the traditional instructor-centered, lecture-homework-exam pedagogy with a student-centered, Web-enhanced, interactive pedagogy. Because the same instructor who had taught BIOL 1010 since 1976 would be one of those teaching the new course, we had a unique opportunity. We could attempt to establish whether or not the Web-enhanced, interactive pedagogy improved learning by assessing learning gains the last time he taught the traditional course in fall 2003 and comparing them with gains obtained when he taught the interactive course to a similar population of students in fall 2004.

In the natural sciences, physics educators were among the first to effectively embrace interactive learning, including peer teaching, as a supplement to traditional lecturing (Mazur, 1997; Hake, 1998; Thornton and Sokoloff, 1998; Atkin and Black, 2003). With the development of the Halloun-Hestenes Mechanics Diagnostic test (MD; Halloun and Hestenes, 1985) and the Force Concept Inventory (FCI; Hestenes et al., 1992), physics educators had a means to determine if teaching innovations improved student understanding of Newtonian mechanics. Hake (1998) used data from MD and FCI in 14 traditional and 48 interactive-engagement physics courses to establish that learning gain doubled in those classes that used an interactive-engagement pedagogy.

Roy (2003) reported significantly higher learning gains in the genetics and evolution course he taught with an interactive format than the one he taught with a lecture format that included interactive lecture demonstrations. In a developmental biology course, Knight and Wood (2005) obtained significantly higher learning gains with student participation and cooperative problem solving than in a course with the same syllabus taught in a lecture-based format. Knowledge surveys (a comprehensive set of questions that students do not answer but indicate their perceived ability to answer) provided similar results (Wirth and Perkins, 2005). Wirth and Perkins established that knowledge surveys accurately measure what students know and used them in several geology courses to demonstrate that in courses employing problem-based and collaborative learning, students learned substantially more than students in traditional lecture-based courses.

These and other positive outcomes from nontraditional, interactive approaches not only motivated us to create an interactive BIOL 1010 course but also to seek a means for assessing learning as we switched pedagogies.

MATERIALS AND METHODS

Traditional Course

In the traditional BIOL 1010 course (T-1010), students were given three 50-min lectures each week that were well illustrated with 35-mm slides and sometimes a video. At the beginning of each class, students were asked for questions. In any given week, only a few questions were asked and often they were administrative. On occasion questions were asked during lecture usually by the instructor but sometimes by a student.

Each lecture had a reading assignment from the text and, for a few lectures, a scientific article. Each week students were required for credit to submit answers to nine, short-answer, essay questions (three questions per lecture). These homework questions were worth 200 of the 600 points available in the lecture part of the course. Although many students completed all assignments because the best 10 of 14 homework assignments were used in grade determination, fewer students turned in assignments in the last four weeks of class. Because ecology was given last, fewer students submitted ecology assignments than evolution assignments for credit: 48% of ecology assignments versus 97% of evolution assignments. Two exams and a comprehensive final exam were given. They contained mostly short essay questions with some fill-in-the-blank, diagram, and definition questions. Evolution, the first topic covered, was tested in an exam, whereas ecology, the last topic covered, was tested only in the final exam. Separate from the regular class a review session was given by the instructor before each exam and the final.

Although attendance at lecture was not required, students had to personally turn in weekly homework assignments on Monday and pick up the next week's assignment on Friday. Attendance was highest on Fridays and Mondays (85 to almost 100%) and lowest on Wednesdays (65–90%). All students also had a required laboratory that was project-based and took ∼5 h/wk. Life science majors comprised 63% of the 102 students in the class. Table 1 summarizes this course and the redesigned course.

Web-enhanced, Interactive Course

In creating the Web-enhanced interactive course, we modified the content of T-1010 by dropping direct consideration of development, behavior, and physiology and spending equal time on just four areas: evolution, genetics, molecular-cell biology, and ecology that we considered critical for all students. We used our WebCT-administered and interactive ecology, and genetics and evolution courses as models on which to build a Web-enhanced, interactive BIOL 1010 (WEI-1010).

Rensselaer uses WebCT as its online course management system. WEI-1010 is totally integrated into WebCT. Online pre- and postclass activities are accessed by students through WebCT. All students have laptop computers in class for Web-based activities. After class, students have online access via WebCT to everything that was presented in class. All sections of WEI-1010 use the same materials that are available on the WebCT BIOL 1010 site.

In WEI-1010, students have two, 110-min, in-class sessions per week. Every in-class session has online preclass and postclass sessions that require ∼60 min each. Students are given three exams and a comprehensive final composed of multiple-choice questions. As in T-1010, evolution is the first topic and tested with an exam. And ecology is the last topic and tested only in the final exam. Differing from T-1010, however, separate pre-exam review sessions are not held by the instructor. Seventy-five percent of the students in fall 2004 also took the same laboratory as did T-1010 students. The other students did not have a laboratory. Sixty-two percent of the 56 students were life science majors.

All preclasses have a short essay question to be answered online. All pre- and postclasses have four multiple-choice, review questions. The short essay and review questions are evaluated for credit and are worth 200 of the course's 600 points. Students quickly realize that they must do essays and reviews to earn a respectable grade. Students begin preclass activities with a brief video intended to create interest in the session's topic (e.g., natural selection, species diversity, DNA structure). The video is followed by a series of activities that include an essay, review questions, and two to four of the following: online experiment or simulation, Web investigation, data analysis, short article on classic and recent discoveries, and textbook reading. These activities are integrated topically and prepare students for the upcoming in-class period.

In-class sessions involve active student participation, and attendance is required. Although a student was permitted three absences, the majority had no absences and only a few had more than one absence. The in-class activities de-emphasize lecturing. Many of the activities employ interactive peer teaching (Mazur, 1997). In these activities students work in small groups (2–5) answering questions, interpreting data, discovering concepts, resolving differences, and learning vocabulary by using it. The instructor and teaching assistants are more guides and coaches than purveyors of facts and knowledge. For example, in an interactive-learning activity (modeled after the Interactive Lecture Demonstration used first in physics classes; Sokoloff and Thornton, 1997) students are given an observation or a set of data and asked to answer a particular question(s). They work first individually and then as groups before reporting to the class. Concepts and topics are also explored in interactive discussions, mini-tutorials, simulations, and Web-based research exercises. Learning is monitored and enhanced by concept queries using a WebCT tool that is similar to “clickers” now used by many instructors. (Note: we have changed to clickers because they are much easier to use than the WebCT tool.) Concept queries are multiple-choice questions that are first answered individually without discussion. Student responses are then compiled and shown to the class. If not answered correctly by every student, the concept query is posed a second time, and answers are solicited after being discussed in small groups to resolve misunderstandings. These peer-based discussions are facilitated by teaching assistants and the instructor. After the in-class period students engage a set of postclass, online activities similar to those in preclass, but with fewer activities. These activities iterate or expand on concepts and information previously presented.

Most of the in-class activities and many of the pre- and postclass activities are interactive in nature. They have, or at least encourage, the student to engage actively with the material. For example, pre- and postclass review questions can be answered correctly only if the student has done the reading, a simulation, or other activity. The online essay questions also relate to the out-of-class activities that must be done if the student is to write a meaningful answer that merits credit.

A concurrent second section of WEI-1010 with 58 students was taught in fall 2004 by another instructor using the same materials, and both sections took identical exams. The 114 students in these two sections matched the population of students that have historically taken BIOL 1010: about two-thirds life science majors with the other third being a mix of other majors, many of whom were interested in medical school. Students randomly chose a section and were similar in ability and background, as indicated by the equivalent scores on identical exams. Exam grades in the two sections were as follows: evolution, 18.2 and 19.2; genetics, 16.2 and 16.3; molecular-cell biology, 17.2 and 16.6; final, 53.2 and 51.9; and total points, 104.8 and 104.0. This second section's instructor elected not to have his students take the evolution and ecology concept inventories because they took too much class time.

Evolution and Ecology Concept Inventories

Many types of formative and summative learning assessments are available (for example, various types of questions such as multiple choice, matching, fill-in-the-blank, true-false, and essay; one-on-one interviews; standardized concept inventories; knowledge surveys; and focus groups). The relative merits of each continues to be debated (Thornton and Sokoloff, 1998; Roy, 2001; Anderson et al., 2002; Udovic et al., 2002; Brewer, 2004; Wirth and Perkins, 2005). We elected to assess learning gain with conceptual questions, following the model of FCI (Mazur, 1997; Hake, 1998; Thornton and Sokoloff, 1998; Cummings et al., 1999).

Although some biology educators have developed assessment tools for their courses, the biology community has not produced assessment tools like FCI that we could employ to measure learning gains objectively (Roy, 2001; Anderson et al., 2002; Udovic et al., 2002; Klymkowsky et al., 2003; Brewer, 2004; Handelsman et al., 2004). As a consequence, we developed evolution and ecology concept inventories to assess learning gains in BIOL 1010. These two subject areas were selected because they were covered in depth in both T-1010 and WEI-1010.

To develop the evolution inventory, we independently compiled lists of evolution concepts. We then made a group list and reconciled it with what was presented in the T-1010 course and what was covered in evolution chapters in several textbooks including Biology: Life on Earth (Audesirk et al., 2002), the textbook used in T-1010. We identified 47 core evolution concepts (see Supplemental Material A). Biology: The Unity and Diversity of Life (Starr and Taggart, 2004) was used in WEI-1010. Both texts are similar in level of presentation and are designed for courses that combine life science majors with students majoring in other disciplines.

Questions related to the identified evolution concepts were written and then vetted with half a dozen colleagues. In making these questions we reviewed and used one question from the Conceptual Inventory of Natural Selection (Anderson et al., 2002). We also drew on the extensive literature on misconceptions in biology (D'Avanzo, 2003; Sinatra et al., 2003). In this way, our 20-question evolution concept inventory was created (see Supplemental Material B). A 17-question ecology concept inventory, based on our list of 38 fundamental ecology concepts, was similarly created (see Supplemental Material C and D).

In the concept inventories given in T-1010, about half of the questions had more than one correct answer. To facilitate machine grading of the concept inventories given in WEI-1010, we modified the format for some questions. If the wording clearly indicated the question had only one answer, it was not changed. If the wording implied more than one answer might be correct, the format was changed so that each multiple-choice answer required a response on the answer sheet. This change makes the concept inventories given in the traditional and interactive courses slightly different in format, but the influence this may have had on the results is likely to be minor because the student had to make the same choices in both formats. The evolution and ecology concept inventories that were used in WEI-1010 are in Supplemental Material B and D, respectively.

If answers were randomly picked, the score in evolution would be 7.95 of 20 and in ecology 7.03 of 17. If students were randomly selecting answers, then half would have scores less than these values and half above these values. In each of the concept inventories, between zero and five students had scores equal to or less than the random-pick score. Chi-square analyses of the predicted and actual distributions of scores indicate students were not randomly answering questions (for all inventories p < 0.001).

Learning Gain

Learning gain is defined as follows: learning gain = (postinstruction score − preinstruction score) ÷ (total score − preinstruction score). For example, a score of 12 on a preinstruction inventory of 22 questions and a postinstruction score of 16 on the inventory would give a learning gain of 0.4 (0.4 = [16 − 12] ÷ [22 − 12]). Thus, the student learned 40% of what could have been learned as measured by the inventory. If the student had gotten all correct on the postinstruction inventory, the learning gain would be 1.0. No student scored 1.0 on either a pre- or a postinstruction inventory. Thus, we did not have a ceiling effect where students maxed-out the learning gain that the inventory could measure.

Learning gain was calculated in T-1010 and in WEI-1010 for each student who had taken the pre- and postinstruction evolution and ecology inventories. These individual learning gains were averaged to give the course learning gain. Students did not know a concept inventory was being given when they came to class, and no credit was given for taking the inventory. Learning gains were calculated in T-1010 at the end of the semester. In WEI-1010 gains were calculated at the end of the semester and at the time of the final exam. For the final exam, concept inventory questions were imbedded in the exam and were graded for credit. Students did not know the inventory questions would be part of the final.

RESULTS

The preinstruction scores for evolution were T-1010 = 11.15 ± 2.16 (mean ± SD; n = 64) and WEI-1010 = 10.95 ± 2.21 (n = 49; Table 2). For ecology they were T-1010 = 10.28 ± 1.56 (n = 83) and WEI-1010 = 9.73 ± 1.54 (n = 54). A two-tailed, two-sample, equal variance t test indicated that the T-1010 and WEI-1010 preinstruction scores for evolution were not significantly different (p = 0.64), and for ecology they were just significantly different (p = 0.04). Thus, students in the two classes started with very similar understandings in evolution, and the 2003 class knew a little more ecology than did the 2004 class.

In T-1010 the end-of-the-semester conceptual learning gains were evolution, 0.10 ± 0.20 (mean ± SD; n = 64); and ecology, −0.05 ± 0.25 (n = 83; Table 2). A t test indicated that T-1010 preinstruction scores for evolution were significantly different from end-of-the-semester scores (p = 0.008), establishing that students knew more evolution concepts at the end of the semester than they did at the beginning of the semester. In contrast, a t test indicated that T-1010 preinstruction scores for ecology were not significantly different from end-of-the-semester scores (p = 0.69), establishing that students did not know more ecology concepts after instruction.

In WEI-1010 the end-of-the-semester gains were evolution, 0.19 ± 0.24 (n = 49); and ecology, 0.14 ± 0.21 (n = 54). As indicated by t tests, WEI-1010 preinstruction scores for evolution and ecology were significantly different from end-of-the-semester scores, establishing that students knew more evolution and ecology concepts at the end of the semester than before instruction (evolution, p = 0.00007; ecology, p = 0.0003).

Evolution and ecology learning gains were significantly greater in WEI-1010 than in T-1010: evolution, p = 0.024; and ecology, p = 0.000009.

Plots of preinstruction scores versus learning gains for T-1010 and WEI-1010 do not show a strong correlation for either evolution (Figure 1) or ecology inventories (Figure 2). All four linear regressions have negative slopes, indicating that students who knew less at the beginning learned more than those who knew more. This pattern was observed in both T-1010 and WEI-1010 and was more pronounced in ecology. Because the line slope is substantially steeper for T-1010 ecology than for WEI-1010 ecology, the traditional course appeared to facilitate learning more than did the interactive course for students who knew little ecology at the outset. However, the distribution of students with a positive learning gain was similar in both courses for students who had lower preinstruction ecology scores, whereas the data show that more students in T-1010 than in WEI-1010 who had higher preinstruction scores also had negative learning gains. Apparently many students with better preinstruction scores not only learned little but also were confused more by instruction in T-1010 than in WEI-1010. As a result, they had negative learning gains, causing the slope to be steeper in T-1010 than in WEI-1010.

When learning gains were measured from concept inventories imbedded in the WEI-1010 final exam, learning gains increased 95% in evolution and 143% in ecology when compared with learning gains measured with the concept inventories given in the last week of classes: evolution, 0.37 ± 0.19 (n = 49); and ecology, 0.34 ± 0.21 (n = 54; Table 2). As indicated by t tests, these learning gains in both evolution and ecology were significantly different from those measured in the last week of classes (evolution, p = 0.0001; ecology, p = 0.000003).

DISCUSSION

When our biology department decided to radically change the pedagogy used in BIOL 1010, a considerable body of research indicated that the student-centered, interactive activities we proposed to develop would improve learning (Hake, 1998; Mazur, 1997; Thornton and Sokoloff, 1998). The question remained, would learning really be enhanced in our particular case? In the absence of established biology concept inventories, we created sets of evolution and ecology concept questions to measure learning gains objectively.

Learning gain for the evolution component of our traditional biology course was 0.10. A significantly higher gain (0.19, p = 0.024) was obtained in the redesigned, Web-enhanced, interactive course. Likewise the learning gain for the ecology component of the redesigned course was also significantly improved (T-1010, −0.05; WEI-1010, 0.14; p = 0.000009). The interactive pedagogy used in WEI-1010 is likely responsible for some of the increased learning.

The most interesting data in this regard is the lack of learning in ecology in the traditional course (−0.05) when compared with significant learning in evolution in the same course (0.10). Why did the traditional format lead to learning in evolution but not in ecology? The substantially reduced amount of interactive engagement with ecology concepts is one of two major factors that differ in comparison with evolution. In the traditional course, students did not have an exam on ecology during the semester, whereas they did have an exam that covered evolution. Preparing for an exam is a time when students actively engage the subject and learn. Students also actively considered concepts when doing homework essays, but because of the requirement that only the 10 best of 14 assignments were used for grade determination, far fewer students did ecology homework than evolution assignments. Ninety-seven percent of evolution assignments were done for credit, whereas only 48% of the ecology homeworks were. Thus, before taking the postinstruction ecology inventory, attending lecture (along with associated reading, if done) was the primary exposure students had to ecology concepts.

The second major difference was when ecology was presented. Evolution came first in the semester when students are fresh, whereas ecology was presented at the end when students were tiring of school and were burdened with end-of-semester projects and papers. The when-presented factor appears, however, to be significantly less important than active engagement. Students in the WEI-1010 course had evolution first and ecology last, as well as no exam on ecology. Yet they had a significant ecology learning gain (0.14, p = 0.0003). The lack of learning in ecology in T-1010 (p = 0.64) indicates that lectures and possibly the associated reading, in and of themselves, are not a very effective vehicle for learning in an introductory course.

The significant increases in learning gains in the redesigned course are consistent with results from other studies indicating that interactive courses are considerably more effective than lecture-based courses (Mazur, 1997; Hake, 1998; Thornton and Sokoloff, 1998; Roy, 2003; Knight and Wood, 2005; Wirth and Perkins, 2005). However, in addition to the interactive pedagogy associated with both out-of-class and in-class activities, six other factors may have contributed some to the increased learning gains observed in WEI-1010.

First, frequency of assessment was greater in WEI-1010: three versus two exams as well as two essay questions and 16 multiple-choice review questions versus nine essay questions per week. In addition, in-class concept queries and questioning of students by instructor and teaching assistants in WEI-1010 were informal assessments not present in the traditional course. This increased assessment in WEI-1010, as well as the interactive nature of much of this assessment, likely enhanced learning (Casem, 2006).

Second, concepts were iterated more times and in a wider range of contexts in WEI-1010 than in T-1010. In WEI-1010 a particular concept was presented in a variety of ways: video, short article, simulation, text reading, essay question, and review question (pre- and postclass) and class discussion of student essay, interactive discussion, mini-tutorial, concept query, interactive-learning activity, and online activity (in class). A concept was usually not presented in all of these ways, but for a major concept many were used. Thus, students actively engaged with a major concept between half a dozen and a dozen times. In contrast, students in T-1010 were exposed to a concept only several times, and the exposures were in one active engagement format (homework questions) and in two mostly passive activities (lectures and reading assignments). This substantially increased iteration in WEI-1010 that required active engagement likely enhanced learning.

Third, WEI-1010 students had more time-on-task. Additional time was spent by WEI-1010 students in class: 70% more in evolution and 25% more in ecology (Table 1). Students in WEI-1010 spent about an hour on each pre- and postclass session and additional time preparing for exams. T-1010 students spent substantial time doing homework questions and preparing for exams. Although we have no hard data for either class, it is our assessment from talking with students that the interactive course demanded more out-of-class time than the traditional course. Certainly more time on a topic increases the likelihood of more learning, especially if students were actively engaged as was the case here.

Fourth, students in WEI-1010 were guided and paced in their learning, whereas students in T-1010 set their own agendas. T-1010 students had required homework questions due once a week, whereas students in WEI-1010 had pre- and postclass activities for each in-class session. These online assignments were focused on specific concepts that were iterated or enhanced with activities in the recent or up-coming in-class session. Students were accountable for answering a question with a short essay in the preclass session and for doing a set of four, multiple-choice review questions in pre- and postclasses that required a student to engage actively with the pre- or postclass activities. This directed focus and iteration in an interactive manner likely increased learning.

Fifth, attendance was required in WEI-1010 but not in T-1010. Students in WEI-1010 were permitted three absences, but most students missed no classes and just a few had more than one absence. Attendance in T-1010 ranged from 65% to almost 100%, and class attendance averaged ∼80%. The higher percentage of students present in WEI-1010 probably had some effect on the learning gains measured because a higher percentage was present to do activities than attended lectures.

Sixth, exams in T-1010 were not multiple choice, whereas those in WEI-1010 were. Because concept inventories were multiple choice, this type of exam question may have improved the performance of WEI 1010 students because they were conditioned to the multiple-choice assessment format. We think this difference is likely to have minimal influence on the learning gains measured because students are skilled at taking multiple-choice exams because they have been tested with multiple-choice questions for essentially all their time in school.

It is impossible with the data available for us to assess the relative contributions of these six factors to the increased learning gains measured. The situation is also confounded because increased interactive engagement with a concept is a substantial component of most of these differences. However, the lack of learning gain measured for ecology in T-1010 (−0.05) in contrast to the evolution learning gain of 0.10 in T-1010, along with the ecology learning gain of 0.14 in WEI-010, indicate that the interactive pedagogy is likely to be an important factor, if not the most important factor, in the increased learning we measured. The ancient assessment of how people learn likely applies here: tell me and I will forget, show me and I will understand, have me do and I will know.

Students were given no credit for concept inventories but rather were verbally encouraged to do their best “because the inventories were important.” In addition, students were not told in advance that an inventory would be given. It was therefore expected that when the concept inventories were part of the final exam, learning gains would be substantially higher; in fact, they doubled (evolution, 0.19–0.37; ecology, 0.14–0.34). Three factors contributed to this result: students had prepared for the final exam, were more motivated because the score was part of their grade, and were seeing the questions for a third time. We do not have data to assess the relative importance of these factors. It is, however, likely that the first two factors were major contributors to the significantly increased learning.

CONCLUSION

The traditional instructor-centered, lecture-based pedagogy used in BIOL 1010 yielded small or no learning gain. The implementation in BIOL 1010 of student-centered, Web-enhanced, interactive pedagogy played an important role in increasing learning gains significantly in both evolution and ecology. Our results strengthen the case for augmenting or replacing instructor-centered teaching with interactive, student-centered pedagogy, and they are consistent with a large body of literature on the effectiveness of active student participation for enhancing learning (Mazur, 1997; Hake, 1998; Cummings et al., 1999; Udovic et al., 2002; Atkin and Black, 2003; Knight and Wood, 2005; Wirth and Perkins, 2005).

REFERENCES