Biological Inquiry: A New Course and Assessment Plan in Response to the Call to Transform Undergraduate Biology

INTRODUCTION

The path for transforming undergraduate biology education was laid out in Vision and Change in Undergraduate Biology Education: A Call to Action (American Association for the Advancement of Science, 2009) and in BIO2010 (National Research Council [NRC], 2003), and similar calls for reform are affecting all science, technology, engineering, and mathematics (STEM) disciplines (e.g., NRC, 1996, 2009; National Science Foundation [NSF], 1996; DeHaan, 2005; Association of American Medical Colleges and Howard Hughes Medical Institute [AAMC and HHMI], 2009; Woodin et al., 2009). However, traveling the path of reform requires serious effort, including engaging with the research on how people learn (e.g., Bransford et al., 2000), adopting active-learning pedagogies that “develop the habits of mind that drive science” (Handelsman et al., 2004), developing students’ core skills through engagement in authentic, open-ended research, and being more intentional about providing the context for the role of science in addressing current societal problems (e.g., Labov et al., 2010). In addition, we must become more intentional in our efforts to reach out to learners from nonmajority groups. Because traditional teaching practices “worked” in decades past for those in the present professoriate, it can be a challenge to convince current educators of the cryptic obstacles that impede the success of groups that do not fall into the dominant group. While challenging, this work is being inspired through developing the educators’ understanding that science itself is socially constructed and therefore influenced by financial, political, and social privilege (Shaw et al., 2009).

There is growing evidence that the calls for reform are paying dividends, as evidenced by stories of new courses that embrace best practices (Herron, 2009; Mathews et al., 2010) and reports of effective methods of assessing higher-order skills, such as scientific reasoning (Timmerman et al., 2010) and critical thinking (Bissell and Lemons, 2006). Several organizations of practitioners, including Science Education for New Civic Engagements and Responsibilities at (SENCER), Project Kaleidoscope (PKAL), Science Education Resource Center (SERC), Engineering Projects in Community (EPICS), and the BioQUEST Curriculum Consortium, are dedicated to sharing best practices, developing faculty skills, and motivating and guiding curriculum reform, and these groups and others have informed our work at Wofford College.

CONTEXT FOR CURRICULUM REFORM IN BIOLOGY AT WOFFORD COLLEGE

The apparent success of existing structures creates real obstacles for extensive curriculum reform, and several members of our biology department quite reasonably argued for incremental change, rather than transformation of our curriculum. Biology is Wofford's largest major (approximately 20% of all graduates), and our alumni (surveyed biennially over the past 16 yr, most recently in 2006 with a 70% response rate) unanimously report that they are well prepared for graduate and professional programs (e.g., 42, or 13%, of all 2010 graduates went on to medical or dental school). Therefore, gaining buy-in for curriculum transformation required other internal and external evidence and motivation.

To appreciate the magnitude of our reform effort, it is important to understand where we started. For at least the past 30 yr, the required core curriculum for all biology majors consisted of a four-semester sequence of courses that spanned the freshman and sophomore years. Zoology was followed by Botany, Genetics, and Cell Biology. Zoology was a traditional, textbook-driven, phylum-by-phylum survey course. This course was generally popular among the first-year students who completed the course (although it is important to note a 21% attrition rate), and students praised it on course evaluations for its concreteness, the clarity of lectures and notes, adherence to material in the textbook, and predictability of material for exams. The very characteristics that students praised about the course are criticized in the literature on cognitive research for failing to require higher-order thinking, such as the transfer of knowledge to novel situations (Bransford et al., 2000).

At the same time, our department also staffed biology courses strictly for students pursuing the BA degree (including many of the prospective science majors who dropped out of Zoology and out of the BS track). These courses fulfilled one of the two laboratory-science course requirements, one in the life sciences and one in the physical sciences, specified in our General Education (GE) program. Professors were mostly autonomous in their approach to these BA-track science courses, as long as their courses covered the scientific method and a representative subset of topics in the discipline. As a result, these courses ranged widely in content, teaching pedagogies, and rigor. Although some were exemplary by all assessment measures, particularly those that were developed as part of Wofford's Learning Communities program (Goldey, 2004), the ubiquitous use of the diminutive term “baby science” for these BA-track courses underscored the opportunity to raise expectations for student engagement and performance.

An internal review of Wofford's GE program in 2005–2006 revealed that our introductory-level courses, whether for BS or BA students, did not target the goals outlined in our GE program. The College catalogue states: “Wofford's [GE] program seeks to develop skills in reading, written and oral communication, use of technology, critical thinking, creative expression, numerical reasoning, problem solving, and collaborative and independent learning.” While these goals are all consonant with a strong foundation in scientific literacy, prior to the creation of Biological Inquiry, few (if any) of these competencies were targeted learning outcomes of introductory biology courses, whether for majors or nonmajors.

Graduating seniors, surveyed from 2005–2007, indicated that the biology curriculum had contributed little to their development of quantitative and writing skills, and their ratings of the first-year core courses were significantly lower than for upper-level courses. Similarly, a department-by-department analysis of Wofford's results from the National Survey of Student Engagement (NSSE) revealed our majors were writing less and placing more emphasis on lower-order cognitive skills than students in many other departments.

Motivated by our internal assessment evidence and inspired by the national calls for transformation of STEM education and improving liberal education (Association of American Colleges and Universities, 2008), we submitted a proposal to and received a grant from the NSF for the project “Biological Inquiry: A Model Course and Assessment Program” to develop, implement, and assess a completely new course for incoming students (combining BA- and BS-track students). This funding was a primary force in overcoming lingering resistance to such a major reform effort and provided nominal financial compensation for the remarkable amount of time and energy required for such work.

BIOLOGICAL INQUIRY: COURSE PLANNING AND FACULTY DEVELOPMENT

The Target Audience for Biological Inquiry

Biological Inquiry, first implemented in the Fall of 2009, is taken by a majority of all incoming first-year students at Wofford College (232 of 446 incoming students in Fall 2011), having completely replaced Zoology (173 students in 2008) and our nonmajors course for BA-track students (typically 50 students per year).

Biological Inquiry is taught in nine sections of 24–26 students each, and each section is taught by one of seven faculty members (all full-time, tenure-track appointments). Zoology (the old course) had been taught by two professors with large lecture sections (>80 students) and cramped labs (>36 students), so we have received noteworthy administrative support for our curriculum reform efforts, particularly in the form of two new faculty hires in 2008 and 2009.

The Goals of Biological Inquiry

The goals of Biological Inquiry are to develop core scientific competencies (as defined in AAMC and HHMI, 2009) in all students, while also preparing biology majors for upper-level coursework. Our course builds these competencies (core knowledge and core skills) through a guided-inquiry approach in which students engage in science as it is practiced by professional scientists: reading the primary literature, writing in the style of the discipline, presenting ideas orally, developing and testing hypotheses, conducting authentic research experiments, analyzing data using statistical methods, graphing and interpreting results, thinking creatively and critically, working effectively in teams, and applying knowledge to novel situations and civic problems. As our students build the competencies of scientist scholars, they are also developing the capacities and habits of mind consonant with the liberal arts mission of the College.

It is perhaps helpful to those considering curricular reform to know about an early step in our journey. Five years ago, all members of our department participated in a daylong retreat to start mapping out a new first-year program. The work of the retreat centered on articulating our responses to three questions: what do we want our students to know, what do we want them to be able to do, and what do we want them to care about as a result of this course? This strategy (Suskie, 2009) expands course design beyond core content/textbook coverage to consider the knowledge, skills, and dispositions that we see as most essential for our students, and we found the exercise helpful in identifying shared priorities. Readers will find a list of our responses to these questions in Supplemental Material 1.

Carving Out Time for Course and Faculty Development

We recognized that accomplishing our ambitious goals for Biological Inquiry would require a substantial investment of time and resources devoted to faculty development. In the year prior to implementing Biological Inquiry, we carved out time to work together. In the Spring of 2009, all members of the biology department were free of teaching duties during the Tuesday/Thursday 8:00 am time slot. All full-time members of the 12-member department met every Thursday morning for an 80-min development workshop and/or planning session. For example, we practiced learner-centered pedagogies, reflected on William Perry's scale of cognitive development (Perry, 1970), brainstormed open-ended experiments that would engage our students, and sought consensus about course content. We shared with each other knowledge gleaned from extramural conferences and workshops, and we learned from Wofford colleagues from other disciplines. A psychology professor shared how she teaches first-year students to use statistics; a chemist shared how he uses Process Oriented Guided Inquiry Learning (POGIL; http://pogil.org/about); and a religion professor and biologist led a workshop on ways to move people beyond the dualistic rhetoric that surrounds the teaching of evolution (these professors have led a similar version of this workshop at the SENCER Summer Institute each year since 2008).

We also hosted campus-wide events: Dr. Marc Chun led a “Collegiate Learning Assessment (CLA) in the Classroom” workshop on designing exams and assignments that require higher-order thinking (Chun, 2010), and Dr. Barbara Tewksbury led a workshop on course design (see http://serc.carleton.edu/NAGTWorkshops/coursedesign/tutorial/index.html). These workshops were well attended by faculty members and administrators from across the campus; our project therefore had a broad impact.

Collaborating with Students in Course Planning and Implementation

In the summer preceding the Fall 2009 launch of Biological Inquiry, work (and anxiety) intensified. Seven professors devoted their summer to the project, and for 10 wk, we worked in partnership with four undergraduate students (then rising sophomores): Abbey Ellison, Patrick Harbour, James Mills, and Arsalaan Salehani, who became known as preceptors. These students were invited to work with us based on the exceptional mental agility, strong work ethic, and leadership skills they had demonstrated in their first year. Partnering with preceptors mimics the strategy we employed in the development of our sciences–humanities Learning Communities program (Goldey, 2004), and collaborating with students was once again vital to the success of this project. The preceptors tested out ideas for open-ended experiments, identified course readings, and contributed remarkable energy and creativity to the process. Once they trusted that we valued their input, they became more confident in expressing their opinions, and their insights were invaluable. The preceptors also served as laboratory facilitators in the Fall of 2009, troubleshooting unanticipated problems (there were several), training the other laboratory assistants on each week's activities, and being strong role models for all students. Because the preceptors shared “ownership” of the new course, they also championed it across the campus, thus calming suspicion and anxiety (especially among the students) regarding the curriculum reform. The work of these students was showcased when they presented at a panel during a plenary session at the 2010 SENCER Summer Institute. All four of these students graduated from Wofford in 2012 at, or near, the top of their class, and all remained especially close to the department's faculty members. They also all report that their plan for incorporating research and teaching into their future careers (all are headed to medical school) was inspired by their partnership with us. We believe that such close collaboration with students on curriculum development is just as mutually beneficial to the students and professors as the more traditional model of mentored research.

BIOLOGICAL INQUIRY: COURSE STRUCTURE

Course Technology and Resources

Communication among professors and students in Biological Inquiry is facilitated using Moodle (Modular Object-Oriented Dynamic Learning Environment), which is a free and open-source course-management system (http://moodle.org). We post to the Moodle page the syllabus, all handouts, primary literature, videos, and other supporting documents, and we can easily update these resources daily.

Guided Inquiry and Other Pedagogies of Engagement

Over the past 3 yr, the course instructors have shifted farther and farther from lecture mode to more engaging pedagogies. We have adapted for our course several case studies (e.g., Gulf Dead Zone) from the National Center for Case Study Teaching in Science (http://sciencecases.lib.buffalo.edu/cs). We have modified each case to better fit our needs by adding questions for which students must apply information from their textbook, Campbell Biology: Concepts and Connections, 7th Edition (Pearson Education, New York, NY), and by updating the case resources with new images and findings from primary research articles. The organization of the textbook, which uses distinct, stand-alone sections in each chapter, makes it particularly useful for our needs, as we can easily draw on sections from multiple chapters when creating an assignment.

We have also developed our own case study/guided-inquiry investigations, and the “CLA in the Classroom” workshop noted above was very helpful in building our capacity for this work. Our cases include one in which students use observation, deduction, hypothesis development, and interpretation of actual results from published literature to uncover the role of ventral photophores in some species of bioluminescent marine organisms. In another, students learn how to keep a good laboratory notebook through the hypothetical reconstruction of actual experiments that investigated the role of behavioral and wing pattern adaptations of tephritid flies in protecting them from attack by jumping spiders.

Beginning in Fall 2011, malaria became a recurring topic of study throughout Biological Inquiry; thus, the study of malaria is a tool through which students learn the science and better appreciate the relevance of biology to societal problems. This method adopts the approach advocated by the SENCER project (www.sencer.net), with which we have a decade-long history of close collaboration. Malaria is investigated at a number of different biological levels at appropriate time points throughout the semester. Our course begins its study of biology at the level of ecosystem ecology and ends at the molecular level, so students build their knowledge of ecology through a guided investigation of malaria early in the semester. Students work with their teammates to formulate answers to a series of questions, for which they must synthesize information from multiple sources, including their textbooks (e.g., sections on terrestrial biomes, global climate patterns, and parasitism), figures and graphs from research studies (e.g., distribution and abundance of the Anopheles mosquito and Plasmodium species), and articles investigating anthropogenic factors that contribute to human vulnerability to malaria infection. At the end of the course, when we get into molecular biology, we engage students in guided exploration of the findings from DNA-sequencing research that are uncovering the evolutionary relationships among Plasmodium species and the evolutionary origins of those species that infect humans.

The coevolution of Plasmodium, the parasite that causes malaria, with its mosquito and vertebrate hosts make malaria a remarkably complex and interesting topic for study, and there is abundant research literature to draw from in supplementing students’ exploration of the topic. After we complete revisions to these assignments and test them with our students this next year, we plan to publish the malaria-based exercises and our other guided-inquiry exercises, but we are happy to share them now with interested individuals.

Finally, various professors incorporate a number of other active-learning pedagogies into their classes, including role-play, such as building a DNA molecule using students in the roles of sugars, phosphates, and nitrogenous bases; acting out the very complex life cycle of malaria; and “dancing” the stages of mitosis and meiosis using variable numbers of chromosomes. Students report that such activities make learning fun and that when they recall their role in an activity, it helps them remember its broader context. They also report that it helps them look forward to coming to class.

Building Students’ Teamwork Skills

We want students to appreciate the increasingly collaborative nature of scientific inquiry, so we have made developing effective teamwork skills a priority. Students work in teams throughout Biological Inquiry, and we begin the first lab meeting with an exchange of contact information among team members, followed by “ice-breaker” team-building exercises, after which we brainstorm and discuss some essentials of good teamwork.

Students are assigned to teams by their professor, who attempts to balance the four- to five-person teams as to gender and background in biology and statistics (based on the information provided on a note card that each student fills out during the first class meeting). At midsemester, new teams are formed, and after each iteration, students fill out custom-designed, confidential evaluations in which they are asked to reflect on their own contributions and those of their teammates. We have found students to be remarkably honest about their own performance, perhaps because they know what the other students are likely to say about them. The information provides remarkable insight for the professors to use to guide improvements among diverse learners (i.e., some students need to become less dictatorial and more inclusive, others are shy and may benefit from writing down their ideas prior to their team meeting, and still others are not carrying their share of the workload and often benefit from a one-on-one conversation with their professor to uncover the reasons why).

Developing Students’ Skills in Mastering Primary Literature

Like research scholars, students become familiar with the primary literature related to their topic of investigation prior to conducting each major experiment. Students develop the skills to master published literature through guided practice, and we devote significant laboratory time to this work several times during the semester. During these sessions, each of the six lab teams is assigned a different article relevant to the research topic. The six articles, selected in advance by the instructors, share the following characteristics: overall brevity (most are fewer than six pages), straightforward purpose and experimental design, and results that are depicted in uncomplicated histograms or line graphs. Our decision to preselect the articles solved two serious problems that arose the first year, in which we directed students to find relevant articles and bring them to lab. First, we avoid students selecting articles that would challenge even an expert scholar, and second, because the instructors have read all of the preselected articles in advance of lab, they can readily guide a team when the members run into trouble.

Each team is given about an hour to master its article, and during this time the professor moves among the teams addressing questions that arise. Each team then gives a presentation about their article to the other teams, describing to their classmates the purpose of the research, along with methods, relevant graphs, and main conclusions. Students are encouraged to use a skit and/or props (we provide playdough, pipe cleaners, colored dots, etc.) to help their classmates understand the article. Student feedback has shown that the work is challenging, empowering, and fun (Supplemental Material 2 provides a representative sample of student reflections from the first primary literature lab), and they often refer on exams to topics from these articles when applying their knowledge to novel situations. As an added benefit, we have found that the presentations give students practice in describing an article's findings in their own words, and this helps them avoid plagiarizing the language of the article's authors when they prepare their research posters.

Building Students’ Oral Presentation Skills

The primary literature lab sessions give students practice in public speaking in a nongraded/low-risk setting, and the skit format, in particular, seems to bring out “star” qualities of even the shyest student. Students also gain practice in giving more formal presentations, as each team must present on the ecology, life history, and “social relevance” of the research organism(s) we will be using in their research projects. For example, presentations on Trichoplusia ni and Arabidopsis thaliana are given at the beginning of lab during week 3, just prior to setting up and running the first experiment. When assigning a grade and/or providing feedback for oral presentations, the instructor will often seek comments from the class; asking each student in the audience to provide anonymous feedback (on note cards) in the form of at least one positive comment and one constructive suggestion for improvement.

This same approach is used on a designated lab day later in the semester, in which we set aside about 30 min for each team to swap its poster drafts with another team, and each team provides constructive feedback to the other. Because peer review is one of the hallmarks of good scholarship, we want them practicing this skill, too, and we have seen student comments become more substantial and helpful as the semester progresses. We also believe that this approach lessens the sense of competition among students in favor of a more constructive and collaborative ethos.

Utilizing Open-Ended Research Experiments to Build Science Competencies

Students engage in open-ended research projects, and each project spans several weeks. In the first project, students adapt the methods of DeVos and Jander (2008) to pursue a novel research question that explores the adaptations of plants that defend them against herbivory. We use the pest species Trichoplusia ni (U.S. Department of Agriculture permit #P526P-09-03135) along with different Arabidopsis ecotypes and/or mutants each year. In a later multiweek project, students investigate oviposition choice in bean beetles, Callosobruchus maculates (see Blumer and Beck, 2008; www.beanbeetles.org). In the last major project, students use DNA extraction and sequencing analysis to investigate the validity of species labels on salmon purchased from various vendors (adapted from Kline and Gogarten, 2012).

Each year we modify the independent and/or dependent variables in these experiments to address a novel research question that builds on the prior year's project. This approach keeps the experiments open-ended and interesting, but maintains enough instructor control over methods and protocols that we can be well organized to replicate the experiments across 10 to 11 laboratory sections of 24–26 students each. The instructors have a lot of fun discussing which parameters to investigate each year, and the work is much more intellectually stimulating for both professors and students than the cookbook labs that dominated our old courses.

Practicing the Scientific Method as Research Scholars

Through guided inquiry, we mentor and coach the students through the research process, but the students must work through each step in consultation with their teammates. For example, although we give an overview of the basic experimental protocols, students must describe in their lab notebooks the purpose of the research project, their hypotheses, and their predictions, which they must also sketch out in graphical form in their lab notebooks. Prior to running the experiment, they must also develop a table in their notebooks in which they will enter their experiment's data and in which they clearly label the dependent and independent variables.

Along the way, we may ask teams to share with the rest of their classmates the products of their work at the end of each step, but we avoid giving students answers to their many questions, lest they come to believe that they lack the skills or expertise to move forward or that there is only one right way to approach each step/question. Although many students are initially frustrated with the guided approach and its apparent inefficiency (i.e., reacting with “Just tell me what I need to know” or “Just tell me what I have to do next”), the room soon comes alive with teams discussing their ideas.

After running their experiment and recording the data in their lab notebooks, students also record their teams’ data into a shared computer database so that the data can be pooled across all lab sections. The data are then analyzed and the results interpreted during lab the following week.

Building Students’ Quantitative Skills

Devoting lab time to statistical analysis of the data from the students’ research allows us to focus on building students’ quantitative skills. We provide an overview of the statistical test we will be using (e.g., analysis of variance [ANOVA]), drawing on the custom text written by biostatistician Dr. Clarence “Ab” Abercrombie, a coauthor on this article. In his folksy style, Dr. Abercrombie provides descriptions in his text of inferential statistics, the principles and utility of particular tests, how to interpret a p value, and step-by-step protocols for performing the appropriate analysis using the statistics software we have adopted. The book is available online at www.wofford.edu/biology/quantitativeTechniquesText.

We guide students in using the statistical software program JMP (www.jmp.com), a desktop program developed by the SAS Institute (Cary, NC). We are pleased with the JMP product, and in 2010, we received administrative approval to purchase an institutional site license, eliminating the need for the student edition, which cost each student $14 and expired at the end of 12 mo. The institution-wide license has been especially useful, as we now use JMP in almost all of our upper-level biology courses, and other departments are also beginning to use it.

Using a t test or ANOVA, as appropriate, students first analyze the data collected by the six teams in their particular class/lab section, and then they analyze the pooled data from all course sections (typically 10 to 11). This multilayered approach allows students to visualize how increasing sample size reduces variance around the mean and may increase statistical power. Students also learn the importance of making sure their teams’ data are as accurate as possible, because good-natured teasing occurs when extreme outliers are identified and evaluated.

Once the students have analyzed the pooled data, they must then interpret their results, comparing them with their predictions and considering them in the context of findings reported in the primary literature. This approach has given us valuable insight into students’ misconceptions about statistics; for example, some students assume that getting a low p value is the goal of the experiment, so we find that it is important to guide them past that point with more probing questions about the purpose of their research and its broader relevance (i.e., we ask them to consider the bottom-line question, “So what?”).

Building Writing and Communication Skills through Research Posters

Each team of students prepares a professional research poster to communicate the findings from their research projects. Students work within each of their assigned teams on two such posters, reporting on two of the multiweek projects they conduct. This work introduces students to an important method that professional scientists use to communicate their work to others, gives them practice in writing in the style of the discipline, and engages them in applying to their own work the findings from the primary literature they have read. One poster is due at midterm and the other at the end of the semester, and students are given the opportunity to revise their first drafts after feedback from other teams, their lab assistant, and their professor.

We provide students with a poster template (Supplemental Material 3) and grading rubric (Supplemental Material 4), the latter of which students use to evaluate their progress, while we use it to assign final grades to their work. Qualitative feedback, discussed below (e.g., see Insights from External Evaluator's Assessment), indicates that although the research poster assignments present challenges for both professors and students, these assignments have been very important in driving desired student learning outcomes.

METHODS AND RESULTS OF ASSESSMENT

Our direct-assessment methods include reviewing student work from guided-inquiry assignments in class and lab (described previously), weekly online and/or in-class quizzes, the use of grading rubrics to assign and elucidate for students the rationale for grades on major assignments (e.g., research posters), and in-class exams that ask students to apply their knowledge to novel situations in order to assess higher-order thinking skills (i.e., skills students practice regularly during class, lab, and homework activities). Our indirect-assessment techniques include a customized Student Assessment of Learning Gains (SALG) survey (see www.salgsite.org), reflective essays, and focus groups and interviews conducted by an internal and an external evaluator. Each of our assessment methods and findings is described below.

Findings from Direct Assessments of Student Learning Outcomes

Student Grades.

Although final grades may be a blunt instrument for measuring student learning, they are of keen importance to students, and we discuss them here as one piece of the assessment puzzle we have constructed. There was no statistically significant grade inflation in Biological Inquiry over the old Zoology course it replaced. On a four-point scale, the average grade (±SE) in Zoology was 2.6 ± 0.07 and in Biological Inquiry it was 2.8 ± 0.05; final grade distributions were also remarkably similar between the old and new courses. However, it is worth noting that of the 173 students enrolled in Zoology in 2008, 36 students (21%) withdrew from the course, most often to avoid a poor grade. In Biological Inquiry, student retention has averaged 95% over the 3 yr, despite the fact that students report feeling very challenged by the course (see Findings from Indirect-Assessment Methods).

In the old course, the main tools of assessment were objective quizzes and exams that emphasized recall of large amounts of content, and students who performed poorly or were overly stressed by such tests dropped the course. We speculate that the improved retention for students in Biological Inquiry is due to a variety of factors, including the stronger bonds formed among students and their professors (the guided-inquiry approach breaks down some of the hierarchical barriers between professors and students, because the professor is typically moving among the students rather than staying at the front of the room), as well as the variety of direct assessments that we use to determine grades, which allows students to utilize more than one type of ability.

We work to develop exams that assess the higher-order thinking skills (Bloom et al., 1956; Anderson and Krathwohl, 2001) that students are practicing in class and lab. Although each instructor of Biological Inquiry develops his/her own exams, we share exam questions with one another—especially those questions that require students to apply their knowledge to novel situations. For example, students might be given a figure and caption from a research article they have never seen before and asked to identify the dependent and independent variables, interpret the p value, and/or propose a follow-up experiment to address questions that emerge from the findings. We also included more traditional questions (multiple-choice questions and essay responses to prompts) to assess more basic understanding of course content, as we believe that a blend of styles is appropriate.

Research Posters and Other Assignments That Directly Assess Learning Outcomes.

Other direct assessments include evaluation of the quality of the research posters, and we have provided in Supplemental Material 5 an unedited example of one team's poster that received an “A” grade. While not all of the posters are this good, considering that these students are in their first year of college, we have been pleasantly surprised (sometimes amazed) by the generally high level of work our students have produced. We have noted that the students’ writing improves from the first to the second poster, with even the rough draft of the second poster being notably better than the first.

Other graded assignments include in-class and online quizzes, written responses on in-class guided-inquiry assignments, and the completeness and quality of laboratory notebooks and other homework assignments. Beginning in 2012, we plan to implement “open-book” laboratory exams (midterm and final) for which students can use only their lab notebooks. We believe this will encourage students to keep excellent records in their notebooks and will help us better evaluate who is truly engaged in, and understanding, the various steps of the research process.

Findings from Indirect-Assessment Methods

Use of the Student Assessment of Learning Gains (SALG) Survey.

One of our indirect-assessment methods is a customized SALG survey, which students complete online during the last week of class. The SALG survey has been especially useful, because we developed and used our SALG survey in advance of implementing our new course, so we have been able to compare responses to the survey questions from students who took the old course (Zoology) with responses from students in the new course, and we have also been able monitor changing perceptions of Biological Inquiry in each year it has been taught. Of those students who completed Zoology in 2008, 89% (122) filled out the end-of-course SALG survey. For each year Biological Inquiry has been taught, 80–90% of students (n = 186, 233, and 221 each year, respectively) completed the survey, thus providing a robust sample from each year. Students earn five points toward their final exam if they complete the survey (student responses on the SALG are strictly anonymous, but the SALG website provides a list of those who have taken it), and this very modest incentive (∼0.05% toward their final grade) has proven to be very effective.

Our particular SALG instrument is titled “Biological Inquiry, Bio 150” and is available on the SALG website for others to adapt for their own use. We developed our SALG questions to address students’ perceptions of their gains in the targeted-learning outcomes, and students respond to some questions by selecting a score on a Likert scale (e.g., 1 = no gains to 5 = strong gains) and to other questions by providing open-ended responses to prompts. Quantitative data from the SALG were exported into Microsoft Excel (Redmond, WA) and downloaded into JMP for statistical analysis, and results are graphically displayed using Delta Graph (Salt Lake City, UT; www.redrocksw.com).

SALG Results

Students’ Perceptions of Intellectual Behavior.

One set of SALG questions reflects Bloom's Taxonomy of Intellectual Behavior (Bloom et al., 1956) and is taken directly, word-for-word from the National Survey of Student Engagement (NSSE); this material is used with permission, because Wofford is a NSSE user. In Figure 1A, each Bloom's level has been abbreviated under each set of columns. For the full prompt for each question, please see question set #2 on the NSSE survey, which you may download at http://nsse.iub.edu/html/survey_instruments.cfm?survey_year=2012).

Although we see value in students developing their capacity to store and recall factual knowledge, the exams and assignments of our old courses rarely moved students’ thinking to higher levels on the Bloom's scale. Therefore, we are pleased to note (Figure 1A) that students in Biological Inquiry perceive that they use less memorization and recall and more application and judgment/evaluation than their predecessors in Zoology. Figure 1A also reflects that our ongoing efforts to improve the course each year are having a positive impact on student perceptions of their learning. At the end of the Fall 2011 semester, many of the instructors felt particularly good about how the course had gone, and we perceived that students had been operating at a higher intellectual level than in prior years. Therefore, it is encouraging to see student perceptions matching our own. Biological Inquiry is targeting a student-learning outcome for higher-order thinking that was identified through our departmental NSSE scores as an opportunity for improvement. Therefore, we now have both direct and indirect evidence that we are making inroads on improving this outcome through our work in Biological Inquiry.

An article by Crowe et al. (2008) has been helpful in guiding our work to raise students’ thinking to higher Bloom's levels. These authors (each from different campuses) make creative use of Bloom's Taxonomy in several undergraduate biology courses and provide excellent study suggestions for students to practice in each domain. We share and discuss these suggestions among the instructors and with our students, and we have learned that most students have never heard of Bloom's Taxonomy. Through these discussions, we increase the transparency and student understanding of course goals. We have found that being explicit with students about the purpose of our pedagogical strategies creates buy-in and helps them engage more constructively in the process.

Students’ Perceptions of Gains in Skills.

Perhaps the most dramatic results of Biological Inquiry are seen in the student perceptions of gains in skills. Figure 1B shows the students’ responses to the prompt: “As result of your work in this class, what gains did you make in the following skills (left to right on figure): Recognizing a sound argument and appropriate use of evidence (Use Evidence), Identifying patterns in data (ID Data Patterns), Critically reading and using information from research articles relevant to a particular problem (Read 1° Lit), Writing documents in discipline-appropriate style and format (Writing in Disc), and Developing a Logical Argument (Dev Logical Arg)?”

We are particularly pleased that students perceive gains in their scientific writing ability. Not only is improved writing a course goal, it is a college-wide goal and the target of our Quality Enhancement Plan for accreditation by the Southern Association of Colleges and Schools, Commission on Colleges (SACSCOC). Therefore, through Biological Inquiry, more than half of all incoming students are getting substantial practice and guidance in writing as it is done by scholars in biology.

We are also pleased that students perceive that they are making strong gains in their ability to use evidence, identify patterns in data, and develop a logical argument. These findings are consistent with the increased focus on these issues in Biological Inquiry as compared with the old course. Although we have not seen a significant increase in perceptions of stronger teamwork skills in the new course compared with the old course, we devoted similar effort to team-building in the old course, so it is not surprising that students’ perceptions of gains in this skill are quite high in both the old and new courses.

In each year of Biological Inquiry, students’ perceptions of learning gains have continued to climb in areas that we have targeted for further course revision, and this indicates that we are successfully “closing the loop” on assessment (Banta and Blaich, 2010). This is best seen in the students’ perceived gains in their ability to read and use research literature, an area that we devoted significant efforts to improve in 2011.

Because we did not include questions about understanding statistics until after we implemented Biological Inquiry (and therefore have no comparison for Zoology), these results are not shown in Figure 1B. However, we were pleased that students’ perceptions of gains in this area rose significantly in 2011 compared with prior cohorts in 2009 and 2010, especially because this was another area we had targeted for improvement in 2011.

Our direct-assessment measures, discussed previously, support student perceptions that Biological Inquiry improves reading, writing, and quantitative skills, as the research posters and exams also assess them. In sum, we are confident that the many students who take Biological Inquiry are bringing better quantitative and communication skills to their upper-level courses and beyond.

Students’ Perceptions of Gains in Content Knowledge.

Our SALG survey was also used to detect students’ perception of knowledge gains in particular content areas. Students were asked: “As a result of your work in this class, what gains did you make in your understanding of each of the following [left to right on Figure 1C]: How studying biology helps people address real world issues (Bio in Society), Agents of evolutionary change, e.g., mutation, natural selection (Evolution), The ecological relationships between individuals of the same or different species (Ecology), Diversity and commonalities among organisms (Org. Diversity), The similarities and differences between meiosis and mitosis [Meiosis/Mitosis], and The process of transcription (of DNA) and translation (into proteins) [T-script/T-slate].”

Students’ perceptions of their knowledge gains differed across the years, and their perceptions generally matched our expectations. For example, there was a decrease in students’ perceived knowledge gains of organism diversity, which had been the primary focus of Zoology, whereas perceived gains in understanding transcription and translation, which are emphasized in the new course, but were not covered in Zoology, jumped up dramatically in Biological Inquiry. Although we had hoped for even higher gains in the understanding of evolution in Biological Inquiry over Zoology, the phylogenetic approach used in Zoology appears to have been effective in elevating students’ perceptions of their understanding of the topic. Similarly, meiosis and mitosis were topics covered in Zoology and Biological Inquiry, and no differences in perceived learning gains were found between the courses.

We were disappointed to see a decrease during 2009 and 2010 in students’ perceptions of their understanding of how biology helps people address real-world issues. This was one of the areas on which we focused for improvement in 2011, which is the first year that we incorporated malaria as a recurring topic (as described previously), and it appears that our new efforts have improved students’ perceptions of biology's real-world applications. We also hope to see improvement in students’ perceived understanding of ecological relationships as a result of the changes we plan to implement in 2012.

Students’ Dispositions.

We have been most discouraged with the significant drop in student disposition scores in Biological Inquiry compared with Zoology. For example, perceived “gains for enthusiasm for biology” due to Biological Inquiry are lower (average 3.21 ± 0.09 SE) compared with Zoology (3.76 ± 0.12 SE). Some faculty members suggested that this may be due to biased, negative attitudes of BA students (who are no longer segregated from the BS students, as they were prior to 2009), but when we compared the scores for BS and BA students, this was not the case. It is worth noting here that this finding has helped to erase the negative stereotype that BA-track students “do not like” or “cannot do” science (there was also no difference in grade distributions among students who identified as BA vs. BS track), which had been used as part of the rationale for segregating them from science majors.

Further analysis of the SALG data on a professor-by-professor basis indicates that student “enthusiasm” scores did not decrease in the sections of Biological Inquiry taught by the two professors who had taught Zoology. Therefore, the drop in scores on this endpoint are due to lower scores in sections taught by the other instructors, and are not due to the inherent nature of the course itself.

We had expected that student attitude scores would rise across all instructors’ sections as we continued to improve the course each year, but these results have proven to be stubbornly stable. Therefore, we have turned this finding into a positive opportunity for further faculty development, particularly as it relates to learning skills useful for engaging traditional-age, incoming college students—skills that the two most “successful” professors (for this outcome) honed for more than a decade and that may be shared and adopted by their colleagues. All of the Biological Inquiry instructors have made it a regular practice to sit in on labs led by their colleagues, and we hope to continue to learn from this practice, especially with regard to this outcome.

Insights from Internal Evaluator's Assessment.

Dr. Dennis Wiseman (Professor of Foreign Languages and Director of Institutional Assessment) was our intramural evaluator. Dr. Wiseman met with biologists, witnessed workshops and summer planning sessions, and interviewed preceptors and faculty throughout the project. His insights were keen, and because he knows us all well, he was able to offer helpful suggestions in context. Dr. Wiseman also guided us in documenting our project's assessment so as to highlight our work within our next reaffirmation by SACSCOC.

Insights from External Evaluator's Assessment.

Dr. Michael Reder of Connecticut College (New London, CT) was our external evaluator for the NSF-funded project to create Biological Inquiry. His first campus visit occurred at the project outset, and we shared with him our early excitement and anxiety. His second visit was near the end of the course's first semester, and he conducted hour-long interviews and focus groups with biology professors, top college administrators, Biological Inquiry students, and preceptors and lab assistants. His report underscored our successes and uncovered the challenges that helped guide changes to the course for 2010. Returning the second year near the end of the Fall term, Dr. Reder again debriefed with the biology professors, interviewed the dean of the College, and held focus-group discussions with other STEM faculty members, as well as with seven classes of Biological Inquiry students (in the absence of the professor).

One of Dr. Reder's focus-group questions was, “What assignment was the most challenging and why?” This was similar to the open-ended SALG question “How did the assignments in this course challenge you in ways that were different from high school?” And in both assessments, the vast majority of students chose the team-based research posters as the most unique and challenging assignments. One student's response does a particularly good job of summarizing the majority of students’ sentiments:

Although we hope to improve students’ attitudes toward the poster assignments, we believe that the antipathy reflected here indicates that we are moving students from being passive recipients of knowledge, the comfort zone of most young learners (Perry 1970), to higher levels of intellectual development.

In summary, we are confident that indirect assessment done well (i.e., well-designed surveys with high response rates, student self-reflection, unbiased evaluators, etc.) is of value equal to similarly well-designed, direct-assessment measures in assessing the outcomes of a course (or any project). It is the integration of the different types of evidence that provides nuanced insight about the successes of a project, as well as areas to target for improvement.

BROADER IMPACT AND KEY LESSONS

Fairweather (2008) argues that if more faculty members could be encouraged to shift their pedagogical practices even slightly in favor of inquiry-based techniques, it would have more impact than further developing the skills of those already fully engaged in reformed practice, and we concur. However, it is important to be attuned to the very human vulnerabilities inherent in such work. Following a particularly intense week of discussion of “best pedagogical practices” prior to launching Biological Inquiry, one longtime colleague admitted feeling that his life's work as a dedicated teacher was being dismissed as substandard. Therefore, it is essential to acknowledge prior excellence and to respect that such a major change instills frustration, anxiety, and fear of failure, perhaps most especially in senior colleagues who are being asked to change longtime practices.

Because we worked as a dynamic team and respected and listened to one another's ideas, our disagreements were part of the creative enterprise, and no one person's ideas dominated the final product. Whereas newer colleagues brought fresh ideas from their doctoral training, seasoned professors provided keen insights based on years of close work with students. We are a closer group of colleagues for engaging in this process, and we are practicing the teamwork that we seek to develop in our students.

We have had numerous opportunities to share our work with Wofford's entire faculty, and two departments (Psychology and Environmental Studies) have followed Biology's lead by reforming their introductory science courses. Anecdotal evidence also suggests that the new course is having a broad impact; as one science chair noted to Dr. Reder, “students who have taken Biological Inquiry are more comfortable than other students, even upperclassmen, when interpreting ambiguous experimental results—they are better able to handle this situation and are less frustrated.” Another noted that the “new bio students seem to better appreciate the importance of the laboratory experience rather than simply looking for the answer.” The establishment of Wofford's new Center for Innovation and Learning, directed by Dr. Wiseman, was also stimulated, in part, by calls from the faculty to sustain the workshops we had initiated, and the center will provide ongoing development opportunities for the entire faculty.

The development and implementation of Biological Inquiry is our response to the national call to adopt best pedagogical practices and build the science competencies of our undergraduates. A few key strategies of our work are worth recapping: First, we developed the capacities of our faculty team by increasing their awareness of the national call for reform, expanding their pedagogical repertoire, respecting their prior contributions, and acknowledging the anxiety that surrounds such momentous change. Second, we communicated regularly with top administrators and trustees, educating them on the purpose and goals of our work (e.g., our dean was twice a member of Wofford's team at the SENCER Summer Institute), and they supported us with additional personnel and ongoing encouragement. Third, we partnered in this work with students whose creative energy and insights motivated us to do our best work. Fourth, we used existing assessment evidence to help motivate the reform effort, and we implemented a robust assessment plan of our new program, the evidence from which showcases our successes and guides continuous improvement. We hope that others will be encouraged by our story to undertake similar reform as we share in the nation's collective effort to ensure the best education for all students.

FOOTNOTES

REFERENCES