Improving Exam Performance in Introductory Biology through the Use of Preclass Reading Guides
Abstract
High-structure courses or flipped courses require students to obtain course content before class so that class time can be used for active-learning exercises. While textbooks are used ubiquitously in college biology courses for content dissemination, studies have shown that students frequently do not read their textbooks. To address this issue, we created preclass reading guides that provided students with a way to actively engage with the required reading for each day of class. To determine whether reading guide completion before class is associated with increased performance, we surveyed students about their use of reading guides in two sections of a large-enrollment (400+ students) introductory biology course and used multiple linear regression models to identify significant correlations. The results indicated that greater than 80% of students completed the reading guides before class and that full completion of the reading guides before class was significantly positively correlated with exam performance. Reading guides in most cases were used similarly between different student groups (based on gender, ethnicity, and aptitude). These results suggest that optional preclass reading guides may help students stay on track to acquire course content in introductory biology and thus result in improved exam performance.
INTRODUCTION
Recent calls for reforming undergraduate biology education have recommended increasing student engagement and the teaching of scientific literacy skills (American Association for the Advancement of Science, 2011; President’s Council of Advisors on Science and Technology, 2012). For class time to be devoted to active learning or delving deeper into the scientific literature, course content needs to be delivered other than during the class meeting time. One way to do this is to hold students responsible for learning the course content before class—this is the basis of “flipped” courses (in which students acquire content before class and then perform active-learning exercises in class) or high-structure courses (in which students acquire content before class and then complete graded preclass assignments, participate in active learning in class, and complete graded weekly review assignments) (Freeman et al., 2011; Eddy and Hogan, 2014; O’Flaherty and Phillips, 2015; Shaffer, 2016). By having students acquire content before class, the course instructor can “hit the ground running” and use active learning to formatively test students on what they have learned, address misconceptions, investigate scientific and ethical issues, and practice scientific literacy skills (Bonwell and Eison, 1991; Allen and Tanner, 2005).
Students can acquire content before class in several ways, although two may predominate in many college courses: watching videos or reading the textbook. While watching videos may be a useful way for students to acquire content, instructors may not have the time to create them nor may they be able to find suitable ones for students to watch (Milman, 2012; Jensen et al., 2015; O’Flaherty and Phillips, 2015). However, it is very common for college biology courses to require students to purchase a textbook for class. Ranging in length, topics, the amount of detail, and the level (majors vs. nonmajors), textbooks can be a useful way for students to learn course content before coming to class. However, many studies have shown that students across a wide variety of college courses do not regularly read their textbooks. Sikorski et al. (2002) surveyed psychology students at two universities and found that, of students who purchased the course textbook, 80% read less than 3 hours per week and 95% valued attending lecture and studying course notes more than textbook reading. Clump et al. (2004) found that only 27% of psychology students regularly read the assigned reading before each day of class, but that number increased to 70% before an exam. Podolefsky and Finkelstein (2006) found that, while 97% of physics students purchased the course textbook, only 41% of students regularly read their textbooks before class, 60% read after class instead of before, and there was little correlation between reading habits and course performance. Berry et al. (2011) found that, while students in finance courses report that they know it is important to read their textbooks and that it will likely impact their grade, only 8% of students spent >3 hours a week reading their textbooks. These studies and others paint a somewhat grim picture that college students may not be preparing for class by reading their textbooks, even when readings are explicitly assigned by instructors.
Why is it that students do not read their textbooks? With regard to biology and science textbooks, many possible reasons exist. First, students may not be familiar with the style of writing that is found in science textbooks (Hodges, 2015). Science writing is denser than narrative writing and other styles; often assumes the reader has specific prior knowledge on which to scaffold new concepts; and uses figures, graphs, and equations that may prove to challenging for students (Snow, 2010; van den Broek, 2010). Additionally, science textbooks introduce students to countless new vocabulary words that are often unusual and linked to languages (e.g., Latin and Greek) that students are not familiar with (Groves, 1995; Fang, 2006; Shanahan, 2012). Because of the nature of science writing, students may feel overwhelmed and develop a seemingly insurmountable cognitive load that they cannot overcome, which then results in them failing to read (van den Broek, 2010; Hodges, 2015). Second, students may not understand the purpose of reading their textbooks. While students may think that the goals of textbook reading are to immediately prepare for exams or for long-term accumulation of knowledge to prepare for careers (Elshout-Mohr and van Daalen-Kapteijns, 2002), instructors may have other goals that students do not realize. Instructors may wish for students to critically evaluate the text, analyze data from figures, or make connections to topics covered in past courses. Additionally, instructors may simply want students to read before class in order to be better prepared for in-class lessons (Hodges, 2015). This potential disconnect in purpose may lead to students not seeing the value in reading, and thus they simply will not read. Finally, students may not be motivated to read in the first place. Students may not be intrinsically motivated to read because they lack interest in the subject matter or they do not experience personal satisfaction when reading. Extrinsic motivations to read (course points, reading quizzes, etc.) may also not exist in a given course (Hodges, 2015), and thus students fail to see the immediate rewards for reading. No matter the reason why students do not read, in the case of a flipped or high-structure course, wherein preclass preparation is critical for success, not reading before class may be a recipe for disaster.
To address these issues, several attempts have been made to design curricular additions to encourage students to read textbooks. Solomon (1979) developed and used the “two-point system” in a psychology course to reward students who read the assigned reading while not penalizing those who did not. To receive two extra points on an exam, students had to submit a written assignment that related topics from the assigned preclass textbook reading to what they were going to cover in class that day. Using this method, 85% of students read the assigned readings before class, but there was no effect on final exam scores. In an introductory biology course, Armbruster and colleagues (2009) developed the “daily dozen,” a set of questions associated with each lecture that was intended to help students read their textbooks before class. In two social work courses, students reported favorable outcomes from using preclass study guides that were designed to encourage students to read their textbooks before class (Vandsburger and Duncan-Daston, 2011). Greater than 95% of computer science students reported that optional preclass “exploratory homeworks,” which were developed to guide students through the preclass reading and coding assignments, were either sometimes or always helpful toward understanding lecture and lab materials (Esper et al., 2012). By assigning preclass reading quizzes in an introductory physics courses and an introductory physiology course that directed students to specific textbook passages and figures, Heiner et al. (2014) found that 80% of students regularly read the assigned reading before class and that their preclass quiz scores were positively correlated with exam performance. Common themes from many of these studies include that students are given some sort of guidance on what to read before class, and that there was some reward for completing the assigned readings.
In this study, we developed and assessed custom reading guides for a large-enrollment introductory biology course. The reading guides required students to actively engage with their textbooks while reading by defining terms, explaining concepts, making tables, drawing figures, and answering questions. We hypothesized that full completion of reading guides before each class would help students prepare on a daily basis and would thus be positively correlated with increased examination performance. We found that there was a strong significant positive correlation between reading guide completion and exam performance. We also report on how students used the reading guides and offer recommendations for implementing them in other college science courses.
METHODS
Course and Student Description
The course that was studied was a large-enrollment (400+ students) introductory biology course taught on the quarter system (10 weeks) at a large doctoral-granting (R1) institution in the western United States. The course is considered to be the first half in a typical two-course introductory biology sequence and covers foundational concepts in cell biology, molecular biology, and genetics. The entire course enrollment met for three 50-minute lessons per week in a large, fixed-seating lecture hall. Students also attended one 50-minute discussion section per week taught by trained graduate assistants. The course was taught with high structure (Freeman et al., 2011; Eddy and Hogan, 2014; Shaffer, 2016), as students were required to read the course textbook and complete daily online assignments via Mastering Biology (Pearson, New York, NY) before class; participate in active-learning exercises and activities, including clicker questions (iClicker software, version 6.3, iClicker, Indianapolis, IN), group work, and discussions in class; and complete weekly review quizzes (Mastering Biology) at the end of each week of class. There were four multiple-choice exams totaling 80% of the course grade: three midterm exams (15% each of the course grade) and a cumulative final exam (35% of the course grade; approximately half of the final exam covered new material and half was purely cumulative). Greater than 60% of each exam was composed of Bloom’s (Bloom et al., 1956) level-three (application) or higher questions (based on Bloom’s levels assigned by the course instructor [J.F.S.]), and no exam questions were identical to questions found in the reading guides. All exams and reading guides are provided in the Supplemental Material. A custom edition of Campbell Biology, 10th ed. (Reece et al., 2014), was used for the course. A single instructor (J.F.S.) taught the two sections (A and B) of the course in Fall 2015 that are included in this analysis.
The course is taught in the Fall quarter, and the students who enroll in this course are primarily freshman biological sciences majors in the first quarter of their college careers. There were 877 students enrolled in the two sections in Fall 2015, and 790 (90%) provided gender, ethnicity, and Scholastic Aptitude Test (SAT) score data and thus were included in this study (Table 1). Approximately 70% of the students were female, 42% of the students were underrepresented minorities (African- American, Hispanic/Latin@, or Native American), and 58% of the students were biological sciences majors.
| Number of students (%) | |
|---|---|
| Total | 790 (100) |
| Gender | |
| Male | 236 (29.9) |
| Female | 554 (70.1) |
| Ethnicity | |
| White | 84 (10.6) |
| Asian | 372 (47.1) |
| Hispanic/Latin@ | 300 (38.0) |
| African American | 34 (4.3) |
| Major | |
| Biological sciences | 458 (58.0) |
| Other STEM (science, technology, engineering, and mathematics) | 173 (21.9) |
| Non-STEM | 30 (3.8) |
| Undecided/undeclared | 129 (16.3) |
| Average SAT (±SD) | 1648 ± 198 |
Reading Guide Structure
Custom reading guides were created in Microsoft Word 2011 (Microsoft, Redmond, WA) for each lesson (one per day of class) by the course instructor (J.F.S.) and were posted on the course website at least 1 week before each day of class. Students were not required to complete the reading guides, which were not collected or rewarded points for completion; however, students were urged to complete them via an explanation in the syllabus and recommendations in lecture by the instructor (J.F.S.) and in discussion sections by the graduate teaching assistants. Sample reading guide questions are shown in Figure 1, and all reading guides are located in the Supplemental Material.
FIGURE 1. Examples of reading guide questions from throughout the course. The type of question is shown along with an example for each.
The reading guides ranged from two to five pages long (average ± SD = 3.5 ± 1.0, n = 24), with the most common length being three pages. The foundation of each reading guide involved asking students to answer questions or complete tasks in order to encourage them to actively read the textbook (Campbell Biology, 10th ed. [Reece et al., 2014]). Tasks included defining terms, explaining concepts, making tables and drawings, or answering in-chapter textbook questions (Figure 1). Students were directed which pages to read and what sections (if any) to skip. They were also directed to investigate specific figures or tables as deemed necessary. Completed reading guides were never posted to the course website or given to the students. However, students could ask questions about the reading guides on the course discussion board (Piazza, Palo Alto, CA) that were readily answered by the course instructor or teaching assistants.
Data Collection
Data were collected from students enrolled in two sections of the course taught during the Fall 2015 quarter. Section A (441 students enrolled) was taught from 12:00 pm to 12:50 pm, and section B (436 students enrolled) was taught from 1:00 pm to 1:50 pm, with each class being taught on Monday, Wednesday, and Friday. To be eligible for this study and included in the analyses, a student had to consent to being in the study, had to be 18 years old at the beginning of the course, and had to have demographic information available (gender, ethnicity, and SAT scores). A total of 396 students (89.8% of enrollment) from section A and 394 students (90.4% of enrollment) from section B met these conditions and were included in the analysis in this study (790 students total [90%]). Demographic information for these students is provided in Table 1. This study was approved by the University of California, Irvine Institutional Review Board (HS# 2013-9833). Data were analyzed using Microsoft Excel 2011 (Microsoft, Redmond, WA) and the statistical program R, version 3.1.2 (R Foundation for Statistical Computing, Vienna, Austria).
Reading Guide Use
Two separate methods were used to determine how students used the reading guides in this course. First, students completed an online end of course survey via the course learning management system in which they answered questions relating to when they completed the reading guides (before class, after class, or not at all), how often they completed the reading guides before class (on a five-point scale from never to always), and how long they spent completing the reading guides on average (from 0 to 180+ minutes). Second, students were asked to answer a clicker question (iClicker) at the start of each day of class regarding their completion of the reading guide associated with that day of class. Approximately 3 minutes after the start of each day of class, the following clicker question was posed: “How did you prepare for class today?” Students could answer “A—I completed the entire reading guide,” “B—I completed more than half of the reading guide, but not all of it,” “C—I completed less than half of the reading guide,” “D—I did not complete the reading guide at all, but I still read the assigned textbook pages,” or “E—I did not complete the reading guide at all and I did not read the assigned textbook pages.” Student responses to these questions required them to be present in class and have their iClicker device with them. The percent of responses ranged from a maximum of 98.4% of enrollment to a minimum of 75.1% of enrollment. The overall average percentage of responses on a given day of class to this question was 90.7 ± 4.9% of enrollment.
To determine the accuracy of students’ self-reported use of the reading guides via daily clicker questions, graduate student teaching assistants randomly examined students’ reading guides in discussion sections. Without notifying the students ahead of time, two graduate teaching assistants examined students’ reading guides in their discussion sections, which were held 2 to 4 hours before lecture started. The teaching assistants noted the extent to which the students had completed their reading guides, and this was compared with the degree of completion that the students input during the in-class clicker question to determine whether students reported the same level of completion, underreported how much they had completed, or overreported how much they had completed. Data were collected randomly from 62 students throughout the course in this manner.
Reading Guide Completion and Examination Performance
Multiple linear regression models were used to determine the impact of reading guides on student performance, because these models control for extraneous factors that may affect interpretation of the results (Theobald and Freeman, 2014). Models were developed with the response variables being student performance on each of the four exams as a percentage out of 100. The data for both sections were combined, and thus four models were developed in this analysis (one for each exam). The total exam scores (as a percentage out of 100) on midterm exams 1, 2, and 3 were used as response variables in their respective models. The final exam was split into two sections: half on the new material and half on the past material covered in the three midterms. The score on the “new” portion of the final exam (as a percentage out of 100) was used as the response variable for the final exam model. Descriptive statistics for student performance on each exam (as a percentage out of 100) are shown in Table 2.
| Exam | Average exam score (± SD)a | Average number of reading guides completed fully before class (± SD)b | n (percent of study sample) |
|---|---|---|---|
| 1 | 70.83 ± 18.02 | 3.61 ± 2.04 | 602 (76) |
| 2 | 63.23 ± 16.65 | 2.56 ± 2.28 | 570 (72) |
| 3 | 71.42 ± 15.55 | 2.97 ± 2.22 | 523 (66) |
| Final | 65.82 ± 16.15 | 3.07 ± 2.19 | 426 (53) |
The impact that reading guide completion may have on student performance was determined in the following manner. Students’ answers to the daily clicker question about reading guide completion were examined for how many times over the course of the class that a student responded “A—I completed the entire reading guide” and were summed. Each exam covered six new lessons; therefore, the maximum number of times a student could respond “A” was six (for the six lessons of the class), and the minimum was zero. This explanatory variable was therefore treated as a continuous variable with a value of 0–6 in the models. The average numbers of reading guides that students completed fully before class for each exam are shown in Table 2. To be included in these analyses, students had to attend class and respond to the daily clicker question all 6 days of class covered on each exam. This requirement limited our data set, as students had to routinely attend class and answer the clicker question each day. The final numbers for students included in each model are presented in Table 2. The total number of students in each model ranged from 602 (exam 1; 76% of the study sample) to 426 (final exam; 53% of the study sample). The number of students included in the final exam model was low, because this set of six lessons included a lesson the day before Thanksgiving, and many students did not attend (attendance on that day was only 75%, the lowest of the course). To summarize, we are focusing in these analyses on the extent that full completion of reading guides before class contributes to exam performance. We did not assess the impacts of partial reading guide completion on performance in this study.
In addition to the number of reading guides, several other explanatory variables were included in this analysis to attempt to control for confounding factors. Students’ average preclass assignment scores that corresponded for a particular exam were included to control for any effect that completion of the preclass assignments may have on examination performance. SAT math and reading scores were included to control for student aptitude (SAT writing scores were not included, as they were not found to significantly contribute to the models). Student demographic factors, including gender and ethnicity (treated as a binary variable as URM [African-American, Latin@, or Native American] or majority [Asian or white]), were also included to account for any demographic influence. Because the data from both sections of the course were combined, a binary variable for course section was also included. The final model was 
Reading Guide Completion among Student Groups
Multiple linear regression models were also used to determine whether different student groups (based on gender, ethnicity, or SAT scores) completed the reading guides before class to different extents. Models were created using the number of reading guides completed fully before class as the response variable and student gender, ethnicity, SAT math score, SAT reading score, and course section as the explanatory variables. Student usage was analyzed for each individual exam (six reading guides per exam). The final model was 
Only students who had complete data for all of the above-mentioned variables were included in each of the four models. Estimates for each variable in the model were reported as mean ± SEM and were considered statistically significant at p < 0.05.
RESULTS
Reading Guide Use
Student use of the reading guides was measured through an end of course survey. A total of 677 students completed this survey. First, students were asked whether they completed the reading guides before class, after class, or not at all. From this survey, ∼86% of students reported that they completed the reading guides at least sometimes before class. Approximately 9% of students reported that they completed the reading guides after class, while ∼5% reported that they did not complete them at all. Next, students were asked to report on a five-point scale from “never” to “always” to determine how often they completed the reading guides before class (again, this was the way we recommended reading guides be used). As shown in Figure 2A, > 86% of students reported completing the reading guides at least sometimes before class. Finally, we sought to determine how much time students spent completing the reading guides before class. As shown in Figure 2B, students reported a wide range of times, with the most common time being 60–90 minutes per reading guide.
FIGURE 2. Summary of student use of the reading guides from end of course survey data. Students reported how often they completed reading guides before class (A) and the average time they spent completing the reading guides (B).
Students’ daily clicker responses were analyzed to examine student reading guide use at a more granular level. Figure 3 shows the average percent responses to each of the five clicker question options over the entire course. On average, >75% of students reported completing at least some of the reading guide for a given day of class, with nearly 50% reporting that they completed the entire reading guide before class each day. The daily clicker responses were also analyzed to determine the average number of reading guides the students completed fully before class (i.e., “A—I completed the entire reading guide”) for different time periods in the course. As shown in Table 2, the average number of reading guides that students completed fully before class for each exam period (maximum of six reading guides per exam) varied across the duration of the course.
FIGURE 3. Summary of daily clicker response data to assess what proportion of the reading guides students completed before class. Data are presented as mean ± SD for the entire course.
To determine whether different student groups (based on gender, ethnicity, and SAT math and reading scores) fully completed reading guides similarly before class, we developed multiple linear regression models using the number of reading guides completed fully before class (i.e., the number of times that a student responded “A—I completed the entire reading guide”) as the response variable and student gender, ethnicity, SAT math scores, SAT reading scores, and course section as the explanatory variables. Table 3 presents these models for each individual exam. Student demographics significantly contributed to reading guide completion before class in few cases and in an inconsistent manner.
| Exam 1 | Exam 2 | Exam 3 | Final Exam | |||||
|---|---|---|---|---|---|---|---|---|
| (n = 602; 76%) | (n = 570; 72%) | (n = 523; 66%) | (n = 416; 53%) | |||||
| Regression coefficient | Estimate ± SEM | p value | Estimate ± SEM | p value | Estimate ± SEM | p value | Estimate ± SEM | p value |
| Model intercept | 2.32 ± 0.72 | 0.001 | 1.71 ± 0.86 | 0.05 | 1.38 ± 0.85 | 0.10 | 2.86 ± 0.97 | 0.003 |
| Gender (male) | −0.62 ± 0.19 | 0.001 | 0.02 ± 0.22 | 0.91 | −0.24 ± 0.22 | 0.27 | 0.16 ± 0.25 | 0.51 |
| Ethnicity (majority) | −0.09 ± 0.19 | 0.62 | 0.50 ± 0.22 | 0.02 | 0.18 ± 0.22 | 0.41 | 0.30 ± 0.24 | 0.22 |
| SAT math | 0.002 ± 0.001 | 0.11 | 0.0005 ± 0.002 | 0.73 | 0.004 ± 0.002 | 0.01 | 0.002 ± 0.002 | 0.20 |
| SAT reading | 0.0006 ± 0.001 | 0.66 | 0.0008 ± 0.001 | 0.57 | −0.001 ± 0.001 | 0.48 | −0.002 ± 0.002 | 0.21 |
| Section (B) | −0.007 ± 0.17 | 0.97 | −0.43 ± 0.19 | 0.02 | −0.43 ± 0.19 | 0.03 | −0.43 ± 0.21 | 0.04 |
| Adjusted R2 | 0.013 | 0.018 | 0.026 | 0.018 | ||||
Impacts of Reading Guides on Exam Performance
To determine the impact that completing reading guides fully before class (i.e., the number of times that a student responded “A—I completed the entire reading guide”) had on exam performance (as a percentage out of 100), multiple linear regression models were constructed controlling for potential confounding factors (gender, ethnicity, SAT math score, SAT reading score, preclass assignment scores, and course section). Table 4 displays the models for each exam in the course. To be included in these models, a student had to attend all lessons covered on each exam (six each) and had to answer the clicker question in each day of class. The model estimates for the impact of completing a single reading guide before class on exam score varied from a low of 1.35 ± 0.26 (exam 3) to a high of 1.71 ± 0.26 (exam 2). The estimates for completing reading guides before class were statistically significant (p < 0.05) for each of the four exams when controlling for the potentially confounding factors listed earlier (the other factors that significantly contributed to each model are shown in Table 4). SAT math and reading scores contributed significantly to nearly every model, and gender and ethnicity contributed in a few cases.
| Exam 1 | Exam 2 | Exam 3 | Final Exam | |||||
|---|---|---|---|---|---|---|---|---|
| (n = 602; 76%) | (n = 570; 72%) | (n = 523; 66%) | (n = 426; 54%) | |||||
| Regression coefficient | Estimate ± SEM | p value | Estimate ± SEM | p value | Estimate ± SEM | p value | Estimate ± SEM | p value |
| Model intercept | −39.20 ± 9.56 | 4.69e-05 | −22.87 ± 7.09 | 0.001 | −14.79 ± 6.98 | 0.04 | −16.54 ± 9.52 | 0.08 |
| Total number of reading guides | 1.59 ± 0.30 | 1.29e-07 | 1.71 ± 0.26 | 7.59e-11 | 1.35 ± 0.26 | 3.32e-07 | 1.54 ± 0.30 | 3.44e-07 |
| Preclass assignment score | 0.28 ± 0.09 | 0.002 | 0.17 ± 0.06 | 0.003 | 0.18 ± 0.06 | 0.002 | 0.24 ± 0.09 | 0.006 |
| Gender (male) | 1.82 ± 1.35 | 0.18 | 4.25 ± 1.34 | 0.002 | 1.85 ± 1.28 | 0.15 | 4.06 ± 1.50 | 0.007 |
| Ethnicity (majority) | 3.25 ± 1.34 | 0.015 | 1.29 ± 1.35 | 0.34 | 0.88 ± 1.28 | 0.50 | 1.47 ± 1.48 | 0.32 |
| SAT math | 0.08 ± 0.01 | <2e-16 | 0.06 ± 0.01 | 9.14e-11 | 0.08 ± 0.01 | <2e-16 | 0.06 ± 0.01 | 4.81e-09 |
| SAT reading | 0.05 ± 0.01 | 2.16e-08 | 0.05 ± 0.01 | 5.88e-09 | 0.03 ± 0.01 | 0.001 | 0.04 ± 0.01 | 4.58e-05 |
| Section (B) | 0.36 ± 1.18 | 0.76 | 1.96 ± 1.15 | 0.09 | 4.82 ± 1.13 | 2.51e-05 | −4.59 ± 1.30 | 4.64e-04 |
| Adjusted R2 | 0.380 | 0.344 | 0.358 | 0.345 | ||||
Based on these results, completing reading guides fully before class contributed significantly to exam performance. Taking the most conservative estimate (the model output from exam 3), completing a single reading guide before class yielded a gain of 1.35 ± 0.26 percentage points (out of 100) on the exam. Therefore, if a student completed all six reading guides, the model predicts that his or her exam score would be ∼6.5–9.6 points (out of 100) higher than if the student completed zero reading guides (controlling for other demographic factors). This corresponds to a half–letter grade to full–letter grade improvement on the exam.
Comparison of Students Included in the Models
For students to be included in the multiple linear regression models, they had to attend all six lessons associated with a given exam and respond to the clicker question each day. Because of this requirement, many students were left out of each model. As shown in Tables 3 and 4, the number of students included in each model ranged from 76 to 53% of the study sample. Because of this loss of the study sample, there is potential for a bias in the analysis and results in this study toward those students who met the conditions. To determine whether the demographics of students included in each model were significantly different from those not included in each model, chi-square tests were performed between the two groups on the basis of gender and ethnicity for each of the four exams. In addition, paired t tests were performed on the average total SAT scores between the two groups for each exam. There were no significant differences between gender, ethnicity, or SAT distributions between students in the model versus those not in the model for any exam (Supplemental Tables S1–S3).
Validation of Reading Guide Self-Report Data
The reading guide usage data obtained in this study hinge on the accuracy of the student self-reports from the daily clicker questions. To validate these data, graduate teaching assistants randomly reviewed students’ reading guides in discussion sections before class and indicated the extent to which the students’ completed the reading guides. Over the duration of the class, 62 students’ reading guides were examined and were compared with their self-reported reading guide completion results from the lecture portion of the course. Analysis of these data showed that 69.4% of the teaching assistant observations matched the students’ self-reports, 14.5% of the students overreported their completion of the reading guides, and 16.1% underreported their completion of the reading guides.
DISCUSSION
This study assessed the impact of optional, preclass textbook reading guides on exam performance in two sections of a large-enrollment (400+) undergraduate introductory biology course. Our results showed that on average ∼80% of students completed at least some of the reading guides before class (with ∼50% completing the entire reading guide each time [i.e., for each day of class]). When controlling for potentially confounding factors, full completion of reading guides before class was significantly positively correlated with increased exam scores for all exams in the course. In addition, the reading guides seemed to be used equivalently by different student groups (based on gender, ethnicity, and SAT scores), suggesting that the use of reading guides did not exclude any students from potential benefits. These results are significant because they suggest that a relatively simple, low-stakes intervention such as preclass reading guides can be implemented in a large-enrollment biology course to encourage students to read their textbooks and help students stay on track on a daily basis, resulting in a significant positive impact on student learning.
While these results demonstrate that optional preclass reading guides are significantly positively correlated with increased exam scores, they do not provide evidence regarding the mechanism of why reading guides may result in improved performance. However, two hypotheses seem likely. It could be that the reading guides are helping students read and understand the textbook. Completing a reading guide requires students to take an active approach to reading the textbook through pausing to answer questions in their own words, to examine figures, to complete tables, and so on. Indeed, several studies have highlighted the importance of textbook reading strategies that can result in increased reading comprehension (Simpson and Nist, 1990; Digisi and Willett, 1995; Best et al., 2005; Ozuru et al., 2009). Alternatively, it could be that the reading guides served as a way to help students stay on track on a daily basis and thus helped students manage their studying throughout the course. By reading the textbook before class and answering questions in the reading guide, students are exhibiting retrieval practice behaviors, which have been shown to be positively related to learning and performance (Roediger and Karpicke, 2006; Butler and Roediger, 2007; Karpicke and Blunt, 2011; Pennebaker et al., 2013). One possibility that is not likely, however, is that students improved on exams because they were answering verbatim or very similar questions from the reading guides on the exams themselves. While the exams in the courses in this study contained greater than 60% Bloom’s level-three (application) and higher questions, the reading guides contained mostly Bloom’s level-one (knowledge) and level-two (comprehension) questions. Moreover, no exam questions were identical to any reading guide questions (all exams and reading guides are provided in the Supplemental Material). Therefore, it is possible that the solid foundation in lower-level Bloom’s skills provided by the reading guides enabled students to develop their higher-level Bloom’s skills in class and thus apply them to exams. Future studies are warranted to investigate the mechanism of how reading guides may lead to improved learning and success.
The results from this study are based on a single reading guide structure that requires students to define terms, explain concepts, complete tables, make drawings, and answer in-chapter questions. However, the reading guide structure could be modified to achieve even larger gains in future courses by including other activities, such as concept mapping, drawing, and question authoring that have been shown to support student learning (Novak, 1990; Van Meter, 2001; Van Meter et al., 2006; Hay et al., 2008; Bottomley and Denny, 2011; McQueen et al., 2014). Additionally, the reading guides could be modified such that students complete them in groups online (via Google Docs or similar products), which may yield beneficial results (Blau and Caspi, 2009; Chu and Kennedy, 2011). Finally, the implementation of the reading guides could be modified so that they are required and worth course points. This change in grading policy will likely increase the completion rate, but with large courses such as the one in this study (400+ students in each section), collection and grading of the reading guides may be problematic. However, the potential benefits may outweigh the costs in this scenario. It is therefore worthwhile to explore alternative reading guide structures and implementations to determine what structure provides the best learning opportunity for students.
Implementing Reading Guides in Other Courses
While the results of this study are promising, they were found by examining students from a single large R1 university in the western United States. Moreover, the course was taught primarily for majors, with a specific textbook (Campbell Biology, 10th ed. [Reece et al., 2014]), and in the 10-week quarter system. Therefore, replication of these results at other types of institutions with other students and courses is critical. To this end, we wish to offer some advice for instructors who may wish to adopt reading guides for use in their courses. First, a significant time commitment is required to develop them, no matter what textbook is used. The correct page numbers, section titles, and figures must be cited, and specific questions that go along with the textbook need to be written. This could also lead to problems when a new edition of the textbook is published, as the reading guides may need to be updated to reflect changes in the textbook. However, by going through the act of developing reading guides, we found that we were much more in tune with the content of the textbook and were very clear on what we wanted students to read in advance of each class session, which helped align learning objectives between preclass assignments and in-class activities (Biggs, 2003; Handelsman et al., 2007). Additionally, students must have the time to be able to complete the reading guides. We found ∼1/3 of our students spent 60–90 minutes completing a single reading guide, with ∼70% spending between 30 and 120 minutes per reading guide (Figure 2B). If students already have significant demands on their time from course work or other commitments, expecting students to complete reading guides may be too much to demand. Second, once the reading guides are developed, what happens in each class session may need to change so as to hold students accountable for completing the reading before class. Instead of spending time defining terms, explaining basic concepts, or going through textbook examples verbatim, an instructor may wish to first ask the class questions about the definitions or basic concepts to determine whether students did indeed acquire the necessary content from the reading. In this way, significant portions of content delivery can be removed from class, thus freeing up time to challenge students to apply what they have read through the use of problem-based learning, case studies, or primary literature data analysis (Allen and Tanner, 2005; Allen et al., 2011; Herreid, 2007; Hoskins et al., 2011; Round and Campbell, 2013). Third, we advise that the completed reading guides not be posted for students to download, as we fear that students may choose to simply study the completed document without actually reading the textbook. However, students may be encouraged to post questions regarding the reading guides to a course discussion board or to collaborate with classmates. Finally, we recommend incorporating the reading guides into in-class activities. For example, when discussing the stages of cellular respiration in class, we ask students to take out their reading guides and compare their work with each other. This achieves two goals: first, it helps students specifically review content that is required for in-class activities, and second, it demonstrates to students that the reading guides are important and should be completed before class as they will help the students succeed in class.
Limitations
A confounding factor of this study is that we did not directly measure and control for student motivation. It is possible that the students who completed the reading guides before class and arrived to class on time were more motivated and would have performed well on exams no matter what instructional tools were available to them. Thus, the daily clicker question asking about reading guide completion could have been capturing motivation. As we know that motivation is strongly associated with learning (Preckel et al., 2006; Schunk and Zimmerman, 2008; Bryan et al., 2011), this possibility cannot be ruled out. However, we attempted to control for motivation by only including students who attended all 6 days of class associated with each exam. In this case, students had to be at least equally motivated to attend each day of class (and thus they also received identical instruction). We also included preclass assignment scores in each model in an attempt to control for student motivation to complete these assignments.
Another limitation is that our conclusions are based on the analysis of student self-reported data (the daily clicker questions that assessed reading guide completion). We attempted to determine how accurate students’ clicker responses were through visual inspection of students’ reading guides in discussion sections held 2 to 4 hours before a given lecture period. Through this method, we found that ∼70% of students accurately reported their degree of reading guide completion, and ∼15% each either overreported or underreported how much of the reading guide they completed. Accuracy of self-reported data is a concern in many fields, including education research, medicine, health services, and other social sciences, as conclusions critically depend on the quality of the data (Koziol and Burns, 1986; Zapka et al., 1996; Del Boca and Noll, 2000; Okura et al., 2004; Connor Gorber et al., 2007; Boufous et al., 2010). One such example about the concerns of the accuracy of self-report data is available from educational research literature: a meta-analysis that showed that the accuracy rate of students self-reporting their high school or college grade point average varied from a low of 54.3% to a high of 82.4% (Kuncel et al., 2005).
Students may choose to self-report inaccurate information for a variety of reasons, but the concept of socially desirable response theory may apply in our situation. Socially desirable response theory states that people tend to tailor their responses to surveys in order to be viewed favorably by others (Edwards, 1957; Crowne and Marlowe, 1960). In this case, students may have chosen to overreport the amount of the reading guide that they completed because that would be viewed more favorably by their peers and their instructors. This explanation would account for the ∼15% who indeed overreported their reading guide completion. Alternatively, the students who overreported their reading guide completion may have actually completed the reading guide in between the time that their reading guide was checked and the time that class started (a 1- to 3-hour window). On the other hand, students who underreported their reading guide completion may have underestimated how much they actually completed (perhaps by not being sure where the halfway point of the reading guide was). Finally, students may simply have forgotten or accidentally misrepresented the amount of the reading guide that they completed.
Regardless of the reasons why students did not provide completely accurate data, we found that ∼70% of students reported accurate data, leaving a potential error of ∼30% in the self-report data. A possible way to account for this error is to carry it through to the multiple linear regression model results. By doing so we could expect a possible 30% change in the model estimates for the impacts of fully completing reading guides before class (Table 4). In the most conservative example, a 30% reduction in the lowest model estimate (that from exam 3; Table 4) would be 0.95 ± 0.26 percentage points (out of 100) on the exam, which is still nearly a full percentage point gained by fully completing one reading guide before class. Using this most conservative revised estimate, a student who completed all six reading guides would be predicted to earn ∼4.1–7.3 points (out of 100) higher on his or her exam than if the student completed zero reading guides (controlling for other demographic factors). Therefore, even when accounting for a possible 30% error, full completion of reading guides before an exam may yield approximately a half–letter grade improvement on an exam. Further studies are necessary to determine the reasons for inaccuracies in self-report data in studies such as this and the impacts of self-report inaccuracies on model outcomes.
CONCLUSIONS
In conclusion, optional preclass reading guides were used in a large-enrollment introductory biology course to assist students in reading the textbook. The results suggest that, in our course and with our student population, full completion of reading guides before class is significantly positively correlated with exam performance; however, the mechanism behind this result cannot be determined from this study. Future studies are warranted to investigate the reasons why reading guide completion is positively correlated with exam performance and to determine whether varying reading guide structure or implementation differentially affects student learning.
ACKNOWLEDGMENTS
We thank Kelly Hogan for inspiration in designing our reading guides. We also thank Brian Sato, Adrienne Williams, and Pavan Kadandale for helpful discussions regarding this study and for reviews of this article.
