The Annotated Bibliography and Citation Behavior: Enhancing Student Scholarship in an Undergraduate Biology Course

INTRODUCTION

In 2003, the National Research Council (NRC, 2003) presented its case for making substantive changes to undergraduate biology education, especially for students pursuing biomedical studies. Its report, Bio2010, included the injunction that institutions of higher education “reexamine current curricula in light of changing practices in biological research” (NRC, 2003). One critical element of Bio2010 is the assertion that biological education must move beyond the traditionally lecture-based “biology of the past” and begin to provide ample opportunities for contemporary undergraduates to actively participate in research-based learning. The report suggests that “both library- and laboratory-based” projects might be especially pedagogically useful because experiential course work often cultivates engagement and encourages students to thoroughly investigate topics of interest. This emphasis on improving students' research competencies joins a substantial body of literature that examines the value of enhanced exposure to writing, as well as integration of a multifaceted group of library skills known as “information literacy,” into science curricula.

LITERATURE REVIEW

Early investigative work examining the library inexperience of students in the sciences focused primarily on the lack of library research skills demonstrated by undergraduates enrolled in introductory courses. Martin (1986) observed that these students have trouble successfully “defining the topic and scope of their research projects.” Stachacz and Brennan (1990) describe the use of library instruction as a means of helping students improve topic selection and other research competencies such as “writing scientific reports, analyzing statistical data, using computers in biology, and … reading and searching the scientific literature.” According to Burns (1994) and O'Connell (1996) novice science students demonstrate an inability to recognize and effectively structure the time needed to conduct research and write a substantive paper (procrastination), select an appropriate topic, demonstrate basic library skills, and understand the conventions of plagiarism. In 1998, Sinn surveyed 43 academic libraries to find that approximately 60% offered instruction specifically tailored to students enrolled in biology courses. Recently, Ferguson et al. (2006) asked 151 incoming biology students at the University of Maryland, Baltimore County, to complete an information literacy survey. The results show that even today, many students continue to be unfamiliar with essential library skills. As these studies note, there are many library competencies that students in the biological sciences need to acquire if they are going to be prepared to “tackle increasingly challenging projects with greater independence” (NRC, 2003).

Although academic librarians have long written about the importance of teaching students to effectively use library research tools, members of the scientific community have contributed their support as well. One of the earliest efforts to articulate a need for what librarians today describe as “information literacy” among students in the sciences was Hurd (1998), who noted the rapidly changing nature of scientific literacy. He suggested that science students must be able to recognize important societal forces such as “the emergence of an information age” and “new ways of communication (the cyberworld).” Soon after Hurd's article, the NRC published Being Fluent with Information Technology (NRC, 1999). This document examined the essential technological and critical-thinking skills needed by scientists as members of a rapidly advancing knowledge-based society. Being Fluent also provided an important foundation for the Association of College and Research Libraries' (ACRL) Information Literacy Competency Standards for Higher Education (American Library Association/Association of College and Research Libraries/STS Task Force Information Literacy for Science and Technology, 2000). According to ACRL's publication, students who are information literate determine the extent of information needed; access the needed information effectively and efficiently; evaluate information and its sources critically; incorporate selected information into one's knowledge base; use information effectively to accomplish a specific purpose; understand the economic, legal, and social issues surrounding the use of information; and access and use information ethically and legally.

Because of the widespread dissemination of these two prominent works, members of the library community have worked to further refine ACRL's standards, specifically targeting science-, engineering-, and technology-based disciplines (ALA/ACRL/STS Task Force on Information Literacy for Science and Technology, 2005), even as a number of studies have been published examining various aspects of information literacy within the context of the sciences (Hurd et al., 1999; Leckie and Fullerton, 1999; Newby, 2000; Bowden and DiBenedetto, 2001; Huerta and McMillan, 2004; Lee and MacMillan, 2004; Kearns and Thrasher-Hybl, 2005; Lindquester et al., 2005; Porter, 2005; Ferguson et al., 2006). Three authors have even gone so far as to advocate for the integration of information literacy competencies throughout the entire science curriculum (Brown and Krumholz, 2002; Grafstein, 2002).

Much of the research on information literacy and its significance to students in the sciences has been the result of collaborations between academic librarians and members of the scientific community. These partnerships regularly result in successful interdisciplinary teaching and valuable improvements in student mastery of key library research proficiencies. This project endeavors to demonstrate yet another of these successful collaborations. Our study relies on the triangulation of assessment data to determine the effectiveness of a collaboratively developed research and writing project required of students enrolled in an upper-level undergraduate biology course. By enhancing library-based instruction, adding an annotated bibliography requirement, and making use of multiple assessment techniques, we demonstrate improvements in students' library research abilities that make it possible for them to more effectively and independently complete the kind of increasingly challenging disciplinary projects called for by the NRC.

COLLABORATIVE CONTEXT

Institutional Description

Concordia College, Moorhead is a private, 4-yr liberal arts college enrolling approximately 2800 full-time students. One in four first-year Concordia students initially plan to major in biology or the health sciences. The College's biology department consists of 10 full-time faculty who yearly teach almost 15% of the total student body through selected health-related majors (e.g., biology, nursing, and elementary education). Concordia's biology department consistently graduates more majors than any other department, and it has a strong success rate for placing these students into graduate programs in the health professions.

Course Design

Immunology & Parasitology (BIO 352) is an upper-division elective that is offered once per academic year during spring semester. Over the past 7 yr (the length of this collaboration), BIO 352 has had a mean course enrollment of 17 students. The students who enroll in this course tend to be juniors and seniors with an interest in the health professions, although an increasing number aspire to graduate programs in cellular and molecular biology. Students who take this course are usually biology majors, and throughout our collaboration, approximately 50.6% went on to pursue careers in the health professions, 22.9% went on to graduate school in the biological sciences, and 26.5% did not intend to immediately pursue an advanced degree in the biological sciences (e.g., biology education majors, nursing students, lab technicians).

The lecture portion of BIO 352 concentrates on the human immune system and its regulation at the molecular, cellular, and organismal levels. A 4-wk segment of the course is devoted to the study of parasitology. However, students concentrate the majority of their lab time for this course studying various aspects of the vertebrate immune system. Students are introduced to an assortment of modern immunological techniques ranging from studies in live animals to enzyme-linked immunosorbent assay and immunoblotting techniques. A student's final grade for the course is based on four tests covering lecture and laboratory material as well as the research paper project described in this study. In addition, students prepare and present a short (∼10-min) synopsis of their research project to the class, typically as a PowerPoint presentation. The final research project grade (based on the combined points from the topic proposal, annotated bibliography, final paper, and oral presentation) is equal to 1.5 times the value of an examination in terms of the student's overall grade for the course.

Library Instruction

From the beginning, students enrolled in BIO 352 have received varying degrees of library instruction. Although primarily juniors and seniors, most of these students have had little if any experience using the library research tools that are required to successfully complete a scientific writing project of this nature and size. Therefore, during the library sessions designed for this course, students have typically been introduced to advanced databases such as Current Contents, ScienceDirect, and Medline. Because they have historically voiced concerns about topic selection, librarians have also identified a number of highly specialized, relevant reference works and they have encouraged students to use them to generate or focus their topic ideas. Over time, librarians developed a website specifically tailored to this course. The current version can be found at http://www4.cord.edu/library/rux/biology/352/home.htm. This site provides an up-to-date list of reference sources, links to the library's science-based databases, a collection of high-quality Internet sites, and an electronic version of the project description and an annotated bibliography grading rubric.

We decided early in this collaboration that the instructor's research paper project might benefit from being further segmented into smaller, more manageable research portions across the course as the first version of the assignment seemed a bit ambitious/ambiguous (Supplemental Material 1). For example, students were told they could use “review articles, media sources, and Internet sources” in their final papers but that these sources were to be “distinct from primary literature which is found only in scientific journals.” The description offered no criteria for determining what a scientific journal was nor did it give suggestions for how students might locate such journal articles. And, although the faculty member actively encouraged students to consult with him “about potential topics and how to find information,” students struggled, often using inappropriate periodical articles and unreliable Internet sites.

Although there were things about this assignment that needed developing, from the beginning its laddered approach was a strength. The original description included separate dates for topic approval, an annotated bibliography of at least three primary sources, and a final paper. Librarians encouraged the faculty member to build on this approach, further spacing assignment dates and adding a formal, written paper proposal to further help students navigate topic selection.

Unfortunately, based on the number of poor bibliographies generated by the first assignment description, we quickly realized that if this component of the research project was going to be a meaningful step toward writing a final paper, this portion of the project description would need to be greatly expanded. Therefore, over time, the final research paper project description has developed considerably (Supplemental Material 2). The instructor now requires students to begin their library work earlier in the course. A date for potential topic ideas coincides with the library session so that students can explore their ideas with a librarian. A written, 500-word paper proposal is due shortly after the library session. Students are given detailed instructions about writing an annotated bibliography and their source requirements for this portion of the research project have increased, requiring examination of at least eight sources of information. Additionally, the instructor provides a detailed grading rubric that clarifies his expectations of students' library sources (Supplemental Material 3).

MATERIALS AND METHODS

Assessing the effectiveness of this project has changed a great deal over time. Although we gathered a variety of assessment data throughout the 7 yr of this collaborative project, this article focuses mainly on the results gathered from 2004 to 2006. For us, these past 3 yr represent the time frame during which our assessment tools and the conditions under which we applied them stabilized, allowing us to gather data that was clearly comparable. For example, during these later years, we were no longer making larger structural changes to the course or testing our assessment tools, but rather we were fine-tuning what was already in place.

Assessment Techniques and Evaluation

For novice undergraduate biology students, locating and evaluating primary scientific literature can be especially challenging; yet, as one author notes, this skill is “as important to scientific research as the careful design of adequate controls” (Porter, 2005). The primary constant throughout this collaboration has been the instructor's goal of asking students to identify and critically analyze several primary scientific journal articles within the context of a final paper. Therefore, one especially useful method for assessing students' success with this requirement has been bibliographic analysis.

Although this project is exempt under Concordia's Institutional Review Board (IRB) Policies and Procedures (http://www4.cord.edu/acadAffairs/irb/8Exempt%20Research.doc), students were asked to voluntarily return a copy of their annotated bibliography to their instructor and to sign an IRB release form if they were willing to participate in this study. (The total response rate from 2004 to 2006 was 75%, with students at all points of the grading spectrum voluntarily returning bibliographies.) Once these bibliographies were returned, a librarian examined each one, verifying individual citations and using a standardized categorization scheme to code them by source type. The following categories were used: primary scientific literature, reference sources (e.g., encyclopedias, dictionaries); secondary journal articles (e.g., scientific review articles, articles from trade/professional or popular presses); Internet sites of good quality, Internet sites of poor quality, sources that could not be verified (e.g., incomplete or incorrect citations); and finally, sources called “other” (e.g., course textbooks, monographs, pamphlets, government documents, speeches, videos).

Although most sources cited by students in their annotated bibliographies fit into a single categorization, there have been instances where students include items that could fit into more than one category (e.g., Internet newspaper articles, online books, Internet video clips). Although this occurs infrequently, the librarian categorized these sources based on the search strategy a student would most likely use to locate and/or access the source (e.g., electronic newspaper articles were considered Internet sources unless available via one of the library's subscription databases; online books were categorized as either other or Internet sources, depending upon their inclusion in the library's online catalog).

Two additional data points were part of the librarian's bibliographic analysis. Each source annotation was carefully examined for evidence of plagiarism and evaluated for overall quality. Although we considered these two categories independent of one another, we recognize there is certainly a relationship between them. For example, although not all poorly constructed annotations were the result of plagiarism, when plagiarism did exist, the librarian automatically rated the annotation quality as “poor.” Also, although determining whether a student plagiarized seemed relatively objective in most cases (students copied and pasted complete sentences, or in rare cases, entire abstracts from online databases into their bibliographies), evaluating annotation quality was admittedly a more subjective undertaking. Nevertheless, over the years the students in this course have been given increasingly precise criteria for writing appropriate source annotations, and eventually the librarian was guided by specific requirements as delineated by the grading rubric (Supplemental Material 3). The annotated bibliography assignment excerpt (2006) shown below is taken from the most recent version of the instructor's annotated bibliography description.

From the beginning, students struggled to prepare annotated bibliographies. Not only did they demonstrate a lack of understanding about what kinds of sources a good bibliography should contain, but students had a great deal of difficulty writing meaningful, high-quality source annotations and they showed little concern for or knowledge of scientific citation conventions. Therefore, in 2005 we began using a grading rubric as a means of further structuring the annotated bibliography portion of the research paper project and also as a way of providing additional feedback on the quality of each student's sources before he or she tried to write the final paper. Students were given this rubric as part of their introduction to the overall paper project, and they were able to access it electronically at any time during the preparation of their bibliographies (Supplemental Material 3). In addition to improving the consistency and objectivity of the grading process, one especially valuable aspect of the rubric has been that it allows collaborative grading. A librarian assesses bibliographies for currency, appropriate source level, annotation quality, and documentation conventions. The course instructor evaluates overall source content and authority. Both instructor and librarian provide written feedback to students who may then make adjustments to their research if needed prior to handing in their final paper. Although we have not done a yearly comparison of completed assessment rubrics for this study, we have strong anecdotal evidence that using the rubric has contributed to a substantial improvement in the overall quality of students' annotated bibliographies.

In 2004, we also began to examine student perceptions of the BIO 352 research paper project by asking them to complete a questionnaire upon completion of this course. Again, students were reminded that participation in the survey was voluntary, and they were asked to sign a release allowing us to use their data. A copy of the full survey instrument may be obtained by contacting us.

RESULTS

Annotated Bibliographies

This project shows improvement in students' annotated bibliographies in three especially important areas: appropriate source type use, annotation quality, and plagiarism reduction. Figures 1–3 represent the most recent 3 yr of bibliographic analyses in each of these areas. (Percentages represent the proportion of total bibliographies analyzed for each individual year, but they may not add up to 100%, because they have been rounded.)

Figure 1 shows a small, but meaningful, increase in the overall percentage of primary scientific literature articles examined by students for their annotated bibliographies. In 2004, primary scientific literature represented 46% of the total number of sources cited by students. By 2006, that number increased to 51%. Although the inclusion of secondary resources (reference materials and review or trade/popular press articles) has remained fairly constant over the past 3 yr, totaling 36% in 2004, 34% in 2005, and 41% in 2006, we have managed to reduce the inclusion of more tertiary sources represented by the Internet and other categories (17% in 2004, 17% in 2005, and only 8% in 2006). It should also be noted that when students were initially given this assignment, they were asked only to vaguely consider three sources, whereas students are now required to locate, critically evaluate, and compare/contrast at least eight information sources in their annotated bibliographies (Supplemental Material 1 and 2). Therefore, successfully meeting the carefully structured annotated bibliography requirements today is a more rigorous task for students than it has been in the past.

Figure 2 illustrates another trend in the quality of the source annotations provided by students. As previously noted, the earliest version of this research project description contained no concrete instructions to students about writing a high-quality annotated bibliography. Unhappily, the result was that in 2000, one-third of the very first class turned in bibliographies with extremely poor (e.g., one or two sentences, purely descriptive abstracts rather than comparative/evaluative paragraphs, plagiarized language) or no source annotations. In 2001, the number of students who wrote poorly constructed source annotations jumped to 60% of the total class; in 2002, this percentage fell slightly to 57%. In 2004, the class hit an all-time high, with 64% of students turning in bibliographies demonstrating extremely poor source annotation quality.

One explanation for poor source annotation quality was likely a lack of clear guidelines. Many students told us they had never been asked to do an annotated bibliography before and that they had no idea where to begin. Some students even admitted they did not know what an annotated bibliography was. However, another contributing factor is likely to have been the ease with which students began accessing electronic library research tools. Plagiarism became an especially significant problem within the annotated bibliographies during 2001–2004. These dates directly coincide with the addition to Concordia's library of several electronic versions of the scientific indices students needed for this assignment.

Certainly not all students who created poor-quality source annotations within their bibliographies plagiarized directly from an electronic database, but many did, particularly during the early years of this study. These students either copied and pasted portions of database abstracts or entire abstracts directly into their own annotated bibliography. More frequently, however, students simply rephrased a source's online database abstract rather than actually obtaining, reading, and reflecting upon the study itself.

Based on the bibliographies students turned in during the earliest years, it was painfully obvious that students needed more complete instructions to help them understand the purpose, scope, and usefulness of doing an annotated bibliography. Due to the frequency with which students plagiarized, it was also clear that they needed clearer, more explicit directions for how to construct an annotated bibliography. Therefore, in 2004 the instructor expanded the description for the annotated bibliography, adding several specific evaluative criteria for students to consider when writing their source annotations (e.g., authority, audience, compare and contrast, applicability, bias). Unfortunately, even with the new written guidelines, the instructor did not explicitly tell students that merely rephrasing database abstracts would constitute a misuse of sources, and a number of students still relied on this plagiaristic strategy. However, in 2005, we included written and verbal injunctions against this practice. Once these instructions were further reinforced by our grading rubric, we began to see major improvements in the originality, evaluative nature, and comparative quality of students' annotated bibliographies.

Student Questionnaires

We began making use of a research project evaluation questionnaire in 2004. Students are given this questionnaire after they complete the course and they are ensured their participation is voluntary. Figure 4 shows that student response rates have been high.

In general, students seem to enjoy this course. They report high levels of satisfaction with their instructor, librarians, and their overall experience with writing a scientific research paper. Because their comments have been uniformly positive, student responses to five of the 12 survey questions are particularly worthy of note here.

We were interested in how students might rank their experience with this paper project compared with other projects they had been assigned in other courses. We also wanted to gauge whether the means we were using to increase rigor of this project would result in any increase in dissatisfaction among students. Based on their interest in the project, the project's difficulty, and the extent to which they felt “stressed out” by their library research, students were asked to evaluate the BIO 352 writing project in comparison with research papers they had completed in other biology and nonbiology courses. Figures 5–7 display response data over time for each of these questions.

We were pleased to note that despite this assignment's increasing rigor, many students continue to rate their interest in the BIO 352 writing project as “higher” than their interest in writing research papers for other courses. Figure 5 shows that over the past 3 yr, in comparison with other biology and nonbiology courses, students consistently report strong levels of interest in this particular writing project.

However, student responses to questions about project difficulty are mixed. Figure 6 illustrates student responses to questions asking them to compare their perceived difficulty with the BIO 352 project to research paper assignments they have completed for other courses.

In general, a higher proportion of students continue to rank the overall difficulty of this project as “about the same” or “lower” than research paper projects they have completed for courses both in and out of the biological sciences. Yet, there is evidence that some students perceive this particular project as being more difficult than papers they have written in other courses, especially nonbiology courses. Although some students continue to report finding this project demanding, we remain optimistic about this aspect of the project for several reasons. First, BIO 352 is a course intended for juniors and seniors who are majoring in biology; thus, the content of the course necessarily requires assignments that challenge and stretch highly motivated upperclass students. Second, we do not see an inordinately dramatic change in students' perceptions of difficulty over time (i.e., at this point, there is no trend toward an increased perception of difficulty). Finally, students undoubtedly have varying perceptions of the difficulty of this writing project for any number of variables that we cannot control. For example, some students will have taken fewer courses requiring significant amounts of writing than others. It is also likely that students have differing perceptions of the challenging nature of the course and the writing based on their own motivations for enrolling in the course (e.g., they liked the professor, they thought the parasitology portion of the course would be “cool,” a friend may have told them the class was easy, fun, or interesting).

Overall, we are satisfied with student rankings in terms of the writing project's difficulty, especially when viewed in conjunction with the assignment's increasing rigor and the improvements most student work demonstrates. Although some students continue to find this assignment difficult, the majority complete this project effectively. Additionally, based on the number of students reporting the project as “about the same” as paper assignments they receive in other biology courses, we think the difficulty of this particular writing project falls in line nicely with the members of our Biology Department's expectations.

Regrettably, students continue to report unease with the library research portion of this project. Figure 7 represents students' responses to questions asking them to rate their “library stress” during the BIO 352 project compared with research paper projects completed for other courses.

Although students consistently rank library support for the project as “excellent” (Figure 8), librarians had hoped that their close involvement with this course would result in a trend reducing student library anxiety compared with the library stress students may have experienced in courses where less collaboration between faculty and librarians occurs. Unfortunately, students still report higher rates of library stress during this project than they do in comparison to other biology course projects. And a definite upward trend is evident for high levels of library stress when the BIO 352 project is compared with writing projects in other nonbiology courses.

Although librarians continue to plan ways to alleviate students' library stress, we think one explanation for why some students report high levels of library stress during this project relates to the amount of prior library work students have been asked to do in other courses. If, as we suspect, students have not been required to do much library research in their lower-level courses, the learning curve for successfully completing the BIO 352 research assignment is undeniably steep. Therefore, the students' response to this question lends some credence to the argument for greater integration of information literacy throughout the entire curriculum so that students can build their research skills throughout the undergraduate experience rather than trying to learn everything they need to know all at once for individual course assignments.

Figure 8 shows how students ranked their overall experience with the BIO 352 research paper project over the past 3 yr.

Students used a four-point Likert scale of excellent, good, fair, or poor to rank their instructor's clarity of expectations, their overall learning experience, and their satisfaction with the support they got from librarians. Although we realize that student perceptions data can be highly variable and has the potential for overgeneralization, Figure 8 shows reasonably stable upward momentum in the students' responses in all three areas (instructor clarity, learning experience, and library support) over the past 3 yr, when the most significant of our changes were made to this assignment. Again, we feel especially good about these rankings in light of the overall increase in rigor of this project and student performance.

DISCUSSION

This study demonstrates the efficacy and pedagogical value of a collaborative teaching approach specifically designed to enhance student information literacy competencies within the context of writing a formal scientific research paper. We think significant improvements have been made in several key areas.

Source Use

Based on the difficulty early students had referencing reliable sources, our earliest library instruction efforts focused primarily on teaching students how to find suitable scientific studies. During these initial years many students struggled to locate primary scientific literature and regularly resorted to poor-quality secondary review articles from trade/popular publications or the Internet. In 2000, a full 10% of all the sources students selected were poor selections found using the Internet and 41% qualified as secondary rather than primary scientific articles. However, as library research technology improved in subsequent years, we intentionally shifted to a focus on ways to improve students' understanding of and ability to interpret and critique primary scientific literature rather than simply how to find it.

We think that going through the process of not only locating but also critically evaluating several scientific sources for an annotated bibliography has been particularly useful. By asking students to refine their bibliography into a research paper using only their three best studies, students must carefully consider and focus in on a small area of high-quality research. As a result, students by and large do not seem to get as mired down by the search process and they tend not to waste time looking through large quantities of low-quality sources. Of all the sources cited by students in their annotated bibliographies in 2006, 51% were considered excellent primary research studies (Figure 1). And, although students do continue to use secondary scientific literature in their bibliographies, the quality of these sources is also greatly improved. Instead of articles from periodicals such as Time, Newsweek, and USA Today, today's students reference high-quality review articles that are in scholarly scientific journals. By clearly articulating source expectations, coordinating library instruction, and intentionally asking students to place an emphasis on scientific quality rather than quantity, one of the main goals of this project (engaging students with the primary scientific literature of immunology/parasitology) has been met.

Annotated Bibliographies

In a chapter titled “Writing and Other Technologies,” Leamnson (1999) identifies an important challenge that certainly played out during this study. He notes, “No matter how much data (true or otherwise) becomes available, learning will still result only when some person with a well-structured brain sorts it out and makes sense of it.” During the years of this collaboration, information technology has undergone several dramatic transformations. Concordia's library is now able to provide access to a myriad of electronic and print resources that simply did not exist for students during the earliest years of this study. For a novice biologist today (one whose brain may not be so “well structured”), making sense out of the vast amount of available scientific information can be difficult. The library research challenges facing contemporary undergraduates in Bio 352 have expanded well beyond the difficulties of locating information to complex issues of evaluation, critique, and synthesis. Students no longer struggle to find information; yet, when it comes to selecting reputable sources, many are overwhelmed by their options. The faculty instructor has always hoped that this assignment might give students an opportunity to invest their time reading and reflecting on a few high-quality scientific studies, rather than superficially summarizing the first five articles that result from a search on Medline for, e.g., acquired immune deficiency syndrome. Therefore, one of the outcomes for this assignment has been to go beyond simply showing students how to use certain library tools.

As electronic databases improved their search interfaces and became more readily available, it became painfully clear that the research assignment needed to be carefully structured so that students spent more time earlier in the semester thinking and writing about reputable scientific research studies that were especially relevant to their topic. Because students tended to select sources at the last possible minute, based solely on their ease of quick access rather than quality or in many cases relevance, we increased the requirements for and rigor of the annotated bibliography assignment considerably. Before making these improvements (2000–2004), as many as 86% of the students enrolled in this course demonstrated a lack of engagement with their sources by handing in annotated bibliographies that were hastily done, plagiarized, or simply of fair-to-poor quality.

Today, students are given specific guidelines for the types of sources that must be included, annotation length, and annotation content. They are told they cannot simply summarize sources but that they must compare studies to one another and discuss the way in which a source might be used in their final paper. Students are also asked to note which of their sources they feel were most useful to their understanding of the topic as well as those that may be of less import. Plagiarism is clearly defined both orally and in the written assignment description. Although it took several years to quantifiably improve the evaluative quality of students' annotated bibliographies, we think many of the bibliographies completed by students in 2006 were of the highest quality since the project's inception. Additionally, several students have commented both in the surveys and informally in person that although it is a demanding assignment, they especially appreciate the annotated bibliography portion of the research project as it “makes writing the final paper easier,” and it serves as an effective means for helping them avoid procrastination.

Plagiarism

The assessment tools used during this project have proved especially valuable. Results from each tool (annotated bibliography analysis, grading rubric, and student survey) have pushed the instructor and librarians to immediately target instruction at issues where a need for improvement has been identified. For example, during the early years of data collection, an especially high rate of copying and pasting in the annotated bibliographies signaled a definite need to further develop this portion of the research project. In 2001, 50% of the students enrolled in Bio 352 either knowingly or unknowingly plagiarized their source annotations. In 2002, the percentage dropped; yet, nearly one-third of the entire class (29%) continued to misuse source material. Finally, in 2004, the class hit an all-time high when 64% of the annotated bibliographies contained evidence of some form of plagiaristic behavior. Regrettably, we learned that students simply either did not understand what plagiarism was or did not want (or have the time) to invest much effort in the evaluative/comparative analyses their instructor wanted them to demonstrate in an annotated bibliography. Interestingly, a student's use of plagiarism generally did not correlate with his or her overall course performance. Some of the top students from each class, as evaluated through tests, labs, presentations, and other methods, were found to have plagiarized; conversely, there were poorer-performing students who did not.

Once the problem of plagiarism became obvious, focused efforts at dealing with it were particularly successful and satisfying. We added explicit written definitions and oral explanations of plagiarism and its seriousness to the assignment description and classroom presentations. Students were given examples of especially well-constructed, evaluative source annotations, and two high-quality annotated bibliographies from prior classes were distributed (with permission) as tangible evidence of what the instructor expected. Heightening students' awareness of the criteria used to evaluate their annotations for plagiarism through the grading rubric was also critical to the successful outcome of this initiative. The result of this work has been an especially satisfying drop in plagiarism over the past 2 yr. In 2005, only 17% of the bibliographies evidenced plagiarism, and remarkably, in 2006, we found no evidence of plagiarism in any of the participating students' annotated bibliographies.

Student Satisfaction

Our data suggest that student satisfaction with the Bio 352 research project continues to be high, despite the instructor's increasingly rigorous expectations (Figures 5–8). During 2004–2006, students self-reported high levels of interest in this project as compared to papers written in other biology and nonbiology courses (Figure 5). Although students reported some concerns with the project's difficulty (Figure 6) and library stress over time (Figure 7), many felt that this writing project was not out of line when compared to papers written in their other courses. Ultimately, this may be a student's first experience having to write a scientific research paper of this magnitude, and we are encouraged that over the course of our study, students continued to report high levels of satisfaction with the project's clarity of expectations, their learning experience, and library support for Bio 352 (Figure 8).

Faculty and Librarian Satisfaction

Over the years, this instructor's initial dissatisfaction with the described library research and writing project has transformed into a productive and rewarding experience for instructor and students alike. Two incrementally implemented changes seem to be especially responsible for enriching this particular collaboration. First, we intentionally worked to improve student information literacy competencies in the biological sciences by establishing close working relationships between the biology faculty instructor and library faculty. Meticulous planning and direct communication between the instructor and librarians resulted in an easily accessible group of knowledgeable individuals to whom undergraduates could bring questions concerning various aspects of their assignment and research challenges throughout the writing process. Second, by breaking the project into discrete and well-defined sections rather than an “all-or-nothing” paper due at the end of the semester, the instructor is able to monitor and guide the project throughout its completion, providing valuable feedback to students at several points before they hand in their final paper.

We attempted to quantify several pedagogical elements of this project over the years as a means of triangulating enough data to strengthen our argument for the success of this collaboration. Yet, there is an important element of this project that we found difficult to quantifiably assess. Although we can document important improvements in students' performance of certain tasks leading up to their writing of the final paper, we continue to rely on the faculty instructor's admittedly subjective analysis as to the overall quality of the final papers. Although we did not attempt a comparison of research paper grade averages for each individual class over time, qualitative evidence reported by the course instructor suggests our collaborative efforts at enhancing this assignment have been successful. Restructuring the assignment in a way that allows students to practice writing a science-based, brief project proposal followed by the evaluative annotated bibliography, requires students to spend more time thinking critically and writing about their topic. The instructor thinks that the reorganization of this research project has resulted in the inclusion of source materials that are more appropriately selected, better understood, and carefully evaluated, as well as final papers that are more highly reasoned and coherent, structured more tightly, and much more readable than earlier papers.

Although we made no attempt to quantifiably link the reduction of plagiarism in students' annotated bibliographies to the occurrence of plagiarism in their final papers, the instructor thinks that students are much less inclined to plagiarize within the body of their papers than they were at the start of this collaborative project. Students who are required to invest time and effort in the initial research stages of this assignment, particularly the annotated bibliography, often report fewer difficulties and less anxiety when it comes to writing their final papers. They are more familiar with their sources and, as a result, they feel more confident about their own writing. Also, as noted previously, the collaborative nature of this project provides students with a wide range of available expertise and support as they navigate through the research paper assignment. This collaboration has benefited the faculty instructor, allowing him to teach content and focus primarily on science questions related to the research project, whereas librarians benefit from the relationship by gaining familiarity with disciplinary concerns and an increase in the opportunities for teaching students about effective source location, evaluation, and use.

CONCLUSION

In a recent chapter titled, “Developing Faculty-Librarian Partnerships,” Curzon (2004) describes several commonly used programmatic models used by librarians to reach novice student researchers. She ends her chapter by noting,

Certainly, the collaborative model described here has taken time and effort to develop and assess. Yet, the many encouraging outcomes resulting from our work together have been extremely rewarding for the biology faculty instructor, librarians, and students enrolled in Bio 352. The bottom line is that students seem to identify and critique better sources, organize their research projects more efficiently, and use information more effectively in their final papers. One of the additional benefits of this project has been increasingly common student comments such as, “I enjoyed being able to look at a specific aspect of immunology that related to my background and interests,” “I was interested in my topic way more than I thought I would be,” “I never knew there were so many different aspects to immunology/parasitology,” “it [the paper] gave me a much better understanding of one specific area, making me more interested in the topic in general,” and “makes you realize the amount of work and time behind the information we are given in class so you have a much higher appreciation for research,” in response to the student questionnaire. A member of the Department of Biological Sciences at the University of the Sciences in Philadelphia makes the following observation: We have attempted to quantify the success of significant pedagogical refinements made to a term paper project assigned to novice researchers in an upper-division, undergraduate biology course. Like Porter (2005), we ultimately found the collaborative process valuable, interesting, and most importantly, beneficial to students.

REFERENCES