|
|
|||||||
Articles |



*Office of Undergraduate Evaluation and Research, College of Letters and Science, University of California-Los Angeles; and
Department of Biological Chemistry, UCLA Medical School, University of California-Los Angeles, Los Angeles, CA 90095
Submitted February 13, 2006; Revised June 14, 2006; Accepted July 14, 2006
Monitoring Editor: Martha Grossel
| ABSTRACT |
|---|
|
|
|---|
| INTRODUCTION |
|---|
|
|
|---|
The program explored two hypotheses: 1) Training undergraduates in the primary literature is an effective tool for increasing scientific literacy; and 2) literature training will significantly facilitate the students' transition to postgraduate study. A number of assessment techniques have been used to gauge the outcomes of program participation. Annual student evaluations have been distributed and analyzed since the program's inception. In addition, during the 20042005 academic year, program alumni were contacted to participate in an e-mail survey, and interviews were held with a number of professors who served as advisors to multiple program participants. Findings from the three assessments indicate that participation in the program is advantageous to students not only during their undergraduate experience but also during the application and interview process for graduate and medical school and throughout their graduate training.
| BACKGROUND INFORMATION |
|---|
|
|
|---|
Muench (2000) reports on the "unique potential" (p. 255) of primary literature to familiarize students with the nature of scientific reasoning and communication. Smith (2001) states that scientists must be able to read, comprehend, and discuss primary literature, but warns that "many science majors reach their junior and senior years fearing and being intimidated by having to read and interpret primary literature" (p. 466). A number of instructors have addressed this deficiency in the standard curriculum by using primary literature as part of an undergraduate lecture class. DebBurman (2002) found that the utilization of this pedagogical approach successfully imparts strong scientific subject content and develops sophisticated scientific and intellectual skills, and Mulnix (2003) indicated that her students reported gains in self-confidence, communication skills, and their ability to read and understand the methods and results sections of journal articles.
Whereas it is clear that these two active processes provide different benefits to students, there appears to be little or no research on the effect of engaging students in the complementary processes of conducting research and participating in an intensive journal club experience. In addition, none of the studies cited above examine the long-term effects of these programs on students as they move from undergraduate to graduate-level study. This article addresses this gap in the literature by reporting on the structure and outcomes of an innovative program at UCLA.
In 1998, the Howard Hughes Medical Institute (HHMI) provided a grant to UCLA's undergraduate College of Letters and Science to support a number of projects designed to engage undergraduate students in research and encourage them to pursue graduate education in the sciences. Part of this funding was used to create the Howard Hughes Undergraduate Research Program (HHURP), a collaboration between the College and the University's Medical School. Under this competitive program, students who are engaged in research in a UCLA lab and are interested in pursuing graduate degrees (M.S., Ph.D., M.D., or M.D./Ph.D.) are encouraged to apply in the fall of their junior year to become Howard Hughes scholars. Applications are judged on the basis of the students' academic accomplishments, career goals, and research plans; a letter of support from their research mentor; and an interview with the three program coordinators. Approximately seven students are accepted each year for this 18-mo program that offers the following components: a weekly journal club, research presentations, seminar speakers, career guidance, and a scholarship paid out over the course of the program. Thus, the aggregate class size each year for juniors and seniors is typically 14 students.
The students selected for the program represent a range of academic interests and backgrounds. Over the last 5 years, 23 males and 20 females have participated. They encompass many ethnicities and include underrepresented minorities, but these parameters are not formally monitored. Their majors, in order of popularity, are as follows: molecular, cellular, and developmental biology; microbiology; neuroscience; biochemistry; psychobiology; ecology and evolutionary biology; and physiological sciences.
Three faculty members are dedicated to different aspects of the program, not including the students' research mentors. This low student-to-faculty ratio is essential for realizing the program goals. The involvement of the Medical School faculty, including Dr. Carey and Dr. Colicelli as program coordinators, distinguishes this program in three ways. First, the program coordinators, based on their experience teaching at UCLA's Medical School, recognized that the lack of exposure to primary literature is a major weakness of incoming graduate students. A survey of first-year students in the UCLA interdepartmental graduate program for biomedical sciences found that only a third had any prior experience in a journal club learning environment. Instruction in medical and graduate school programs requires students to interpret and build on the primary literature, and these skills are also essential for students to continue learning independently. Consequently, a portion of the graduate curriculum is devoted to what might be considered remedial learning (development of skills needed to evaluate the primary literature). The HHURP is intended to remedy this situation by allowing them to develop these skills as undergraduates. Second, the coordinators designed the journal club, both in terms of the articles selected and the level of discussion expected, to treat the HHURP scholars like graduate students rather than undergraduates. They felt this was the best approach for preparing students for a graduate school environment that relies on studentmentor and peerpeer learning interactions. Third, the coordinators serve as mentors to the students, providing academic and career counseling as well as feedback on scientific knowledge and presentation skills.
| PROGRAM COMPONENTS |
|---|
|
|
|---|
Weekly Journal Club
The central component of the HHURP is student participation in a weekly seminar entitled, "Topics in Contemporary Biology." Most of the class sessions are run as a journal club, with one student presenting the results of a published research article. The remaining sessions are entitled "research seminars" and provide students with the opportunity to present the results of their own research. Each student is expected to present both a journal article and his or her own research once each academic year. The sessions are held on Friday afternoons from 4 PM to 6 PM.
For the journal club presentations, students can select an article from a list drawn up by the coordinators from journals such as Science and Nature, or they can get faculty approval for an article selected on their own. The week before their presentation is to be made, the students meet with one of the coordinators for 1 to 2 h to discuss any questions they have about the article and to receive input on their electronic presentation. The students are also responsible for creating a list of questions focused on key techniques and scientific principles discussed in the article. These questions, along with the selected research article, are distributed in advance to the other HHURP scholars, who are each assigned one question to which they must respond during the presentation. This process ensures that all students become "experts" in one aspect of the presentation and that they are actively engaged in each of the presentations.
A UCLA faculty member with expertise in the subject area of the journal article being discussed is invited to serve as a resource to the student making the presentation and to attend the class to provide feedback and insights based on his or her own related research. This individual's presence is intended not only to provide additional content expertise, often including unpublished advances in the field, but also to increase the students' comfort level in discussing scientific research with future colleagues.
After the presentation, the students participate in a round-table discussion during which they react to a series of focus points distributed in advance of the class. This discussion allows the students to evaluate the presenter's conclusions and provide constructive suggestions about the research described. At the end of the class, the faculty coordinators meet privately with the presenter to provide them with critical feedback, both to point out effective content and presentation techniques and to guide them in areas that can be strengthened for future presentations.
Research Presentation Opportunities
HHURP scholars are provided with a number of opportunities for presenting their own research. As discussed earlier, each student presents findings from his or her own research projects once a year during a "research seminar." As with the journal club presentations, students meet with a coordinator before their presentation to discuss content and visuals, and they are expected to use electronic presentation software. Background research papers or relevant review articles may be distributed in advance to familiarize the other scholars with the presenter's area of research, because the student's research has not been published. The students' research mentor and other lab personnel are invited to attend these presentations. Students receive feedback on their presentations from the coordinators after the session.
The scholars are also required to participate in UCLA's undergraduate Science Poster Day, which is held each spring. This event is open to all undergraduates who have completed one or more quarters of research on a life, biomedical, or physical science project. HHURP students use their peers and the program facilitators as resources in preparing their posters. During Poster Day, students make themselves available at assigned times to respond to questions from other students, faculty, and assigned judges. Dean's Prizes are awarded for outstanding research. Students are also encouraged to attend national scientific conferences and are eligible for funding to cover meeting expenses if they submit an abstract for a poster presentation. They also become involved in editing and contributing to The UCLA Undergraduate Science Journal, a student-run academic journal featuring original research articles, review articles, and faculty interviews. Finally, during the annual spring banquet honoring HHURP scholars and their mentors, the graduating seniors give formal research presentations.
Seminar Speakers
To enhance the students' access to exceptionally accomplished biomedical researchers and to give them the chance to interact informally with these individuals, the scholars select and invite a number of speakers to campus each year. These individuals are asked to give a seminar presentation and to join the HHURP scholars for lunch. Research articles from the invited speakers are incorporated into the journal club presentations to promote greater interaction between the scholars and the guest. During a lunch, the students are able to talk with the guest about their own research projects and to get input and advice on career plans. One of the seniors introduces the guest speaker at the start of the seminar, and other students host the visiting scholar throughout the day, escorting him or her around campus.
Career Guidance
One of the three program faculty meets individually with each student during the year to provide him or her with feedback on performance in the program as well as career guidance and advice on graduate school admissions. The program-related conversations are focused on helping the students develop skills for working as part of a group. In terms of career advice, the faculty coordinators have experience on admissions committees at the UCLA Medical School and interdepartmental graduate program and are able to provide insightful guidance about what graduate programs look for in successful applicants. One of the coordinators conducts an M.D./Ph.D. workshop that brings faculty from UCLA's School of Medicine to speak with students about the different postgraduate tracks available at the school. Students are also encouraged to speak with other researchers in a variety of career paths. The faculty track students during their graduate application and interview processes and are available to provide letters of reference and advice as the students move through this process. During the journal club sessions, the seniors who are in the midst of applying to medical and graduate school are able to share their experiences with their peers.
| PROGRAM OBJECTIVES |
|---|
|
|
|---|
First, the program exposes students to a broad range of biomedical research topics at a sophisticated level well beyond what is taught in lecture classes. The intent is to give them a sense of how scientists in different areas of biology identify relevant questions and test their hypotheses. This is done through an examination of current primary literature, not simply by having students summarize the results and conclusions of other scientists. This emphasis on discovery complements the students' concurrent experience in the laboratory.
Second, the students are expected to develop the skills necessary for critically analyzing scientific research articles. This process not only strengthens their ability to evaluate others' research as they move forward in their own scientific careers, but ultimately affects their own efforts as researchers. The critical analysis of research is a crucial skill for graduate students and scientists. The types of issues that the students are instructed to examine as they prepare their journal club presentations include the following:
Third, the students practice effective scientific communication by doing two presentations during the course of the academic yearone on a journal article of their choosing and one on their own researchand by observing their classmates' presentations. Preparing and delivering basic research and journal club presentations helps students to understand how basic science is performed and requires that they master the content of the articles to the extent that they can coherently relay the central ideas to the group. Through meetings with the faculty coordinators and a set of instructions available to all the HHURP scholars, the students are given guidance on the basic techniques for making presentations understandable and engaging. These range from simple guidelines about making text and figures sufficiently large to be read from the back of the room to more sophisticated advice about focusing on key points and being prepared for the questions and discussions that their presentation might evoke.
The structure of the journal club ensures broad-based participation. When students do not actually make the presentations, they are assigned a technical question to answer, which means that all students read the article and each one of them becomes the "expert" in one small area addressed in the article. Finally, students are expected to present in front of noncourse faculty, specifically their mentors and subject area specialists, as well as their peers. They are also able to talk informally with the seminar speakers who visit the campus. These interactions with experienced researchers build the students' confidence and convey that they are actually part of a community of scholars.
| ASSESSMENT PROCESS AND RESULTS |
|---|
|
|
|---|
In interpreting these assessment results, it is important to be aware that these students had an impressive record of academic achievement before their acceptance into the HHURP. In addition to their participation in the program, each of them also benefited from their involvement in laboratory research and the mentoring provided by the professors and graduate students with whom they worked. Although these assessment efforts have attempted to differentiate the effects of the program from the impact of other types of support they received, this cannot be done definitively without a comparison to a control group. Such a group was not identifiable on the campus. Therefore, these findings should be interpreted in that context.
Annual Evaluations
Since the 19981999 academic year, the faculty coordinators have distributed an annual student evaluation to the HHURP scholars during the spring quarter. The surveys used a 19 scale, with 1 indicating "disagree strongly" and 9 indicating "agree strongly." The use of the 9-point range is largely historical and based on the evaluation scales used by many UCLA departments for seminars and didactic courses. In particular, the Biological Chemistry Departmental Teaching Committee feels that an expanded scale allows students to more accurately express their positive or negative feelings about a course and helps the Teaching Committee to distinguish above average to exceptional performance in the class. Over this period, 64 responses were received, including students who completed the evaluations twice (during their junior and senior years). All students completed the annual evaluation.
Table 1 presents the mean scores for the evaluations conducted between 19981999 and 20032004.
|
Cumulatively, these results provide a uniformly positive assessment of the program in terms of the goals of reading and presenting scientific research. It should be noted that a self-assessment of being "more comfortable" reading and presenting papers is likely to reflect not just gaining skill in these tasks but also becoming familiar with something with which they previously had little experience. The interesting discrepancy between the students' high ranking for presenting their own research (8.68) and the lower overall ranking for the research seminars (8.19) might be attributed to the relatively low 20012002 score (as detailed above) or the fact that students are involved in a variety of research projects that are unfamiliar to their colleagues, so the presentations may be more challenging to understand, especially given the lack of a formal paper to read in advance of the presentation. From a practical standpoint, these differences are minimal, with all rankings averaging above 8.0. The ratings also reflect positively on the inclusion of outside experts as commentators during the journal article presentations, which the course faculty believe significantly increases the level of discussions and helps the students to evaluate the real contribution of the article to the field, both of which are essential skills necessary for scientific success.
Advisor Perspective
Four UCLA faculty members have served as research advisors to more than one HHURP scholar during the first 7 years of the program. These individuals were interviewed to ascertain their perceptions of the HHURP program in general and to determine specifically whether they were able to attribute any academic or personal growth observed in these students to their participation in the HHURP. The advisors emphasized that each of the scholars with whom they worked were already outstanding students before they were chosen as HHURP scholars, but the faculty were able to perceive some additional advantages that the program afforded the students.
The faculty identified two benefits to students participating in the HHURP. First, they saw a clear boost in the self-confidence of the students selected for the program. Second, they were aware that the students benefited from a substantial amount of assistance and guidance with their presentation skills. However, one professor mentioned that he addressed presentation skills in his weekly lab meetings, so, in his case, it was unclear whether his students' improvement in this area was attributable to HHURP participation or to their involvement in the lab meetings.
As a general confirmation that the journal club discussions were targeted at a high level, one of the professors, who had attended a number of the journal club presentations as the subject area expert, remarked, "I thought the programs were exceptional. They were very much like at least first- or second-year graduate students."
Their comments provide additional anecdotal evidence about the value of the program, although the small number of professors interviewed and the informal means of communication limit our ability to draw broad conclusions from these observations.
Alumni Perspective
Since the 19981999 academic year, 40 of UCLA's science majors graduated from the university as HHURP scholars. All of these alumni were contacted by e-mail during the 20042005 academic year and were asked to respond to a survey that gave them the opportunity to assess the long-term value of the program from the perspective of a graduate or medical student. The survey included both closed- and open-ended questions and focused particularly on the effect of participating in the weekly journal club. The survey was sent twice by e-mail, and responses were received from 33 of the alumni, for a response rate of 82.5%.
The alumni were asked to rate the impact of their journal club participation on 11 different areas using the following scale: 1 = negative effect; 2 = no effect; 3 = somewhat positive effect; 4 = positive effect; and 5 = very positive effect.
The choice of a five-point Likert scale is consistent with much of the education assessment literature (Sundberg, 2002). The scale was designed to be positively skewed, based on the cumulative responses to the annual surveys that had been conducted over the prior 6 years. Students' assessment of the program had averaged higher than 8.0 on the 9-point scale, and the researchers were concerned that a fully balanced scale (1 = very negative effect to 5 = very positive effect) would decrease the variation in responses because it would only allow students to choose from two, rather than three, positive choices. The inclusion of open-ended questions allowed the researchers to examine whether the responses to the scale questions were consistent with the narration provided for the open-ended questions.
The mean responses from the alumni are provided in Table 2.
|
The alumni were then asked to reflect on, "How, if at all, did your experience as a Howard Hughes Scholar and journal club participant assist you with your graduate/medical school applications and interviews?" Alumni responses fell into four main categories. First, alumni indicated that the journal club enhanced their ability to discuss their own research in an articulate, sophisticated manner, which was very helpful to them in the interview process. Second, the range of articles presented and discussed in the journal club broadened participants' scientific knowledge base beyond their own area of expertise, which allowed them to converse comfortably with scholars in a number of fields during their interviews. One alumnus said, "Having presented my research to the HHURP, I was very confident and skilled in discussing my research with both seasoned researchers and lay people. I was also able to quickly understand others' research and formulate thoughtful questions to ask them. These skills helped me in interviews with scientists, physicians, and medical students." Third, a number of the alumni mentioned that they received letters of recommendation from the program coordinators. Finally, the responses indicated that their association with the HHMI caught the attention of admission committee members. A number of responses included comments similar to the following: "It seems as though being a Howard Hughes Scholar greatly distinguishes me from the thousands of other applicants."
Another question asked respondents to relate whether their participation in the journal club provided them with any relevant skills they have been able to use in graduate studies. Every person enrolled in a graduate program responded affirmatively to this question. In fact, a number of them indicated that the format of the journal club was similar to their graduate courses, so it gave them an advantage over their classmates because they had already learned some of the skills required for participation in an interactive seminar. One person went so far as to say that in the journal club, "I was around people smarter than me and more ambitious than me, which inspired and pushed me. As a result, the first year of grad school was a step back." Some of the specific benefits resulting from participation in the journal club were the ability to read research articles quickly and efficiently while extracting the most important information from them, increased comfort in "conversing scientifically with colleagues," confidence in dissecting and criticizing a research paper, and the ability to organize and present scientific data to an audience. One of the alumni reported, "The analytical skills I learned in the program are still used in my education at the graduate level. However, in addition to specific skills, the most important thing I gained from HHURP is just an overall increased intellect and ability as a scholar."
The alumni were also asked a general question about the primary benefits they received from participating in the journal club. In addition to the skills they acquired, as described above, a number of them mentioned several affective outcomes of the program. First, they found a supportive community with a cohort of like-minded students, or as one alumnus mentioned, "a very bright group of people who had fun talking about science over pizza." Alumni indicated they enjoyed getting to know both the faculty and students involved in the journal club. Second, the journal club fostered a sense of excitement about science and research. One alumnus indicated that "it got me excited about research because it pushed me to think critically," whereas another stated, "it made me love basic science research." Third, participants gained confidence in their own abilities as consumers and presenters of research. "My participation gave me more confidence in presenting my research, thinking about research questions, and conversing with faculty about research." One additional benefit mentioned repeatedly was the exposure they received to scientific studies and ideas outside their major or main area of research. Alumni felt this broadened their knowledge base and allowed them to communicate with a wider array of scientists.
One additional question asked alumni for their suggestions for improving the organization or format of the journal club, from their perspective as a graduate student. Many of the responses indicated that the alumni would change nothing about the program, and a number of suggestions from the early cohorts of participants, such as utilizing electronic presentations, had already been addressed by the program coordinators. However, there were two areas in which the alumni offered constructive suggestions for the program. The first was, perhaps, an inevitable consequence of a program in which the academic level of the participants is so high. Some alumni felt that participants were afraid to ask clarification questions for fear of looking unprepared or ignorant. "I worried sometimes that students were intimidated by their peers and would therefore avoid asking questions when they were confused." Another person emphasized "the importance of fostering an environment where students feel comfortable sharing their views and/or asking questions." A number of the alumni did indicate they felt that the journal club provided a supportive environment for students, but that an extra effort should be made to encourage the active participation of shy students. The second suggestion was to consider doing away with assigning questions for students to respond to during the journal article presentations and instead allow students to come up with their own questions about the articles. "I would suggest a more open discussion that encourages everyone to participate on their own, and less of a direct questions approach." The common thread in these suggestions seems to be an interest in increasing student involvement.
The alumni overwhelmingly felt that their experience with the journal club provided them with many benefits, including analytical and presentation skills, knowledge outside their area of research, confidence in their own abilities as researchers, assistance in the application and interview process for graduate or medical school, and a preview of the graduate school experience.
| CONCLUSION |
|---|
|
|
|---|
Above and beyond the perceptions of the students, advisors, and alumni of the program, perhaps the strongest indicator of the success of the program is the record of achievement of its participants at UCLA and after graduation. In 2003, 2004, and 2005, a total of 321 undergraduates participated in UCLA Science Poster Day. HHURP scholars received 11 of the 28 Dean's Prizes awarded at those events. After graduating, 70% of the alumni moved on to Ph.D. (11 alumni), M.D. (13 alumni), or combined M.D./Ph.D. (five alumni) programs at top universities, including Massachusetts Institute of Technology, Harvard, UCLA, Cal Tech, Yale, Columbia, and the University of Chicago. Five of the students enrolled in medical scientist training programs and two more in other types of master's programs. The others are either working in biomedical research or pursing other postbaccalaureate studies. They partially credit their HHURP experience for these outcomes. As one alumnus said of the journal club, "it was one of the best educational experiences I had as an undergrad in terms of preparing me for a future in medicine and science."
We can conclude that an intensive literature-based training program increases student confidence and scientific literacy during their undergraduate years and facilitates their transition to postgraduate study. Although we used a team of three faculty to train 14 undergraduates per year, elements of our program, i.e., those that enhance student participation, should be applicable to undergraduate journal clubs led by individual faculty members. A key issue in the future will be determining how to extend this approach to the many undergraduate students who would benefit from primary literature-centered training at universities with large undergraduate enrollments.
| FOOTNOTES |
|---|
Present address: Research, Planning, and Assessment, Los Angeles Pierce College, 6201 Winnetka Avenue, Woodland Hills, CA 91371. Address correspondence to: Carol A. Kozeracki (kozeraca{at}piercecollege.edu)
| REFERENCES |
|---|
|
|
|---|
Creswell, J. W. (1998). Qualitative Inquiry and Research Design: Choosing among Five Traditions, Thousand Oaks, CA: SAGE Publications.
Glazer, F. S. (2000). Journal clubsa successful vehicle to science literacy. J. College Sci. Teach 24, 320324.
Isaac, S., and Michael, W. B. (1981). Handbook in Research and Evaluation, 3rd ed., San Diego, CA: EdITS Publishers.
Lopatto, D. (2004). Survey of undergraduate research experiences (SURE): first findings. Cell Biol. Educ 3, 270277.
Muench, S. B. (2000). Choosing primary literature in biology to achieve specific educational goals. J. College Sci. Teach 29, 255260.
Mulnix, A. B. (2003). Investigations of protein structure and function using the scientific literature: an assignment for an undergraduate cell physiology course. Cell Biol. Educ 2, 248255.
National Research Council (2003). BIO 2010, Transforming Undergraduate Education for Future Research Biologists, Washington, DC: National Academy of Sciences.
Smith, G. R. (2001). Guided literature explorations: introducing students to the primary literature. J. College Sci. Teach 30, 465469.
Sundberg, M. D. (2002). Addressing student learning. Cell Biol. Educ 1, 1115.
This article has been cited by other articles:
![]() |
K. M. Gehring and D. A. Eastman Information Fluency for Undergraduate Biology Majors: Applications of Inquiry-based Learning in a Developmental Biology Course CBE Life Sci Educ, March 1, 2008; 7(1): 54 - 63. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | ARCHIVE | SEARCH | TABLE OF CONTENTS |