Development, Implementation, and Assessment of a Lecture Course on Cancer for Undergraduates

INTRODUCTION

Cancer represents a substantial healthcare and economic burden in the United States. It is the second leading cause of death in the United States (American Cancer Society, 2008), with 1.4 million diagnoses and 559,650 Americans dying of cancer in 2007. The cancer healthcare costs in 2006 were $206.3 billion (National Cancer Institute, 2008). It is also a disease that is not shared equally by all races and ethnicities. African Americans have the highest cancer mortality rates, and a 5-yr relative cancer survival rate that is 10% lower (at 57%) than that for Caucasians (at 67%; 1996–2003 data presented by the American Cancer Society, 2008).

Despite great strides in many areas of medicine over the past 60 years, cancer mortality has not decreased to the same extent as mortality from other diseases during this period (American Cancer Society, 2008). Because the burden of cancer remains high and disproportionately affects minority and underserved populations, there are compelling arguments for increasing the diversity of the cancer research workforce. As reviewed by Handelsman et al. (2007), groups with diverse membership solve problems more creatively than more homogeneous groups, suggesting that scientific advancement will be enhanced by increasing the diversity of the research workforce. In addition, increasing the pool of minority researchers and providers is repeatedly identified as a strategy to reduce or eliminate cancer disparities. For example, as referenced in Johnson et al. (1998), Newman et al. (2003), and Smedley et al. (2004), minority providers are more likely to serve minority and underserved communities, and a diverse faculty and trainee pool increase the cultural competency of all trainees. At the undergraduate level, campus-level diversity increases learning by all students, and minority providers and faculty members serve as role models and mentors for underrepresented individuals seeking biomedical careers.

Despite the compelling arguments for maximizing the diversity of the cancer research workforce, the proportion of minority cancer researchers is not representative of that of the U.S. population. Using earned doctorates as an indication of the pool of research trainees in 2006, 15,549 U.S. citizens earned science and engineering doctorates. Of these, 9.6% were earned by American Indians or Alaska Natives, Blacks, and Hispanics (Falkenheim, 2007). This is in contrast to >25% of the U.S. population making up these groups (United States Census 2000, 2000). A lack of racial and ethnic diversity is also observed among faculty in biological science departments. Of the 2377 full professors in the top 50 biological sciences departments in the United States, only 0.75% are Black, 1.6% are Hispanic, and 0.1% are Native American (Nelson, 2007). In the field of cancer research, <2% of the American Association for Cancer Research membership is African American (Newman et al., 2003).

Several approaches have been proposed to increase minority representation in medicine and cancer research, many of which are targeted toward the education “pipeline” (Johnson-Thompson et al., 1996; Johnson et al., 1998; Newman et al., 2003). Numerous studies have reported medical school interventions to enhance cancer knowledge (Sanidas et al., 1993; Geller et al., 2002; Leinster, 2004; Gaffan et al., 2006) and the value of an internship or other intensive experience for master's students, medical students, pharmacy students, and undergraduate students (Legardeur et al., 1993; Jazieh et al., 2001; Pasick et al., 2003; Kelly et al., 2006; Yancey et al., 2006).

New Mexico State University (NMSU) is a minority-serving land-grant institution with the resources to provide low-cost, lecture-based courses. We therefore decided to determine whether we could attract a diverse population of students to the field of cancer research by offering an engaging lecture course on cancer. We consulted the published literature and found that there were few publications on the role of undergraduate lecture courses on cancer biology as a means to promote interest in careers in cancer research. Torabi and Seffrin (1989) reported on the assessment of an undergraduate course for nonscience majors with the overall goal of promoting knowledge about (and accompanying behaviors associated with) cancer prevention. Rushton (1999) describes an honors course entitled “Learning to Live with Cancer” designed for both nursing and nonnursing students. The course objectives are to increase cancer knowledge, work with cancer patients in the community, and prepare students to be informed consumers of cancer-related healthcare. Although both of these courses serve valuable roles, neither seems to address the recruitment of a diverse population into the field of cancer research, which is one of the goals of our work.

Given the small number of published reports on lecture-based courses on cancer (directed to future cancer researchers), we addressed the following questions.

1. Given the small number of publications about undergraduate cancer biology classes, is it feasible to successfully teach cancer to undergraduates at a land-grant institution? (i.e., are undergraduates able to learn cancer principles from a lecture course, without an accompanying laboratory, at an institution that does not have a predominant biomedical research focus?)

2. Can we develop an effective course-design template that has the potential to be used by others in the development and implementation of an undergraduate cancer course?

3. Can such a course stimulate interest in careers in cancer research among a diverse population of students?

MATERIALS AND METHODS

All protocols were reviewed and approved by the NMSU Institutional Review Board (IRB applications 6184 and 6592) and the Fred Hutchinson Cancer Research Center (FHCRC) IRB (IR 6617). The data presented here represent the spring 2007 semester offering of this course, the second time this course was offered at NMSU.

Course Design and Advisory Board

To ensure that our course goals and objectives (Table 1) were both reasonable and important, we relied on the expertise and input of a curriculum advisory committee, made up of five scientists and cancer information specialists from the FHCRC in Seattle, WA. We met twice with our curriculum advisory committee, before and after the course was taught for the first time. The input of the curriculum advisory committee with respect to content and assessment was used to refine the course for the spring 2007 semester presented here.

Course Structure

The course was a three-credit, upper-division lecture course with a total enrollment of 30 students. The prerequisites were listed as one of the following courses: Genetics (a 300-level course), Biochemistry (a 300-level course), or Cell Biology (an advanced 300-level course). It met twice a week (Tuesdays and Thursdays) for 75 min per lecture. The topics covered are listed in Table 2, and they fall into three focus areas of basic cancer biology, cancer epidemiology, and translational research. The course textbook (Pecorino, 2005) was chosen because it provided a good balance of these four focus areas (with particular strength on translational research), and it presented material at a level that could be understood by a student who had only taken genetics before taking this course. Four topics (breast cancer, diet and cancer, human papillomavirus [HPV] and cervical cancer and immunotherapy) were enhanced by video-teleconferenced (VTC) lectures given by FHCRC researchers. In these cases, the NMSU instructor gave an overview lecture preceding the FHCRC VTC lecture. In this way, the students could develop a basic foundation before learning about current research from experts in that area. Another two topics (cell cycle and angiogenesis) were presented as face-to-face guest lectures by an NMSU faculty member with research expertise in these areas. Students completed three individual out-of-class assignments and a group out-of-class poster assignment, and took three in-class exams and a final exam. The exams were primarily short-answer questions (see Supplemental Material for pretest questions that became embedded exam questions for sample questions).

Content and Skills Assessment

After developing our learning goals and objectives, we designed a pre- and posttest. The initial pretest had 15 questions (worth a total of 66 possible points) and was administered during the second week of class. Students could elect to voluntarily participate in the assessment (and indicate their willingness by signing an informed consent). There were no course points associated with the pretest. Fourteen of the pretest questions became exam questions on various exams during the course (i.e., we used an embedded posttest design). The last pretest question was not used as an embedded posttest item because the content did not end up being explicitly covered during the semester.

The pre- and posttest included questions addressing cellular behaviors (and molecules) associated with invasion and metastasis; targeted cancer therapies; mutations, cancer, and aging; behaviors that influence cancer risk; carcinogens and their mechanism of action; and proto-oncogene activation and clinical trials. We also addressed applied knowledge, including the ability to evaluate media reports about cancer and the role of peer-reviewed publications in science; the ability to evaluate data (e.g., in one question, a description of a New England Journal of Medicine publication and two key figures from the article were presented to students for their interpretation; The Breast International Group (BIG) 1-98 Collaborative Group, 2005). On the recommendation of our curriculum committee, we excerpted six multiple-choice format questions from a Graduate Record Examination (GRE) preparation book (Levin, 2004) to assess general cell and molecular biology knowledge based on questions written by someone other than the course instructors. Most of these questions dealt with general concepts (e.g., enzyme inhibitors, mitotic chromosome segregation, and cell cycle progression), but the questions were phrased in the specific context of cancer. Two of the questions were multiple-part questions, based on data (graphs) interpretation questions, and the specific context was again related to cancer. Although these questions were administered on the pretest, they proved challenging to embed on in-class exams, because they were not specifically related to the course material. Thus, although they were included as “bonus” questions on different in-class exams, we have elected not to analyze them here, because we are not sure that students were equally motivated to answer them, and we are not sure that they were effectively delivered in the embedded posttest format. Thus, eight questions, many with multiple parts, totaling 54 points were included in this analysis.

At the end of the semester, pre- and posttests were paired for each student who consented to participate in the research analysis (n = 27; 90% of the enrolled students) and scored according to a consistent set of criteria. The pre- and posttest scores for each question were recorded for each student. Normalized learning gains were calculated for each student on the entire pre- and posttest and on each individual question. The normalized learning gain is the amount of learning that did occur, expressed as a proportion of the total learning that could have occurred, based on the baseline knowledge (as assessed by the pretest). In brief, the normalized learning gain is the difference between the posttest score and the pretest score, divided by the difference between the maximum possible score and the pretest score ([post − pre]/[max − pre]). Note that it is possible for a student to have no learning gain (=0) or even a negative learning gain on a particular question. In such cases (questions on which one or more students had zero or negative learning gains), the zero and negative gains were included in the calculation of the average gain for that question. The only instance in which an individual gain was not included in the calculation of an average learning gain was when a student had a perfect score on the pretest. In these cases, the calculation of a normalized learning gain is not possible, because it requires division by zero (the denominator [max − pre] is zero). These students were left out of the calculation of the average gain for the one question for which the learning gain was not calculable for one or more students.

Attitude Assessment

To assess students' attitude toward specific aspects of the course, and to assess their interest in cancer research careers, we included specific questions on the end-of-semester student evaluations. The questions used a 5-point Likert scale (e.g., strongly agree, agree, neutral/no opinion, disagree, and strongly disagree) and included space for students to write additional comments. Three of these questions addressed use of primary literature; five questions addressed knowledge of and interest in cancer research, including intention to pursue future opportunities in this field; four questions addressed the specific areas of cancer research covered in the course and experimental methods used in cancer research; and one questions was an open-ended question about the students' impression of the best aspect of the course. Twenty-nine students completed the anonymous evaluations (96.7% of enrolled students) and responses for each question were recorded.

Student Demographics

Thirty students were enrolled in the class in the spring 2007 semester; 73.3% were female and 26.7% were male. Overall, 46.7% of the students were Hispanic, 36.7% were Caucasian, and 3.3% were American Indian or Alaskan Native. The remaining four students were characterized as “Other.” Twenty-nine (96.7%) of the students were seniors. The major with the greatest representation in the class was biology (66.7% of the students) followed by biochemistry (10%). The majority of the students had either already taken two of the three possible prerequisites for the course, or they had taken one prerequisite and were concurrently enrolled in another.

RESULTS

We had three major goals in designing this course. First, we wanted to know whether undergraduate students could develop an improved understanding of cancer biology through a lecture course. Second, we wanted to develop a template so that others could implement and assess an undergraduate cancer course at their institutions. And third, we wanted to know whether participation in such a course could enhance interest in cancer research careers in a diverse population of undergraduate students.

Course Design and Advisory Board

To provide a “complete” cancer experience, we designed our course around the three areas of expertise of comprehensive cancer centers: basic cancer biology, cancer epidemiology, and clinical cancer research. Specific learning objectives related to these content areas included determining students' ability to describe cancer progression at the cellular level, identifying risk factors for cancer, describing the basis for differences in cancer incidence, and using their understanding of the molecular and cellular biology of cancer to explain the basis for various cancer treatments (Table 1). We also sought to develop and reinforce applied knowledge, including the ability to search for and find relevant articles in the primary literature, analyze primary research, and communicate about cancer on both a professional and a general level (Table 1). Finally, we consciously addressed cancer disparities and discussed several studies in which cancer disparities were explored. This was intended to both address a compelling research need and make the course inclusive and relevant for a diverse student population.

The development and initial implementation of this course were funded through a pilot project in a National Cancer Institute (NCI)-funded Minority Institution/Cancer Center Partnership Program (NMSU/FHCRC): Building Minority Cancer Research Capacity (5U56CA096286 [NMSU] and 5U56CA096288 [FHCRC]). Because this project was a partnership between the NMSU instructor (M.S.) and a staff scientist at the FHCRC (K.P.), we had the opportunity to ask FHCRC scientists and cancer information specialists from the NCI-funded Cancer Information Service to serve as members of a curriculum advisory committee. This is a unique opportunity in terms of course design, especially for a course at NMSU, which does not have faculty with extensive expertise in all areas of cancer research. The advisory committee was composed of three scientists and two cancer information specialists from the FHCRC, and they provided valuable input and feedback regarding our course design. The cancer information specialists had specific expertise on disseminating cancer information to minority populations. After drafting course goals and objectives and a pretest, we met with our curriculum advisory committee (November 2005). Based on this meeting, we revised our materials before the first offering of the course (in the spring 2006 semester). We met again with our curriculum advisory committee in August 2006 to review the first offering of the course and discuss revisions for the second semester (spring 2007, presented here).

Course Format

The course met for one semester, twice a week for 75 min each meeting. The primary format was lecture, with occasional in-class activities designed to engage students in small-group discussion and problem solving. The NMSU course instructor (M.S.) gave most of the lectures. Two lectures were given by an NMSU faculty member with expertise in the topic areas (cell cycle and its regulation; angiogenesis, and metastasis), and four lectures were delivered by scientists at the FHCRC via VTC. The VTC lectures focused on current research in specific areas (HPV, cervical cancer, and the HPV vaccine; breast cancer epidemiology; diet and cancer; immunotherapy for B-cell lymphomas). To ensure that the students were prepared for the VTC lectures, the NMSU course instructor gave an “overview” lecture during the class meeting before each VTC lecture. By introducing and reinforcing content in the context of current research, the lectures were key to meeting the content knowledge goals and to addressing the goal of stimulating an interest in careers in cancer research.

There were also out-of-class assignments that were designed to meet the goals of searching for and reading primary literature, evaluating information, and communicating about cancer on a personal and professional level. Three out-of-class assignments were individual assignments. The first two of these assignments required reading at least one primary article and answering specific questions about it. The second assignment had an additional component of finding and using additional resources to answer all of the questions. The third assignment addressed determining general Internet information literacy, assessing the scientific literature supporting different cancer treatments, and writing an “expert opinion” for a judge considering a case of a teenager wanting to discontinue conventional therapy and pursue an alternative treatment for Hodgkin's lymphoma (this was based on a real case that students read about on an Internet news service). The students needed to write their expert opinion using terms, language, and explanations that were accurate but not technical, so that a smart layperson could understand their arguments.

The final out-of-class project that students worked on was a group poster project. Small groups of students (between two and five students per group) chose topics to present in a poster format, and carried out independent literature research to develop their presentation. Although students had a great deal of freedom to choose their topics, they all had to include some aspect of current research in their presentations. Students had to prepare “installments” of their poster throughout the semester, including their literature search strategy, a bibliography, and an outline, before preparing their final poster. The last two class periods of the semester were devoted to poster sessions, with each group presenting their poster to the instructor; other students in the class; and visitors, including other NMSU students and faculty members. The poster assignment reinforced literature (searching, reading, and interpreting) and communication skills.

Content and Skills Learning Gains

Twenty-seven of 30 registered students (90%) voluntarily participated in and completed both the pretest and the embedded posttest associated with the research project. When considering the complete pre- and posttest (with the GRE questions excluded, as described in Materials and Methods), the average score on the pretest was 30.2% and the average score on the posttest was 65.6%. This represents a significant improvement, based on a one-tailed, paired t test (p < 0.001). Similarly, paired t tests for each individual question showed a significant improvement between the pre- and posttest. The average normalized learning gain for the entire pre- and posttest was 0.51. When looking at individual questions (or related parts of or groups of related questions), average normalized gains ranged from 0.11 to 0.83. Based on these results, it seems that undergraduate students can indeed learn cancer-related content in a lecture course.

Although we cannot immediately identify a clear pattern with respect to questions with higher versus lower normalized learning gains, we have noted some interesting trends in student responses to several questions. One question asked students to agree or disagree with the statement that cancer is predominantly a disease of the elderly, and to explain their answer. The “correct” answer was an agreement, given that cancer progression generally requires several independent mutations in the same cell, which is more likely to occur over a longer period. On the pretest, the average score was 1.06 out of 3 points (35.3%), and on the posttest, the average score was 2.48 out of 3 (82.7%), with a shift to more students answering correctly on the posttest (Figure 1A). The average normalized learning gain between pre- and posttest was 0.74. When we characterized responses on the pre- and the posttest, we were able to ascertain four categories of responses: those that agreed with the statement and discussed the acquisition of multiple mutations; those that agreed with the statement but had other or not fully developed explanations (e.g., cancer “takes time” or referencing increased exposure to carcinogens, without specifically referencing multiple independent mutations); those that disagreed with the statement because “everyone” can develop cancer; and those that disagreed based on personal experience with a young person with cancer (testimonial). On the pretest, 44.4% of the students agreed that cancer is predominantly a disease of the elderly, with the majority of explanations not precisely recognizing the concept of multiple independent mutations. Among the 55.6% of students who disagreed with the statement on the pretest, the majority cited opinions that everyone can develop cancer (Figure 1B). In contrast on the posttest, 92.6% of the students agreed with the statement that cancer is predominantly a disease of the elderly, and 68% of the students agreeing with the statement referenced the concept of multiple independent mutations (Figure 1B). There were no “testimonials” in the posttest responses, indicating that students may have begun to value empirical evidence (e.g., 1996–2003 data presented by the American Cancer Society, 2008) over personal experience during the course.

Another particularly interesting question asked students to identify three behaviors or choices that influenced cancer risk, state whether the behavior elevated or reduced cancer risk, and identify the type of cancer that the behavior had the greatest impact on. On the pretest, there were 76 “responses” from the 27 students (each of the 27 students could identify up to three behaviors). Perhaps not surprisingly, the average pretest score was quite high (6.47 out of 9; 72%), because many students have basic ideas about cancer risk (e.g., smoking increases the risk of lung cancer). The average posttest score was 8.66 out of 9 (96.2%), a significant improvement over the pretest (p < 0.001) by a paired t test. We then characterized the responses on the pre- and the posttest, to see whether there was a shift in the examples or depth of the responses (Figure 2). On the pretest, the common responses were smoking and lung cancer (32.9%), tanning/UV exposure and skin cancer (19.7%), “diet” (in vague terms, e.g., “eating healthy” and “cancer”) (15.8%), cancer screenings (13.2%), hormonal/reproductive factors and breast cancer (5.2%), and other (e.g., unprotected sex, abortion, smokeless tobacco/chew, and oral cancer; Figure 2). On the posttest, there were 82 responses (one student gave four responses), with some shift in the frequency of responses (Figure 2). In particular, there seemed to be more varied answers on the posttest, indicating that the students broadened their knowledge about cancer risk factors. For example, far more responses on the posttest addressed diet and cancer and hormonal/reproductive issues and breast cancer, compared with the pretest. This probably reflects the amount of lecture time devoted to these topics (two diet and cancer lectures, including an FHCRC VTC lecture; and a VTC breast cancer epidemiology lecture, as well as targeted therapies and screening lectures that used breast cancer as an example; Figure 2).

We then examined the responses with respect to specific behaviors and cancer risks more closely. For example, on the pretest, the majority of the responses referring to smoking and lung cancer risk were somewhat vague, with no mechanism provided. However on the posttest, 30.4% of the responses referencing smoking and lung cancer were more “specific,” referring to specific carcinogens in tobacco, types of DNA damage occurring after exposure to smoke, or the role of P450 enzymes in processing carcinogens in smoke. Similarly, responses about skin cancer risk tended to be more vague on the pretest and more specific (with respect to, e.g., pyrimidine dimers) on the posttest. The most dramatic increase in specificity of explanations occurred for diet and cancer; 100% of the pretest responses about diet and cancer were vague (e.g., “eating healthy”), whereas the majority (70.8%) of the postresponses were very specific (e.g., specific food groups such as cruciferous vegetables; charred meats and carcinogens; fluid intake and bladder cancer). These were all examples presented during the lectures, indicating that students were able to use in-class examples to develop a more specific understanding of cancer risk factors.

In terms of other content knowledge-based questions (e.g., genes/behaviors involved in metastasis and angiogenesis, identifying and explaining the mechanism of targeted therapies, mechanism of action of a carcinogen, mechanisms of activation of proto-oncogenes, understanding of clinical trial process), there was significant improvement between pre- and posttests, indicating that the lectures and supporting activities were sufficient to increase cancer content knowledge.

In terms of knowledge application skills, one of the pretest questions addressed an understanding of clinical trials and then presented data (in the form of figures from The Breast International Group (BIG) 1-98 Collaborative Group, 2005) for students to interpret. The question had three parts, addressing both clinical trial design and data interpretation. This study was chosen because it addressed breast cancer (which was a common example during the semester), and it presented data that was graphically “subtle,” in that the two adjuvant treatments (tamoxifen and letrozole) did not seem to be significantly different (based on simple visual inspection of the graphs). However, although a superficial inspection of the graphs may lead a student to conclude that the two treatments did not differ significantly, the p values (provided in each figure) indicate that there was a significant difference between the two treatments. Although there was significant improvement between the pre- (33%) and posttest (63%) scores for the data-interpretation part of this question (p < 0.001), the average normalized learning gain was only 0.39. When examining students' posttest explanations for their interpretation of the graphs, many (48.2%) were either relying on visual inspection of the graphs to make their decision (e.g., “no difference, because lines are very similar”) or on the numerical counts provided below each graph (e.g., “only a small % difference”), and they were not referencing the p values. So, although students were better able to answer this question in its totality, there still seemed to be an incomplete approach to data interpretation that can be addressed in future offerings of the course.

We wrote another question to address scientific literacy in terms of evaluating media reports and understanding the value of peer-reviewed publications (skills reinforced both in lecture and in the third take-home assignment). In this question, we used a timely news report (published online by Health Day/Scout News on December 14, 2006, and linked to Medline Plus) describing a presentation at the San Antonio Breast Cancer Symposium in December 2006, reporting a drop in breast cancer rates after the reduction in the use of hormone-replacement therapy that occurred in 2003 after the 2002 publication of the Women's Health Initiative (Writing Group for the Women's Health Initiative Investigators, 2002). One part of the question asked students to evaluate the media report for qualities that would increase or decrease their confidence in the findings reported; and one part of the question asked them to explain why at the time of this news release, there were no corresponding peer-reviewed publications in the PubMed database. Although students seemed to have a generally good approach to evaluating media reports, a surprising number of students seemed to be unaware of the relationship between the process of scientific discovery and peer-reviewed publications. There was a strong emphasis in lecture on peer-reviewed publications (in contrast to testimonials and anecdotes), and this approach may have failed to acknowledge the developmental nature of scientific discovery, in which findings are presented at scientific conferences before being submitted for peer-reviewed publication. When asked why a recent news article was not supported by peer-reviewed publications in PubMed, only 7.4% of the students acknowledged that the conference report was “too new” to be published in the peer-reviewed literature. Also, 44.5% of the students stated that it must be because the study was not peer-reviewed, was not “real research” or real experimentation, or had only inconclusive results. The remaining students proposed explanations including that news reports are not indexed by PubMed (which is true but fails to recognize the progressive nature of science and the reporting of preliminary results) and that there have been no studies proving the link between hormone-replacement therapy and breast cancer (apparently failing to acknowledge the preponderance of evidence and the nature of epidemiological studies). Again, this suggests an area for clarification and improvement in future offerings of the course.

Attitude Assessment

We used end-of-semester anonymous student evaluations to assess student attitude toward particular aspects of the course. Twenty-nine students (96.7% of enrolled students) completed the student evaluations. In addition to the standard institutional, college, and departmental questions, we analyzed 14 additional questions that addressed students' perceptions of their cancer knowledge, their use of primary literature, the VTC lectures from the FHCRC, and their interest in cancer and cancer research. These questions were all answered using a Likert-type scale, with space left for students to write in additional comments. We also tallied the responses to one open-ended question asking students to identify the “best aspect” of the course.

Because we had collected data to show that students did have a substantial increase in their cancer content knowledge, we were interested to know whether students perceived that they had learned cancer content. We asked a series of questions related to their overall perception of learning, as well as their learning of the specific content areas (e.g., basic cancer biology, cancer epidemiology, and clinical cancer research) and of experimental methods used in cancer research. These questions were all answered using a 5-point Likert scale (strongly agree, agree, neutral/no opinion, disagree, and strongly disagree) by all 29 students who completed the student evaluations. As shown in Table 3, all students had positive responses to all of these questions (100% strongly agreed or agreed that their understanding had improved).

One of our learning goals was for students to become familiar with the primary literature, and this goal was enforced both through lectures (in which data from the primary literature was presented) and through out-of-class assignments. Three items on the student evaluations addressed whether the primary literature helped students better understand a topic, whether students felt more confident in interpreting data presented in the literature, and whether students felt that the primary literature skills that they developed would be useful in other classes. As shown in Table 4, the majority of students strongly agreed or agreed with these items. There was a small minority of students (14%) who had no opinion or were neutral about the primary literature helping their understanding, and a minority (10%) who were similarly neutral about an increased confidence in interpreting data presented in the primary literature. The most positive response was for the development of experience and skills that could be used in other classes, suggesting that students did value the experience.

On a related note, the VTC lectures from FHCRC scientists were focused on current research in the field, with a strong emphasis on experiments and data. Because we were not sure whether students would find these lectures to be accessible, despite the “prelecture” overview by the NMSU instructor, we wanted to survey the students about the VTC lectures. The response was overwhelmingly favorable (Table 5): 100% of the students strongly agreed or agreed (most of them strongly agreeing) that the VTC lectures were interesting and that the VTC lectures helped them learn more about a topic (Table 5).

Another outcome that we were hoping to achieve was to promote an interest in careers in cancer research, to ultimately increase minority participation in cancer research. Although this was not an explicitly stated learning outcome, we hoped that the emphasis on research, the VTC lectures, and the consistent identification of current problems in cancer research during lectures would promote an interest in cancer research careers. To assess the end-of-course interest in such careers, we embedded four questions on the end-of-semester student evaluations. The first question, “Compared with when I started this class, my interest in cancer research is …,” had a 5-point Likert scale, ranging from much higher to much lower. Many (76%) of the students responded that their interest was much higher. Another 21% stated that their interest was somewhat higher, and the remaining students stated that their interest was about the same (Table 6). On a question dealing with considering applying to graduate school in a field related to cancer research, 7% stated that they already had applied in such an area; 59% stated that they would consider it; 24% responded that they would not consider it, because they were going to professional school (rather than graduate school); and 10% stated that they were not that interested in cancer research. When asked about having had the opportunity to carry out research in a field related to cancer research, 3% responded that they had such an opportunity at NMSU, 21% had had a summer internship experience, and 76% had not had an opportunity to carry out cancer research. Finally, when asked about their plans to seek out a research opportunity in a field related to cancer research, 52% responded “yes” (that they were planning to seek out such an opportunity); 7% responded that they were planning to seek out a research opportunity, but maybe in another field; and 41% indicated that they were not that interested in research (Table 6).

In terms of free responses to the question asking about the best aspect of the course, 51.4% of the 35 responses from 27 students referenced the course content (Table 7), suggesting that students were indeed very interested in the field of cancer. The VTC lectures from the FHCRC were noted by 14.6% of the responses to be among the best aspects of the course, again suggesting a strong interest in cancer and cancer research.

DISCUSSION

Our goals were to use a lecture-based course to enhance cancer knowledge and skills in a diverse student population to promote a sustaining interest in cancer research careers and to develop a transferable template for designing related courses at other institutions.

In the second offering of this course, our data indicate that we were successful in reaching at least one of our goals: having a diverse population of undergraduate students learn about three areas of cancer research in a lecture-based format. Based on a pretest and embedded posttest instrument to calculate normalized learning gains on cancer content and related analytical and communication skills, the average normalized learning gain for the entire test was 0.51, and there was a significant improvement between the average pre- and posttest scores (p < 0.001). Average normalized learning gains ranged from 0.11 to 0.83 for different questions on the test, indicating that some topics will probably benefit from revised instructional approaches in the future. A normalized learning gain of 0.51 falls between the learning gains reported by Knight and Wood (2005) for an upper- division developmental biology course (a similar student population to ours) initially taught using a traditional approach (learning gain of 0.46) and then with an interactive approach (0.62). The enhancement of learning gains with adoption of interactive approaches has consistently been reported in biology (e.g., McDaniel et al., 2007) and in physics (e.g., Hake, 1998). We have established that cancer content is accessible to undergraduate students in a lecture-based format, and we can work on enhancing learning gains by adopting increasingly interactive and student-centered pedagogy in a scientific teaching approach (Handelsman et al., 2007). Furthermore, as evidenced by our course demographics, we met our goal of reaching a diverse student population in this course.

Our second major goal was to develop a template for the development of an undergraduate cancer course that could be transferred to other institutions seeking to develop similar courses. Several elements of our course design template contributed to our success. First, we set course goals and objectives early in the process, to be sure that these informed other decisions about the course design. This is consistent with the process of backward course design (e.g., Fink, 2003). Once we set our goals and objectives, we planned course topics that would support these goals, a textbook (Pecorino, 2005) that would allow us to address our content and skills goals, and we designed a first draft of a pretest to assess student learning in the course. We were very careful to continuously refer to our goals and objectives as we made design decisions for the course. At this point, we solicited feedback on these materials (goals, objectives, and pretest) from experts in the field of cancer research and community education, in the form of a curriculum advisory committee. Our committee membership was drawn from the FHCRC and was facilitated by an NCI-funded grant supporting a partnership between NMSU and the FHCRC. This was particularly helpful, because NMSU is a land-grant institution at which cancer research is only beginning to develop as an area of expertise. However, we feel that many instructors will have colleagues either at their home institutions or elsewhere whom they could approach for feedback on course materials during the course design process. By having both research and education expertise on the committee, we were able to address both content and pedagogy during our advisory meetings, and we feel that the feedback from the curriculum advisory committee was essential, especially given that this was a new course. Thus, we would recommend that curriculum advisory committees include both research (content) experts and master teachers, and they should be involved early in the planning process to maximize benefits. Note that these “steps” of backward course design and seeking early feedback are consistent with a scientific teaching approach (Handelsman et al., 2007), as is using assessment to improve teaching and learning.

To “close the loop” of course design, and to continue to adapt and improve our course, we assessed student learning gains with respect to our course goals and objectives. We are using the results of this assessment to make course changes, which we hope will lead to enhanced student learning, as measured by embedded assessment instruments. This will be an ongoing process, and we expect that we will begin to focus more specifically on particular learning goals and approaches, having answered our initial “big question” of whether this material is accessible to undergraduate students. For example, we want to focus on the issue of data interpretation and quantitative skills, because these are areas that students did not seem to master as thoroughly as other areas (average normalized learning gain of 0.39). Similarly, we want to work more on helping students understand the process of scientific communication before the final product of peer-reviewed publications, another area with which students seemed to struggle. Finally, with the increasing importance of genomics and bioinformatics in diagnostics, drug discovery, and genomics-based therapies, we want to incorporate these areas more explicitly into the course, perhaps through an interactive case study, which could also meet our goals of enhancing interactivity and improving quantitative skills.

Our final, and arguably most important goal, was to promote an interest in cancer research careers among our diverse population of students. As noted, 97% of students expressed a higher or much higher interest in cancer research relative to their interest at the start of the course. Despite this higher interest in cancer research, “only” 66% of students stated that they already had or would consider applying to graduate school in a field related to cancer research. Only 24% had a previous cancer research experience, which we are seeking to remedy through active promotion of various research opportunities during our course. However, despite 66% of students declaring an interest in continuing studies in cancer research, only 59% of students stated that they were planning to search out research opportunities (52% in cancer and 7% in other fields). This presents a slight discrepancy—more students expressed an interest in applying to graduate school (66%) than seeking out a research opportunity in cancer research (52%). This discrepancy between interest and “action” begs the question of how students perceive future academic and professional opportunities. Do they feel that graduate school does not require research experience (which would explain the higher interest in graduate school than research experience)? Do they not recognize that medical school (as well as graduate school) presents oncology research opportunities? And perhaps most importantly, how does an expressed interest in cancer research at the end of a lecture course as a senior undergraduate student translate into actual academic and career plans? Legardeur et al. (1993) recognized this fundamental issue in their assessment of a summer research experience for undergraduates (between their first and second years), students who had been accepted to medical school for the following fall, and medical students. Although the majority of students found that the experience went beyond their expectations (79%) and would be willing to repeat the program in another summer (83%), another majority had not committed to pursuing careers in cancer research or medical oncology. In fact, 84% of student participants were undecided with respect to careers in cancer research, and 62% were undecided with respect to careers in medical oncology (Legardeur et al., 1993). As noted by Legardeur et al. (1993), this raises the question of when and how to make the most effective interventions to promote entry into cancer research from the basic or medical research perspective. Although we do not have the answers to these questions, we have instituted a longitudinal tracking component to our course assessment to follow-up with course students for 5 years after participation in the course. We hope that this survey will provide us with at least preliminary insight into how to maximize the future diversity of cancer researchers based on a lecture class serving a diverse student body at a land-grant institution.

REFERENCES