Linking Assessment Questions to a Research Article to Stimulate Self-directed Learning and Develop High-order Cognitive Skills in an Undergraduate Module of Molecular Genetics

INTRODUCTION

Assessment influences every level of the education system and comprises one of the crucial catalysts for pedagogy reform (Sundberg, 2002; Linn et al., 2006). Assessment provides feedback, ranks students, defines and protects academic standards, and directs students’ learning. Essentially, assessment must function to promote self-directed learning and develop high-order cognitive skills (Linn et al., 2006; van de Watering and van der Rijt 2006; Crowe et al., 2008).

Assessment takes a wide range of forms, and each has its unique merits and limitations (Harris et al., 2007). Considerable efforts have been made in designing examination questions for assessments (Baartman et al., 2006; van de Watering and van der Rijt, 2006; Brinke et al., 2008; Crowe et al., 2008). It is commonly accepted that a poorly devised question may push students to merely copy relevant information in an open-book examination or to mindlessly memorize and regurgitate isolated facts in a closed-book examination. However, many assessment questions still tend to emphasize factual recall instead of a deep conceptual understanding, and this inveterate problem continues to exist. The persistence of this problem could be due to many reasons, one of which may be the lack of efforts to dig out new resources for setting suitable questions, enabling the assessment methods to fit the needs and demands of today's information and knowledge societies (Birenbaum et al., 2006; van Gennip et al., 2009).

Because the structure of DNA was first described in 1953, Molecular Genetics has been one of the rapidly growing areas in the life sciences. Its growth has been greatly expedited by the recent technological advances in genome sequencing. Faced with the increasing number of discoveries, an educator is actually engaged in the exciting but challenging mission of introducing students to this field. Despite these circumstances, however, many students in the Molecular Genetics module (LSM1102) in National University of Singapore (NUS) do not feel motivated to come to class. Conversations with them revealed two common responses. One was that students thought that the module content seemed unattractive because many terms such as DNA structure, DNA replication, gene expression, cell division, and Mendelian genetics had been taught in high school or junior college. The other was that students resented the module because they were required to take it and had to memorize a large set of facts for examinations. The remark “heavy in content and a lot of memorization” reflected a typical impression among the students. Sparking the students’ interest in this ever-growing scientific knowledge and reducing rote memorization are necessary for teaching this module successfully.

The ancient proverb “curiosity is the best teacher” led this study to explore a resource that can help improve teaching by tapping students’ curiosity. Textbooks typically present well-established studies, but they seldom cover unanswered or controversial issues. In contrast, research articles come alive with many unanswered and controversial issues, which are keys to stimulating students’ curiosity and motivating them to learn. Incorporating research articles into undergraduate and even high school education in bioscience has helped achieve desired learning outcomes (Pall, 2000; Russell et al., 2004; Sears and Wood, 2005). The efforts to bring research articles into science teaching have been focused mainly on two categories: to develop students’ understanding of the role and process of research in their discipline, thus enhancing their motivation to undertake research (Kozeracki et al., 2006; Sally et al., 2007); and to promote active learning and scientific thinking by introducing criticism and controversy (Yarden et al., 2001; Brill and Yarden, 2003; Eileen, 2006; Gillen, 2006). Pedagogical approaches to incorporating research articles in teaching include journal clubs, tutorials, short training courses, and assignments.

The present research aimed to develop a novel continuous assessment (CA), in which test questions were linked to research articles. Integration of a research article into CA can bring concepts to life, cultivate curiosity, and nudge students to become antotelic or self-directed through the creation of ever-new application questions without concern about repetition. This new assessment mode goes beyond traditional assignment of scores and brings students into a new learning landscape. For comparison, research articles and linked questions were also administered to students in lectures and tutorials instead of incorporation with a CA in different semesters. Our results showed that only the novel CA model could effectively motivate students toward self-directed learning.

MODULE DESCRIPTION

This study involved the LSM1102 module designed for first-year undergraduate students majoring in life sciences at NUS. Approximately 200–300 students took this module in each semester from 2006 to 2009. The module is taught over 68 h and consists of a lecture (24 h), tutorial (24 h), and laboratory (32 h) within one semester. It has three parts. The first part covers DNA structure, replication, transcription, translation, and regulation of gene expression. The second part focuses on cell division, chromosome organization, gene transfer, and gene recombination. The last part deals with Mendelian and population genetics. Each part, which is taught by a different lecturer taking approximately one-third of workload (23 h), has a CA component contributing 10% to a student's final module score. In total, there are three CAs taking 30% of students’ final scores in this module. The new assessment that involved this pedagogical study was only applied for one of the three CAs, which examined the second part taught by the author.

SELECTING RESEARCH ARTICLES

To find a suitable paper from leading peer-reviewed journals was a major challenge for developing the new CA. To accomplish this task, the RICH criteria were followed. 1) Relevant: The paper must link concepts learned in classes to applications. 2) Interesting: It must have components to enhance students’ curiosity in unanswered and controversial biological issues. 3) Comprehensible: It is not too complicated as first-year students are often not well-trained in reading scientific papers. 4) Heart-stirring: It must be impressive and inspiring, allowing it to be used as a successful example to set a high expectation for the students. News in life science often helps get clues for finding such papers. A list of representative papers is presented in Supplemental Material 1.

Based on the LSM1102 module synopsis, foundation areas of chromosome structure and gene transfer were chosen for this pedagogical study. Eukaryotic chromosome consists of several structural/functional regions, the most prominent of which are the centromere and telomere. The centromere plays a major role in chromosomal segregation during cell division. Abnormalities in centromere function lead to an unequal portioning of the genome that can have a profound effect on cell viability and behavior. The telomere, representing the end of the chromosome, consists of simple tandem-repeated DNA sequences. Changes in telomere structure may affect chromosome stability, cell aging, senescence, and perhaps malignancy. Prokaryotic chromosomal structure is relatively simpler than that of eukaryotes without centromere and telomere. In contrast, gene transfer, including lateral gene transfer (LGT) and vertical gene transfer (VGT), are also important concepts with both theoretical and practical potential merits. Among bacteria, LGT is well known in the evolution of antibiotic resistance, pathogenicity, and metabolic pathways. However, our textbook (Brooker, 2009) does not discuss LGT between prokaryotes and eukaryotes, and LGT between two eukaryotes, which raise many concerns, such as genetically modified organisms (GMOs) and emerging viral diseases.

INCORPORATING RESEARCH ARTICLES INTO TEACHING

The selected articles were introduced to the students by using three different approaches, namely, briefing the students on the article during lecture; incorporating the article in a CA; and discussing it in a tutorial, to find which way motivates our students’ learning effectively. The selected article was kept accessible on a website (IVLE; https://ivle.nus.edu.sg/lms/default.aspx) in due course (Table 1). In semester I of academic year 2007–2008 (SemI 0708), the selected article was briefly introduced to students in class immediately after relevant contents had been conveyed in the lecture. The students were then asked to read the paper after the class and be ready to answer linked questions in the next lecture.

In SemII 0708 and SemI 0809, selected articles were incorporated in the CAs. Students were told that the CA questions would be linked to the articles. The articles were released to the students 1 wk before the CAs; however, the CA questions were kept confidential. The CAs were conducted within 30 min after the last lecture with a 20-min break in between. During the CAs, students were allowed to use printed resources, including the textbook, notes, and the selected research articles, to answer the questions individually. Communication with others or accessing the Internet was not allowed.

In SemII 0809, a selected article was distributed to students 1 wk before a tutorial. The students were informed that the article would be discussed in the tutorial. The tutorial lasted for 1 h. In the first half hour, students were asked to answer linked questions to assess their understanding of the article and complete a survey form (see Supplemental Material 2) to collect their opinions on the way the research article was introduced. In the second half hour, the key points of the article were discussed, and the students were given opportunities to reflect their understanding and express their personal opinions. During the tutorial, students could use all available resources and communicate with others.

In all three approaches, the RICH criteria were followed to select articles, and the students had free access to the selected articles before and while they were asked to answer the linked questions. All the linked questions were set based on Bloom's taxonomy, which is described in the following paragraphs. A critical difference between the CA approach and the other two approaches was that a student's answer to the linked question was scored and contributed 10% to his or her final module score in the CA approach, whereas a student's answers made in lecture and tutorial did not contribute to the module score.

SETTING QUESTIONS

Questions were set at three levels, namely, conceptual understanding, applications, and synthesis, which required different cognitive skills to answer. The question at level 1 was straightforward, testing students’ scientific literacy and conceptual understanding. The question at level 2 was focused on the students’ ability to link their prior knowledge or textbook information to a current developing area, testing their ability to correlate different components with one another and understand scientific reasoning. The question at level 3 was linked to daily life. Students had to integrate their existing knowledge and apply the information in a creative way. These questions followed Bloom's taxonomy, which is a widely accepted tool for categorizing thinking skills into six different levels: knowledge, comprehension, application, analysis, synthesis, and evaluation (Crowe et al., 2008). Two sample papers and designed questions are shown in Table 2, and scoring rubrics are explained in Table 3. In addition, a detailed illustration of how these questions were designed to examine students’ learning outcomes and cognitive skills is presented in Table 4. Although there are real dividends in the level of confidence that a student's grade reflects comprehension of important concepts (Prichard and Sawyer, 1994), high scores were assigned to students who answered the questions by applying knowledge and critical thinking.

All the answer scripts were marked within 3–4 d after the CA was administered. Scores were uploaded into a grade book in which students could only see their own scores. A student obtaining a score of >70 was regarded as having a good understanding of the relevant contents. However, the student's performance in answering these linked questions during the lecture (SemI 0708) was not graded and not recorded in the student's academic file. A student's answer to these questions in the tutorial (SemII 0809) was graded, but the score was not released and not counted into his/her ranking in this module.

DATA COLLECTION AND ANALYSIS

Data for this study were collected from several different approaches. The IVLE Course Management System at NUS autorecorded the identities and the number of students who accessed the research articles. Linked questions (Table 1) were used to assess students’ conceptual understanding and cognitive skills. Brief conversations with individual students during lab sessions, before and after classes, were conducted to collect preliminary information on the students’ opinions about the method that introduced the paper in the first trial in SemI 0708. A questionnaire (see survey form in Supplemental Material 3) was used in the following three semesters to collect feedback from the students. In addition, the students were encouraged to send their feedback via the online module feedback system of NUS. Such feedback could be anonymous or onymous.

OUTCOMES OF INCORPORATION OF RESEARCH ARTICLES INTO TEACHING

Research articles provide a valuable resource for stimulating self-directed learning and developing high-order cognitive skills (Gillen, 2006, Kozeracki et al., 2006), but effective strategies must be developed to cultivate the habit of reading. To assess students’ motivation for reading research articles, such articles were first introduced to the students during lecture in SemI 0708. The results of the first trial were disappointing because the students’ feedback showed that only a few of them had downloaded and read the recommended articles after the class (Table 5). Because assessment plays a crucial role in directing students’ learning, a novel assessment linking test questions to research articles was developed to persuade students to take responsibility for their own education and, over time, make them more self-directed.

Chromosome structure and gene transfer are the basis of molecular genetics. These terms seem very familiar to undergraduates, but new findings related to chromosome structure and gene transfer continue to increase rapidly. In example 1 (Table 2a), by linking CA questions to the paper by Dunning Hotopp et al. (2007), the students were directed to learn gene transfer between prokaryotes and eukaryotes, which is a new discovery and absent from the textbook and the lecture. The students’ creative thinking skills were challenged by the third question with regard to controversial issues on GMOs. In example 2 (Table 2b), the students were asked to correlate two seemingly unrelated concepts “centromere” and “telomere,” but these concepts are actually interconnected. They were challenged to solve a question related to human diseases requiring processing and integration of their knowledge.

Data from the IVLE autorecording system and the survey showed that all students were engaged to download and read the paper before the CAs (Table 5). Most of the students (72% in SemII 0708 and 83% in SemI 0809) got scores higher than 70, showing a good understanding of the paper content and ability to connect this CA questions with their own prior knowledge (Figures 1and 2). Although some of them indicated that the paper was too difficult, they still managed to answer the questions that required higher-order cognitive skills when being stimulated by the CAs. One of the students wrote, “It was not an easy experience plowing through the academic jargon. However, the experience was exciting. It is a challenge that attracts the brave and determination. I loved it and still love it and believe I will enjoy this mode of examination in the future.” They were impressed that their learning was useful and rote memorization could not work to solve the linked questions.

All student feedback was classified into five categories: very positive (the CA was innovative, inspiring, and motivating), positive (acceptable due to its innovation and application), moderate (better than a memory-based exam but some modifications were expected), negative (good but not suitable for testing first-year students), and very negative (ineffective in revealing students’ understanding; the questions were too specialized). Fifty-four percent of the students in SemII 0708 and 63% in SemI 0809 supported the new CA method (Figure 2). The students commented that they had spent time understanding the concepts instead of memorizing the lecture notes during their preparation for the CA. They also made efforts to compare content in the textbook and research papers, and they enjoyed acquiring new knowledge on their own. Many wished this assessment method could be adopted in other modules or even in the final examination. Representative feedback is presented in Supplemental Material 4.

The students (23.8% in SemII 0708 and 30% in SemI 0809; Figure 2) who gave moderate feedback embraced the CAs because the CAs exposed them to new information and assessed conceptual understanding instead of factual recall. In contrast, most of them complained that they had not been well trained to read research articles before being exposed to the CAs. There were 22% of students in SemII 0708 and 8% students in SemI 0809 (Figure 2) who criticized the CAs for their poor coverage of lecture content and their unfairness to the students who were nonbiological majors. Scientific jargon was the main obstacle to their comprehension.

It is generally assumed that onymous feedback tends to be complimentary, whereas anonymous feedback tends to be critical. Students’ feedback in SemII 0708 was further dissected to look at the differences between onymous and anonymous groups. Sixty-two percent of the students’ feedback was onymous. The largest cohort (51%) in the onymous group highly favored the CA method, whereas the largest cohort (37.5%) in the anonymous group remained neutral. Nevertheless, most students, even those who gave anonymous feedback, still applauded the CA model because the percentage of students (25% + 34.7%) giving positive feedback was significantly larger than those giving negative feedback (12.5% + 16.7%; Figure 3). This result further demonstrated that the new CA model was welcomed by the students. However, this analysis was not performed for the data of SemI 0809 due to very limited onymous feedback.

Though the merits of incorporating research articles into CAs have been demonstrated in SemII 0708 and SemI 0809, the articles were still introduced to students in tutorial in SemII 0809, because some feedback indicated that first-year students were not well trained to understand the articles. Discussion of the research articles in the tutorial might help the students’ understanding. However, the motivational force generated in the tutorial discussion was much lower compared with that in SemII 0708 and SemI 0809; not many students downloaded the articles, and only a few of them read the articles. It seemed that the majority were not motivated and just waited for the lecturer's explanation and guidance on the articles, which were not the desired outcomes. The materials (research articles and linked questions) used in the four semesters were prepared following the same criteria, but the level of student activities and engagement in reading the articles introduced in the lecture (SemI 0708) and the tutorial (SemII 0809) was dramatically lower than that induced by CAs. It has been well documented that student motivation is affected by both contextual and internal factors (Prichard and Sawyer, 1994). There have also been extensive discussions about students’ intrinsic and extrinsic motivation (Ryan and Deci, 2000). In this study, students’ justifications for being unable to read the articles discussed in the lecture and tutorial were that, from common to rare, they did not have time; they were not really interested in the topics; the articles were too difficult; and there were too many reading materials. In contrast, all students read the articles when they were bound with the CAs. This result may indicate that simply telling students that a research article is interesting and that self-directed learning is important may not be enough to actually make them read it. Students want enticement and a tangible reward, such as grades in this study, which can help build self-confidence, competence, and self-esteem. Their perception of value in the materials may affect their determination to read the paper.

The results from the four semesters showed obviously that incorporation of research articles into CAs was the most effective approach to stimulating learning as all students read the articles (Table 5), and a majority of them could correctly answer higher-order cognitive questions (Figure 1). The new CA model helped students become scientifically literate and develop self-directed learning skills. However, this should not exclude the possibility of finding other strategies to improve students’ engagement in reading research papers. In addition, it would be useful to conduct an investigation of those students who were half-forced/half-enticed to read research articles to find out whether they have become interested enough to actively look for research articles on their own. Based on one student's comment, “I found the CA very interesting and enriching. Usually, I would not be interested in reading science papers such as this, but because I was ‘forced’ to read it for the CA, I realized I could tackle the paper down and understand it eventually, despite feeling bewildered with the contents at the beginning”, it may be reasonable to suggest that the students will be more comfortable when they are asked to read research articles in the upper levels of their undergraduate studies. By the way, as noted by Sundberg (2002), each assessment has important advantages and serious limitations. The CA model developed in this study also had its limitations, which included the poor coverage of lecture content and difficult jargon in the research articles.

SUMMARY

This paper presents a novel approach linking CA to research articles. The results showed that the approach strongly motivated students to step out of their comfort zone (textbook) and to develop higher-order cognitive skills, including correlation, application, and synthesis. The CA set a good model of assessment for learning. It went beyond the traditional role of an assessment and may eventually foster scientific literacy among students and develop self-directed learning. In addition, this CA model could be adapted to other modules using research articles from corresponding disciplines. It could also be modified by linking a CA to other types of information such as mass media and the Internet.

REFERENCES