The C.R.E.A.T.E. Approach to Primary Literature Shifts Undergraduates’ Self-Assessed Ability to Read and Analyze Journal Articles, Attitudes about Science, and Epistemological Beliefs

INTRODUCTION

As scientific information continues to accumulate at a rapid pace, there is a growing sense among science educators that long-established practices need to be reconsidered. Numerous 21st-century science reform documents (American Association for Higher Education, 2000; U.S. Department of Education, 2000; National Research Council [NRC], 2003; Malcom et al., 2005; Alberts, 2005; NRC 2007, 2009; American Association for the Advancement of Science [AAAS], 2010) suggest focusing less on content coverage and more on approaches that reveal science to be an ongoing creative process. Ideally, such a change would help to stem the long-standing attrition of bright students from science majors and, by extension, science research careers (Seymour and Hewitt, 1997; Cech and Kennedy, 2005; DePass and Chubin, 2009). At the same time, student attitudes toward learning (student epistemologies; Schommer 1990, 1993), student self-efficacy (confidence in ability to work effectively in a particular context; Lawson et al., 2007), and student attitudes about science (Osborne, 2003) have been demonstrated to be important factors affecting students’ success in the science classroom. Thus, both changes in how science is taught and consideration of factors influencing students’ ability to learn deserve focus in science education reform efforts.

A variety of new approaches that employ alternatives to lectures, including hands-on classroom activities and small-group work (Klionsky, 1998; Handelsman et al., 2004; Allen and Tanner, 2005; Knight and Wood, 2005), highlighting controversy to stimulate student engagement (Seethaler, 2005; Campion et al., 2009), student participation in ongoing grant-funded research projects (Hanauer et al., 2006; Call et al., 2007; Lopatto et al., 2008; Clark et al., 2009), case study approaches (Herreid 1994a; Chaplin, 2009), use of the popular press (Strauss, 2005; Hoskins, 2010), and analysis of primary literature (Herreid, 1994b; Janick-Buckner, 1997; Lynd-Balta, 2006; Kozeracki et al., 2006; Hoskins et al., 2007; Hoskins, 2008; Schinske et al., 2008; Yarden, 2009), shift classroom focus from a teacher-centered situation in which students are largely passive, to a student-centered classroom (Freeman et al., 2007; Klymkowsky, 2007; Armbruster et al., 2009; Hoskins and Stevens, 2009) more supportive of cognitive activities associated with learning (Bloom and Krathwohl, 1956; Chickering and Gamson, 1987; Zull, 2002).

We have focused on primary literature as a portal into the scientific research process through the C.R.E.A.T.E. (Consider, Read, Elucidate hypotheses, Analyze and interpret data, Think of the next Experiment) method. C.R.E.A.T.E. uses intensive critical analysis of a series of papers generated sequentially from one lab, coupled with email interviews of paper authors via a survey of student-generated questions, to demystify and humanize science. The approach is an iterative method, whereby individual steps of the process (Figure 1 and Table 1) provide students an organized approach to individual journal articles that are “dissected” using a series of novel or adapted pedagogical tools in preparation for intensive class discussion. Papers from a single lab are read in series (Figure 1), allowing students to follow the arc of a research project as it actually progressed. Students are not provided with the full series of papers in advance, nor with the titles, authors, abstracts, or discussions of the papers under consideration. While students could Google the missing information, doing so is ultimately more a hindrance, blunting creative thought, than a help. We encourage students to instead treat the course as a process of discovery. By working with a suite of novel or adapted pedagogical tools to prepare for class, students are empowered to participate actively in the lab-meeting atmosphere of the class sessions, where figures and tables are examined individually, and the logic of the overall study is examined. Previous work has documented precourse versus postcourse shifts in C.R.E.A.T.E. students’ critical thinking and content integration abilities, as well as changes in self-assessed attitudes about science and scientists, as determined by postcourse interviews and the Student Assessed Learning Gains instrument (Hoskins et al., 2007). We report here the results of a survey designed to examine additional aspects of students’ attitudes, beliefs, and self-assessed abilities, comparing responses pre- and post-C.R.E.A.T.E. course.

Students’ beliefs about learning and knowledge affect their ability to learn and their application of metacognitive strategies, including their integration of prior knowledge with the task at hand, studying for understanding rather than superficial recall, and assessing what they do and do not comprehend. Such approaches can significantly facilitate students’ understanding of science (Hartman, 2002; Schraw et al., 2006; Pulmones, 2010). Students’ attitudes about the nature of knowledge, for example, whether knowledge can change over time, and their attitudes about intelligence, for example, whether it is innate and fixed or malleable, comprise a set of epistemological beliefs that affect learning and understanding (Schommer, 1990). Students’ epistemological understandings are typically less sophisticated than those of their professors (Hogan and Maglienti, 2001), and these views can affect study approaches, as well as the extent to which students persist in challenging tasks (Schommer, 1993, 1994; Hofer, 2004). Reasoning ability has also been linked to epistemological beliefs, with students whose epistemologies are more sophisticated showing enhanced skills (Zeineddin and Abd-El-Khalick, 2010).

As the constructivist C.R.E.A.T.E. method uses a number of activities and pedagogical tools (Table 1) designed to increase both student engagement and metacognition, we hypothesized that students’ attitudes toward primary literature, the practice of science, and the nature of learning might change during the semester. We developed a questionnaire aimed at assessing students’ self-assessed views about science, scientists, the research process, and aspects of learning, and administered it on the first and last days of the 14-wk semester. Analysis of student responses indicates that C.R.E.A.T.E. students shifted significantly in their understanding of numerous aspects of scientific research, their approach to reading scientific literature, their confidence in their ability to understand science, and, perhaps most interesting, their epistemological beliefs. Because naïve epistemological beliefs can affect students’ study approaches, learning gains, and ability to interpret complex scientific information (Schommer, 1990; Kardash and Scholes, 1996; Pulmones, 2010), shifting student epistemology can be a first step toward developing attitudes toward knowledge and learning more supportive of student success. In this regard, it is notable the changes reported here were achieved during a single semester in a course that essentially costs nothing to implement and does not involve a hands-on laboratory component.

METHODS

Participants in the study were students in an upper-level elective at the City College of New York (CCNY). The class met twice weekly for a total of 140 min (2005; three credit hours) or 200 min (2006–2009; four credit hours) per week. Class size averaged 27 students (range: 19–32). Seven iterations of the course are included in this study. Most students were junior or senior biology majors who had completed the course prerequisites at CCNY: a year of introductory biology, and one semester each of genetics and cell/molecular biology. A few students (<10% of each class) were participants in the CCNY post-baccalaureate program. These students had earned degrees in other fields and returned to college to complete premed requirements. In the seven classes represented, 65% of students were female and 61% were African American, Hispanic, or Native American, all groups currently underrepresented at all levels of academic science (National Science Foundation [NSF], 2002, 2008; Atwell, 2004).

Presurvey/Postsurvey of Student Attitudes and Self-Rated Abilities

On the first and last days of the semester, students filled out an anonymous Likert-style survey aimed at elucidating their degree of agreement/disagreement with a series of statements. They also answered several open-ended questions on the survey. The survey statements focused on attitudes and beliefs about issues the C.R.E.A.T.E. approach was designed to address, including students’ self-rated ability to understand and analyze primary literature; whether primary literature had influenced their understanding of science; students’ understanding of the scientific research process; and students’ self-rated science reading ability, confidence in their ability to “think like a scientist,” understanding of “scientists as people,” and sense of whether research science was an appealing career choice. We designed the survey based on our experiences teaching from primary literature in previous classes, focusing on a variety of issues we had determined to be problematic for previous students. Open-ended questions requiring written answers focused on students’ understanding of the activities undertaken by research scientists were set aside for later analysis. This survey was administered in each C.R.E.A.T.E. class (two sections per year in 2005 and 2008 and one section per year in 2006, 2007, and 2009). The 2005 cohort of students was included in a previous analysis of the effects of the C.R.E.A.T.E. class on student critical thinking, content integration, and attitudes toward science/scientists (Hoskins et al., 2007).

Surveys were anonymous and coded with numbers known only to the students themselves, to allow alignment of data for within-subject statistical analysis. The survey included 52 statements to which students reacted by marking “I strongly agree,” “I agree,” “I’m not sure,” “I disagree,” or “I strongly disagree” on their survey sheets. Some sample statements were phrased positively (e.g., “I could make a simple diagram that provided an overview of an entire experiment”) and others negatively (e.g., “I do not have a good sense of what motivates people to go into research”).

Three additional propositions were aimed at eliciting students’ overall sense of their ability to read/analyze journal articles, their understanding of the nature of science, and the extent to which primary literature had helped them to understand the nature of science (A, B, C). Question C had four possible responses ranging from “no influence” to “major influence,” and all other questions had five possible responses, phrased in parallel to the question posed (e.g., for question B, on understanding of the nature of science, possible answers ranged from “no understanding” to “understand it very well”). Postcourse, all surveys for which both “pre” and “post” copies were available were scored on a five-point scale, with “strongly agree” = 5 and “strongly disagree” = 1. The additional questions were scored in a parallel way, with scores for C ranging from 1–4 rather than 1–5.

RESULTS

The C.R.E.A.T.E. Survey

The C.R.E.A.T.E. survey included a collection of seven summary items (Table 2) and 52 specific skill and attitude items deemed relevant to the course goals. Following the accrual of 140 cases for the summary items and 155 cases for the skill and attitude items, the data were used to improve the usefulness of the survey. Initial inspection of the 52 skill and attitude items revealed some items were repetitious and others were unrelated to other items (low communalities) or to summary items. These items were set aside, leaving 38 candidates for continued analysis. Of the 38 items, 13 were similar to items used in research on epistemological attitude (Schommer, 1990) and attitude toward science. These are described below (Tables 5, 6, and related text). The epistemological items were a sampling of items developed by Schommer (1990) as part of a much broader investigation of epistemological beliefs. The items here were drawn from across the factors derived from Schommer's survey, and so include items representing the belief that knowledge is certain (e.g., that different scientists will come to similar conclusions) and that ability is innate (e.g., that scientists were born with a special talent), as well as items assessing attitude toward science (e.g., science is a creative endeavor). Because Schommer's previous work showed that these epistemological beliefs do not constitute a single scale, we did not include them in exploratory factor analysis, but analyzed them separately. The remaining 25 items were analyzed by means of a principal component analysis (PCA) to explore underlying factors that might aid in the reduction of the 25 variables to a more manageable set. The PCA was performed on 25 skill and attitude items, with a varimax rotation to aid with interpretation. The resulting analysis yielded eight factors accounting for 64% of the variance in the data; however, some variables were “split” across factors, resulting in two factors that were uninterpretable. These were set aside. The remaining six factors, with their member items and factor loadings, are shown in Table 3. The first factor, which we name Decoding Primary Literature, includes items that refer to scientific language and scientific literature, and indicates the respondent's feelings about reading primary scientific literature. The second factor, Interpreting Data, includes items that have to do with data presented in tables and graphs, as well as with data transformations. The third factor, Active Reading, includes items about diagrams, displays, and method. Visualization, the fourth factor, includes visualizing the method of a study and interpreting graphs. The fifth factor is named Thinking Like a Scientist and includes items about explaining a scientific paper and thinking of experiments. The final factor is called Research in Context and includes items about animal models and controls in experiments.

Pretest–Posttest Differences

Raw scores for the items in each factor were summed, resulting in six pretest scores and six posttest scores for each student respondent. A paired-difference t test was performed on each of the six factors. The results are shown in Table 4. Each pretest–posttest difference is highly significant in the expected direction of posttest gains. The magnitude of the change, estimated as standard deviations in the final column of Table 4, is medium to large (Cohen, 1988). This magnitude of change, as well as the stringent level for significance testing, argues against the presence of a type I error (spurious significant differences).

Epistemological Beliefs

Table 5 shows 13 items related either to Schommer's constructs of certain knowledge and innate ability or to a general attitude toward science. It would be expected that students with an insightful attitude toward science would believe that knowledge is not certain and unchangeable; that scientific ability is not fixed and innate, and that science is a creative and collaborative endeavor. To explore the change in these variables from pre- to posttest, the items under the certain knowledge heading were summed (Cronbach's alpha = 0.66 for this scale). Similarly, the two items for innate knowledge were summed. The attitude items were examined individually. The result of paired-difference t-tests for pre- versus postcourse data are shown in Table 6. There were significant positive gains on all the variables. The possibility of the presence of a type I error is reduced by the stringent level for significance and the moderate effect sizes.

Relationships to Summary Item of Reading Confidence

In an effort to synthesize the findings, we chose to ask whether any of the components of the C.R.E.A.T.E. survey identified so far relate to the student participant's overall view of his or her ability to read science articles. For this purpose, we set the data for the first summary item (rate your confidence in your ability to read and analyze science journal articles) as a dependent variable for multiple linear regression. Specifically, the postcourse confidence data were analyzed using a multiple regression analysis, with predictors including precourse responses to the same item (pretest reading confidence), the sums of the raw data scores indicated by the factor groupings in Table 3 (postcourse data), and the epistemological items in Table 5 (postcourse data). A stepwise multiple regression analysis indicated that a model with three significant predictors, the scores relating to Decoding Primary Literature, the first factor in Table 3 (standardized coefficient 0.33; p < 0.01); the scores relating to Thinking Like a Scientist, the fifth factor in Table 3 (standardized coefficient 0.27; p < 0.01); and the pretest reading confidence level (standardized coefficient 0.17; p < 0.02), significantly predicted posttest reading confidence (r = 0.58; df = 3, 126; p < 0.05).

DISCUSSION

Experiences That Shift Students’ Attitudes about Science and Their Own Scientific Abilities

The one-semester C.R.E.A.T.E. course enhanced students’ confidence in their ability to read, understand, and explain science, as well as to understand how research is carried out (Tables 3 and 4). Students changed significantly on summary variables that assessed self-rated ability to design experiments, visualize methods based on data, visualize lab activities based on the written account in the journal article, manipulate data, relate results of individual experiments to “the big picture,” critically review data, read science with appropriate skepticism, and explain results to others. These topic areas touch on a wide range of activities undertaken by working researchers.

Our survey findings are consistent with a model that predicts that students’ overall confidence in their ability to read and analyze journal articles is based largely on their sense that they can: 1) Decode Primary Literature (Table 3, factor 1) and 2) Think Like a Scientist (Table 3, factor 5). The gains in students’ confidence in these abilities are likely related to the design of the C.R.E.A.T.E. course. Class discussion and homework assignments challenge students to work through the experiments and interpret the findings of published studies as if they had done the work themselves. During the semester, students carry out numerous activities typical of working scientists. As such, the approach may promote more epistemological engagement than does a standard class, thus engendering changes in students’ attitudes and beliefs about science.

Overall, the approaches typical of the C.R.E.A.T.E. classroom are likely to encourage students to employ metacognitive strategies that help them interpret complex information (Hartman, 2002; Nordell, 2009). Gaining deeper understanding of challenging material may in turn help students overcome widely held misconceptions about research science, for example, that experiments are done to demonstrate concepts already known, and therefore research is not a creative activity, and that researchers are simply fact-gatherers (Sandoval, 2003). Students’ repeated opportunities to design and evaluate experiments and models throughout the C.R.E.A.T.E. semester (Figure 1 and Table 1) may contribute to the shifts we noted in their views about the creativity of science (Tables 5 and 6). Developing their own questions for paper authors (e.g., “Do you have to be a straight-A student to become a researcher?” “What would be your ‘dream discovery’?”) encourages students to think beyond the data of the papers and consider the overall process of becoming and being a scientist.

Currently, undergraduate research experiences (UREs) are considered to be one of the most important mechanisms for stimulating students’ interest in science careers. The effects of UREs have been investigated using surveys of students’ experiences (Rauckhorst et al., 2001; Lopatto, 2004a, b, 2007, 2009; Russell, 2006; Russell et al., 2007) and extended interviews (Hunter et al., 2007). These studies have reported high student enthusiasm for UREs, as well as major benefits for multiple aspects of students’ understanding of science, their hands-on research skills, and their attitudes toward research careers. In a longitudinal ethnographic study of the effects of UREs on students at four liberal arts institutions, participants noted in interviews that UREs increased their research confidence and sense of “feeling like a scientist” (Hunter et al., 2007). Students who participate in science UREs are likely to already be interested in scientific research, and UREs clearly reinforce the aspirations of these students. In this respect, C.R.E.A.T.E. may reach a broader group of students, including many who have not previously considered careers in science.

The concept of “nature of science” includes, for many science educators, the ideas that scientists build understanding on observations of nature, that explanations and understanding can change over time, and that creativity comes into play throughout the research process (Lederman, 1992; Karakas, 2009). While there is general agreement that students need to understand the nature and processes of science, or “science as a way of knowing” (AAAS, 1993, p. 2; see also AAAS 1989, 2010; NRC 2009), it is less clear how to accomplish this goal. Teaching approaches focused on inquiry have been suggested as a way to build student understanding of the nature of science (Aulls and Shore, 2008; Shore et al., 2008) and enhance learning (Quitadamo et al., 2008). “Inquiry” alone, however, may not be sufficient to shift students’ concepts of the nature of science (Lederman et al., 1998; Sandoval, 2003; Schwartz et al., 2004) or to encourage students to use metacognitive approaches when studying science (Butler et al., 2008). Our study finds that, although the C.R.E.A.T.E. course does not include a laboratory component or independent research projects, students nevertheless report substantial changes in attitudes and beliefs about science during the semester. Being challenged to devise their own research questions, analyze and interpret data, design experiments, and carry out peer review of studies devised by other students may stimulate C.R.E.A.T.E. participants to examine their personal beliefs about science. In addition, student interview data suggest the email survey of authors plays a role in shifting students’ understanding of “who does science, and why?” (see Tables 1 and S1 in Hoskins et al., 2007). In this context, it is notable that students participating in a novel Deconstructing Scientific Research course, which focuses on intensive analysis of an individual research seminar and also lacks a hands-on component, showed large gains in multiple categories addressed by the Survey of Undergraduate Research Experiences instrument (Lopatto, 2004a, 2007, 2009), including the ability to “understand how knowledge is constructed” (Clark et al., 2009).

Epistemologies, Learning, and the Nature of Science

Our survey also addressed aspects of students’ epistemological beliefs. Schommer and colleagues (Schommer, 1990; Schommer et al., 1992) have identified epistemological beliefs that moderate learning in a variety of intellectual domains. For example, the beliefs that knowledge is certain, that authority should be trusted, that learning is quick and simple, and that intellectual talent is innate can interfere with striving to learn. The C.R.E.A.T.E. program, by uncovering the process of scientific thinking and by providing contact between students and professionals, may influence these epistemological beliefs in a beneficial way. We found substantial changes during the semester in students’ views of “scientists”; moderate shifts in students’ sense of whether knowledge is certain and ability is innate, the creativity of science, or understanding of motives that drive scientists; and a small shift in students’ views of science as a collaborative activity (Tables 5 and 6).

Undergraduates’ epistemological beliefs shift during the college years from a sense that knowledge is certain, typical of freshmen, to a more nuanced view of the relative nature of knowledge, held by seniors (Perry, 1970), and a longitudinal study suggests such views continue to change postcollege (Baxter Magolda, 2004). Epistemological beliefs change slowly during the college years (Perry, 1970), and only a minority of students achieve mature epistemological understanding by senior year (Baxter Magolda, 1992). For both high school (Schommer, 1993) and college (Schommer, 1990, 1993; Hofer, 2000, 2004) students, the sophistication of their epistemological beliefs correlates with their reading comprehension and academic performance, with naïve beliefs linked to lesser achievement. Epistemological beliefs also affect student metacognition (Hartman, 2002; Schommer-Aikins, 2002; Hofer, 2004), ability to integrate information (Schommer, 1993), and persistence when confronted with a challenging task (Dweck and Leggett, 1988). The interrelationships among personal epistemologies, metacognition, and learning are complex, but there is general agreement that naïve epistemologies may interfere with learning (Hofer, 2004; Pulmones, 2010). Overall, students with naïve epistemologies employ fewer of the metacognitive strategies (e.g., setting goals, monitoring progress, self-questioning, and connecting new information to broader concepts) that support self-directed learning (Zimmerman, 1990; Hartman, 2002; Pieschel et al., 2008; Strømsø et al., 2008).

The C.R.E.A.T.E. method's combination of epistemically challenging approaches applied in an authentic context may underlie the changes we saw in students’ epistemological beliefs. Several aspects of the C.R.E.A.T.E. approach present students with novel cognitive challenges and associated epistemic load. C.R.E.A.T.E. students employ visualization when sketching cartoons that fill the gap between the methods section and the charts, graphs, blots, and/or photomicrographs presented. Integrating verbal information with visual information promotes integration of different modalities. Such integrative thinking, reinforced by C.R.E.A.T.E.'s repeated use of concept maps, both as a tool for review and a way to organize papers’ central themes, can facilitate learning (Novak, 1991; Van Meter and Garner, 2005; Schwartz and Heiser, 2006). Class discussion often focuses on a point of controversy, which can both increase student engagement (Bell and Linn, 2002) and stimulate students to “do the real intellectual ‘work’ of synthesizing ideas across subdomains” (Seethaler, 2005, p. 273). C.R.E.A.T.E.'s narrow focus on a few papers may encourage students to work toward deep rather than superficial understanding (Schwartz et al., 2009) as they engage in cognitively stimulating activities corresponding to upper levels of Bloom's taxonomy (e.g., analysis, synthesis, evaluation: levels 4–6; Bloom and Krathwohl, 1956). Extended discussions involving scientific argumentation are rare in lecture-dominated classrooms (Osborne, 2010), but can be of substantial benefit, especially when students feel free to develop their understanding through discussion and to speculate aloud as they do so (Sawyer, 2006). C.R.E.A.T.E. “grant panel” activities encourage student reflection on the research process beyond the details of individual journal articles.

UREs might also be expected to have a strong effect on students’ epistemological beliefs. This has been seen in some cases (Rauckhorst et al., 2001; Lopatto, 2004b; both studies include both science and nonscience URE participants), but in other studies epistemological beliefs appeared not to shift significantly during the URE, either as reported by student participants or their faculty mentors (Hunter et al., 2007). A recent meta-analysis of independent research experiences in science (Sadler et al., 2010) suggests that supplementing research experiences with specific additional activities, such as keeping a journal of reflections on the research experience (Rauckhorst et al., 2001, college students; Schwartz et al., 2004, high school teachers) or interacting with peers also involved in research apprenticeships (Grindstaff and Richmond, 2008; high school students), can expand gains made in UREs and enhance understanding of the nature of science. These researchers further note that developing a deeper understanding of the nature of science will probably require instructional approaches that ensure undergraduates’ participation in developing hypotheses and analyzing data, both considered “epistemically demanding practices” (Sadler and McKinney, 2010, p. 48). Other investigators have suggested that epistemological beliefs are more likely to change if students are trained to think critically in a context that encourages metacognition and includes controversy (Valanides and Angeli, 2005). In studies of high school students (Bell et al., 2003) and undergraduates (Ryder et al., 1999), changes in scientific thinking were seen in students who participated in projects that demanded substantial epistemic engagement. Conversely, classrooms lacking in authentic scientific inquiry activities can reinforce naïve epistemological beliefs, for example, that scientific logic is simple and conclusions certain (Chinn and Malhotra, 2002). Students’ ability to carry out research projects may be constrained by such beliefs (Ryder and Leach, 1999).

We consider it likely that the shifts we see in epistemological beliefs of C.R.E.A.T.E. students are attributable to students’ experiences in the C.R.E.A.T.E. course. We did not measure presemester/postsemester epistemological beliefs in an independent control group of students who did not take the course, as no such control group was available. We are not, however, aware of any studies that show that increased sophistication of epistemological beliefs results from mere maturation or passage of time during a single semester. The experiences of science students in UREs may provide some insight into the malleability of students’ epistemological beliefs. Science students’ UREs would be expected to support or enhance any shifts in their epistemological beliefs that occurred “maturationally” during an academic semester. Thus, the finding that epistemological beliefs tend to remain stable in science URE participants interviewed repeatedly over several years (Hunter et al., 2007), suggests that the epistemological beliefs of undergraduate science students do not shift rapidly. We feel it is likely that the postsemester versus presemester changes we document were brought about by students’ experiences in the semester-long C.R.E.A.T.E. course.

Although the C.R.E.A.T.E. approach is not unique in focusing on primary literature, it is unusual in its combination of intensive analysis of a series of related publications with an email survey of their authors, and its concentration in the classroom on discussion and analysis aimed at simultaneously decoding the figures and tables, modeling the research process, and humanizing the scientists behind the papers. The C.R.E.A.T.E. teaching method encourages students to engage in conversations, debates, and creative thinking, which involve cognitive challenges that can help develop understanding of complex material (Driver et al., 2000; Marbach-Ad and Sokolove, 2000; Bell and Linn, 2002; Seethaler, 2005; Campion et al., 2009) and at the same time encourage creative approaches to such material (DeHaan, 2009). Overall, our findings indicate that the C.R.E.A.T.E. method increases students’ confidence in their ability to read and understand primary literature, improves their self-assessed understanding of the nature and processes of science, and encourages their development of more sophisticated epistemological beliefs. We suggest that complementing existing curricula with inexpensive C.R.E.A.T.E.-style courses could be an effective way to help students develop deeper insight into the nature and practices of science. Finally, students who recognize early in their college years that science is creative and open-ended might be more likely to take advantage of the UREs that can stimulate and reinforce interest in science research careers.

ACKNOWLEDGMENTS

REFERENCES