Grappling with the Literature of Education Research and Practice

INTRODUCTION

Ideally, the breadth of education research informs teaching and learning research in the life sciences. However, the body of theoretical and practical research in education is sprawling, making up >20,000 articles published each year in >1100 journals (Mosteller et al., 2004). The absence of any unified, systematic mechanism for cataloging or accessing this information makes it nearly impossible for scientists to keep abreast of the literature on science teaching and learning, much less use it to inform their own work. When the challenge of locating articles of interest is coupled with the difficulty of comprehending the findings of an entirely different discipline, with epistemologies, cultures, and practices distinct from those of the science community (Feuer et al., 2002; Shavelson and Towne, 2002), many scientists throw up their hands in frustration. As I have transitioned from bench to classroom-based research, I have had both the necessity and the luxury, and sometimes the aggravation, of developing a working knowledge of theoretical and practical research in education.

As I have shared references with my scientist colleagues, I have witnessed several beneficial outcomes. Learning about others' teaching and outreach efforts from practitioner journals has helped us develop a more comprehensive idea of the needs, interests, and priorities of our colleagues in education and avoid “reinventing the wheel” in education programming (Dolan et al., 2004). Reviewing the research literature has honed our thinking about how to document the effects of educational interventions on teaching and learning. I have expanded my vocabulary such that I can have more informed discussions with the evaluators of our precollege outreach and partnership work. From a broader perspective, more departments of science are hiring faculty with education expertise (Bush et al., 2006). Scientists familiar with this scholarship may be better prepared to make informed decisions about the promotion and tenure of their education colleagues, and they may also learn that they themselves benefit from participating in pedagogical endeavors (Bower, 1996; Schultz, 1996; Tanner, 2000; McKeown, 2003; Spillane, 2004; Busch and Tanner, 2006).

My intention is not to encourage scientists to become educational researchers, but rather to better position them to benefit as teachers from the education literature. Even education researchers themselves have noted that much of the literature is written for academics in the discipline, rather than a broader audience of researchers, practitioners, and policymakers (Davis, 2007). Assumptions are often delineated using foreign concepts such as “theoretical framework.” Methodology is a combination of research method and epistemologies of learning. Protocols are described using unfamiliar terminology such as “differentiated instruction.” The data may take unfamiliar forms, such as quotes from focus groups, transcripts of interviews, or videotapes of classrooms. As I have learned to locate, decipher, and evaluate the literature, with significant guidance from colleagues and mentors, I have used analogies to science research and practice to clarify my thinking. Although these analogies have limitations, I have found them to be useful steppingstones in better understanding this body of knowledge, and I share several of them here. Thus, the intent of this essay, and one purpose of the new CBE-LSE feature Current Insights: Recent Research in Science Teaching and Learning, is to serve as a bridge for individuals with scientific expertise to enter the land of education scholarship, and to provide tools that may be useful on the journey.

TOOLS FOR ACCESSING THE LITERATURE

Several online databases provide access to education research citations. For example, the Education Resources Information Center (ERIC; http://www.eric.ed.gov) is an Internet-based digital library of education research and information that provides bibliographic records of journal articles and other education-related materials with sponsorship from the U.S. Department of Education. Print versions of ERIC information are published monthly in two formats: Resources in Education and Current Index to Journals in Education. Two other databases, Educational Research Abstracts Online (published by Routledge of Taylor & Francis Group; http://www.informaworld.com/smpp/title∼content=t713417651) and Education Research Complete (published by EBSCO, Ipswich, MA; http://www.epnet.com/thisTopic.php?marketID = 4&topicID = 639) provide abstracts for thousands of journals, books, and monographs, as well as full text for many journals and education-related conference papers. Finally, Google Scholar (http://scholar.google.com) enables searches of scholarly literature, including peer-reviewed papers, theses, books, preprints, abstracts, and technical reports. Google Scholar uses robotic spider software to crawl links to all scholarly articles publicly available on the World Wide Web. The company has standing agreements with academic publishers, professional societies, preprint repositories, and universities, which have helped maximize the “findability” of relevant education scholarship.

Although these and other indexing and abstract services provide points of access, the information provided may not be sufficiently detailed regarding a study's purpose, setting, participants, research design, or other aspects that would help a nonexpert reader evaluate its relevance to his or her interests. In addition, researchers, policymakers, and even parents are increasingly demanding a rapid way to access concise information about educational outcomes to use “scientifically based research” as the grounds for “evidence-based practice” (Feuer et al., 2002; Shavelson and Towne, 2002; Slavin, 2002; St. Pierre, 2006). These demands have spurred a grassroots effort within the education research community to make the research process, including assumptions, qualifiers, and limitations, more transparent by accompanying manuscripts with a “structured abstract.”

First proposed by Mosteller et al. (2004), the structured abstract is designed to make clearer and more accessible a study's salient features so that practitioners and decision makers can more easily locate studies of interest and assess their implications for teaching practice. Kelly and Yin (2007) propose that structured abstracts be used to “make the argumentative structure of education research articles more apparent and open to scrutiny.” Advocates contend that authors should make explicit the nature of their evidence and claims (e.g., descriptive, correlative, causal), circumstances that may affect the strength of their claims (e.g., study setting, size, context), and other qualifiers that might influence the applicability of their claims to teaching practice. ERIC now requests that authors submit a structured abstract with their contributed materials. As structured abstracts become more commonplace, they will likely result in greater accessibility for researchers and decision makers outside the education community.

Scientists at academic institutions that lack education departments face the additional challenge that their libraries generally do not maintain subscriptions to education journals. Thus, open-access publications, which are freely available for reading and reproduction, have special appeal. Several such journals, including CBE-LSE, were initiated in electronic form to support open-access scholarship (Table 1). Other publishers give authors the freedom to choose whether their publications will be freely available, for example, Springer's Open Choice (http://www.springer.com/dal/home/open+choice) and iOpenAccess from Taylor & Francis (http://www.tandf.co.uk/journals/iopenaccess.asp). Unfortunately, not all of their journals participate in these services, and authors may have to pay substantial fees for providing open access to their work. The service Open J-Gate (a contribution of Informatics [India]; http://openj-gate.org) was launched in 2006 to serve as a portal to the open-access literature, annotating all indexed articles with respect to their “peer-review” status.

As in any field, education journals tend to specialize with regard to research questions and methodologies. Some journals feature descriptive essays, others theoretical research, some ethnographic studies, others statistical analyses, and so on. Articles describing teaching strategies and curricular innovations, described as “practice” publications in Table 1, are usually found in journals tailored to a specific teaching and learning audience (e.g., precollege, undergraduate, graduate, science center/museum). Articles in which theories are tested or developed, including those intended to demonstrate relationships between instructional approaches and learning outcomes or to understand the cognitive, social, and cultural underpinnings of teaching and learning, are generally found in journals of applied and theoretical research. Regardless of the question of interest, journals can be identified that have a mission to feature this kind of work or to reach an audience with similar interests (Table 1).

Journals that feature the following kinds of work were not included in Table 1, but they may be of interest: general education, graduate and professional education, education administration and leadership, teaching and learning in other science disciplines (e.g., geoscience, chemistry, physics), informal and nonformal education (i.e., respectively, learning in unstructured settings such as science museums and learning in more structured but not classroom-based settings such as 4-H), evaluation, technological and applied science education (e.g., agricultural education), and educational psychology. In addition, several scientific journals have forums for publishing education articles (e.g., front matter in The Plant Cell, “Genetics Education” in Genetics, “Education Forum” in Science).

TOOLS FOR INTERPRETING THE LITERATURE

Even if an article of interest is successfully located, it may not be decipherable by nonexperts. Life scientists are familiar with the old adage that learning biology, which is replete with discipline-specific terminology, resembles learning a foreign language. Similarly, grappling with the education literature requires learning the vocabulary of both practice and scholarship. Several tools have been developed to assist nonexperts in learning the jargon, including online and hard copy dictionaries and glossaries. The Lingo of Learning: 88 Terms Every Science Teacher Should Know (Colburn, 2003) and EdSpeak: A Glossary of Education Terms, Phrases, Buzzwords, and Jargon (Ravitch, 2007) define education practice terms and research vocabulary most relevant to schools and classrooms. EdSpeak also includes a handy list of acronyms for phrases, policymaking groups, funding agencies, accrediting bodies, and other relevant organizations. Other resources useful for interpreting evaluative, applied, and theoretical research include Fourth Generation Evaluation (Guba and Lincoln, 1989), Qualitative Research & Evaluation Methods (Patton, 2002) and Research Design: Qualitative, Quantitative, and Mixed Method Approaches (Creswell, 2003). Resources from psychology, sociology, and anthropology can also be useful (e.g., Hammond and Brandt, 2004). For example, the Online Dictionary of the Social Sciences (http://bitbucket.icaap.org/; Drislane and Parkinson, 2007), hosted by Canada's Athabasca University, has 1000 entries covering sociology and related disciplines. Explorations in Learning & Instruction: The Theory Into Practice Database (http://tip.psychology.org/; Kearsley, 1994–2007) includes descriptions of >50 theories relevant to human learning and instruction.

Some life scientists may be well prepared to understand the complexities of the anthropological, psychological, and sociological underpinnings of teaching and learning. Ecologists and evolutionary biologists often study phenomena in which they are unable to control, predict, or even characterize all the variables involved. Similarly, education researchers are often not able to control all the factors at play in a learning situation, and they may not want to do so. Rather, some of their most valuable findings emerge from investigating the real contexts in which learning may occur.

Like all research, investigations of teaching and learning begin with a question. Research questions generally fall into three categories (Shavelson and Towne, 2002, pp. 99–101): description (What is happening?), causation (When and with whom? Is it happening in a systematic or generalizable way?), and mechanism (Why is it happening?). In addition, learning behavior can be examined at different depths and with different time frames in mind, including changes in skills, knowledge, attitudes, or interests (short term), behavior and decisions (middle term), and life condition, status, or values (long term). Choices regarding methods of data collection and analysis are influenced by the outcomes that are of interest to the researcher, as illustrated in this fictional example:

In this case, the researcher intended to document the learning of a limited group of students whose teacher may be well positioned to help them, rather than draw conclusions about how all students learn genetics or what students in general learn by using this curriculum. Her initial research question guided her choices regarding data collection, analyses, and interpretation, as well as the scope of her conclusions. Because she collected data by using several approaches, including discussions with students, she was able to ask them questions that made clear their understanding or lack thereof. Because she collected data over time rather than at just one or two time points (e.g., the beginning and end of the relevant units), she was able to develop hypotheses about what classroom occurrences may have altered students' conceptions. Finally, because she sought feedback from the teacher, who has a greater depth and breadth of experience working with these students, she has enhanced the credibility and trustworthiness of her interpretations.

A researcher's perspective and theoretical framework also guide how and why he or she conducts studies. A life scientist's styles of reasoning and experimental practice (e.g., taking a biochemical or genetic approach to studying the cell cycle) are usually obvious from a quick reading of the methods in a paper or from knowledge about the journal where the work was published (e.g., Journal of Biological Chemistry vs. Genetics). As an instructive example, Bill Sullivan, a geneticist, and Doug Kellogg, a biochemist, both at University of California at Santa Cruz, have authored complementary stories illustrating how their perspectives differ (http://review.ucsc.edu/spring04/twoversions.html; Stephens, 2004). Approaching investigations from a genetic versus biochemical perspective influences the questions that are asked, the experimental tools that are used, the data that are collected, the analytical methods that are used, and the conclusions that are drawn, as well as the hypotheses and subsequent questions that are generated.

Similarly, understanding the theoretical framework that guides an educational study can help readers identify the perspective of the researchers and anticipate the types of questions, data, analyses, and findings that will be included (Bodner and Orgill, 2007). For example, cognitive load theory rests on the premise that learning happens best in ways that are aligned with the organization of the brain and the nature of cognition, as understood from cognitive psychology and neuroscience research (Sweller, 1988; also see http://tip.psychology.org/sweller.html). For example, the cognitive load of learners depends on their experience and expertise, which influences their short-term, long-term, and working memory capacities. Experts' knowledge is organized into schemas that facilitate learning, lowering the cognitive load required for learning and enabling them to process information with greater efficiency (Bransford et al., 1999). Novices have not developed such schemas; thus, they are more limited in the amount of information they can take in and incorporate using working memory. A study framed by cognitive load theory might consider how learning materials could be designed to minimize the amount of information provided to novice learners during the learning process, or to teach novices explicitly about expert schemas to help them organize their thinking during learning.

TOOLS FOR EVALUATING THE LITERATURE

The intent of most education research is to understand or explain social and psychological phenomena related to learning (Firestone, 1987). Researchers have developed models that are cognitive, behavioral, and social, with respective focuses on reasoning and memory, doing and action, and interacting and culture. Regardless of which model is used, research entails subjecting data, in whatever form, to systematic analysis. Yet, human thinking and relationships are extraordinarily complex phenomena that are not straightforward to analyze systematically. How such analysis manifests is again dependent on the research question (Anfara et al., 2002). In general, data should be sufficient, credible, and accurate to be considered evidence in support of a claim (Toulmin, 2003; Kelly and Yin, 2007). In other words, the data should be adequate to support the claim, and there should be a reasonable relationship between the data and the claims they support. The methods for data collection and analysis should be appropriate for supporting the claims, and so on. These points may seem obvious, but, when considering the literature as a whole, the data and methods can be strikingly varied depending on the questions being asked. Thus, a common rubric for evaluating the quality of studies in education must be considered at this broad level, rather than based on any particular methodological approach or type of data (Grossman and Mackenzie, 2005).

In evaluating the methodology of an education study, the reader must take into account what research questions are addressed (Ercikan and Roth, 2006). For example, randomized controlled trials or investigations with well-matched comparison groups are well suited to investigating causal relationships between interventions and outcomes. Yet, in many cases, these study designs are not feasible (i.e., it is unrealistic to randomly assign students to classes) and they are costly (Olson, 2004; Grossman and Mackenzie, 2005). For experimental or quasi-experimental findings to have value, the instrument used for data collection (e.g., an exam, survey, or questionnaire) must be valid (i.e., it actually measures what it is purported to measure in the participating population) and reliable (i.e., the instrument would yield the same responses from the same individual if it is administered at different times). High-quality instruments must be informed by current theory and knowledge about teaching and learning (e.g., what are students misconceptions about cellular respiration and how can they be identified with the instrument?), and they must be validated by pilot testing within the population of interest and conducting appropriate statistical analyses (e.g., confirmatory factor analysis; see Aikenhead and Ryan (1992) as an example).

Insight gleaned about causal relationships between teaching strategies or curricular innovations and student and teacher outcomes may be applicable only to those individuals in that setting at that point in time. For such findings to be generalizable, credible evidence must be collected to demonstrate their applicability across populations and settings. If claims are being made about the transferability of findings to other students or teachers, the individuals in the new setting must resemble in some way the individuals in the original setting of the study. For example, findings from investigations in urban schools may not be applicable for rural schools, because urban schools have larger immigrant populations and more English language learners.

Qualitative approaches provide opportunities to capture unintended outcomes, understand why certain outcomes occurred, and gain a deeper understanding of a phenomenon (Denzin and Lincoln, 2005, p. 5). Such research is intended to describe an experience and infer patterns about it or consider how it is representative of a broader set of experiences (Ercikan and Roth, 2006). Qualitative data tell a story by capturing and communicating someone else's experience, taking into account the perspectives, time, and situation of individuals involved, including the participants and even the researcher. The results can illuminate the actuality of teaching and learning in the real time and setting of a classroom (e.g., what is actually happening in this teaching and learning situation?). In addition, qualitative findings can serve as a proof of principle (e.g., is it possible to teach and learn in this way or using this curriculum?), a basis for generating new hypotheses (e.g., if these students learn in this way, do other students in other settings at other times learn in this same way or in other ways?), and a way to discover unanticipated outcomes (e.g., students did not seem to gain knowledge about cellular respiration, but they did expand their understanding of how scientific knowledge is generated).

Scientists also use qualitative approaches and evidence in research (e.g., photographs to illustrate differences among cells or organisms, rich descriptions to explain the identification of a new species) and in training. A less obvious example is the oral preliminary exam that is the rite of passage to degree candidacy for all scientists-in-training. These exams are designed to ensure that the student is prepared to pursue an original line of inquiry, for example, by demonstrating awareness and understanding of relevant literature and methods, as well as some ability to interpret data, develop hypotheses, and design experiments to test them and rule out alternative explanations. Some aspects of exam content and structure are generalizable across the doctoral student population (e.g., all exams involve questioning by a group of faculty, all exams have a “grade” or outcome for the student). Yet, each exam is unique to the student, the student's research interests and completed course work, and the panel of faculty examiners. Faculty may start with certain questions in mind but may develop new questions or alter the direction of their questioning as the student articulates his or her understanding.

The goal of the preliminary exam is to investigate in-depth the quality of one student's thinking by speaking with the individual and considering the context, not to generalize to other students. Yet, I expect that all students and faculty involved in preliminary exams intend for such experiences to yield trustworthy, dependable, and confirmable outcomes. The structure of the exam helps maximize the likelihood that this is the case. Preliminary exams involve multiple faculty asking many questions from different perspectives (i.e., triangulation of data sources and methods) over a length of time (i.e., prolonged engagement in the field; Anfara et al., 2002). Although preliminary exams are not research studies, they demonstrate how qualitative methods of data collection, analysis, and interpretation can be designed to maximize the sufficiency, credibility, and accuracy of the resulting data and the claims they support.

CONCLUSION

Many of the distinctions apparent between the cultures of science and education research resemble those seen between the cultures of science and education practice (Tanner et al., 2003). Yet, as Tanner and colleagues note and my analogies hopefully illustrate, there is also common ground. The intention of this essay and the new CBE-LSE feature is to provide windows through which scientists can get a clearer and more comprehensive view of education scholarship in a way that can inform their teaching. I invite readers to suggest current articles of interest in life science education, as well as influential papers published in the more distant past or in the broader field of education research, to be featured in the new column. Please send any suggestions to edolan@vt.edu.

REFERENCES