| HOME | HELP | FEEDBACK | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
||||||
|
||||||||
ARTICLES |
* Department of Physics, Davidson College,
Davidson, North Carolina 28035; Department of
Physics, North Carolina State University, Raleigh, North Carolina 27695
Submitted April 2, 2002; Revised May 21, 2002; Accepted May 28, 2002
 |
ABSTRACT |
|---|
 TOP ABSTRACT INTRODUCTIONFORMAL EDUCATIONAL RESEARCH...WHY BIOLOGISTS SHOULD CONDUCT...METHODS OF EDUCATIONAL RESEARCHEXAMPLE OF RESEARCHRESOURCES FOR ADDITIONAL TOOLS...CONCLUSIONAPPENDIXREFERENCES |
|---|
Key Words: qualitative research • research methods • interviews
|
INTRODUCTION |
|---|
|
TOPABSTRACTINTRODUCTIONFORMAL EDUCATIONAL RESEARCH...WHY BIOLOGISTS SHOULD CONDUCT...METHODS OF EDUCATIONAL RESEARCHEXAMPLE OF RESEARCHRESOURCES FOR ADDITIONAL TOOLS...CONCLUSIONAPPENDIXREFERENCES |
|---|
Cell Biology Education represents a commitment by biologists to focus on improved teaching and learning in biology backed by peer-reviewed work. It will provide a wonderful avenue of communication and, it is hoped, inspiration. Nevertheless, how do you undertake a study of teaching and learning? Can you gain valuable insights from your own small class of students? The answer is a resounding yes. In the rest of this article we argue the necessity of such research and provide a brief guide to appropriate methodology.
|
FORMAL EDUCATIONAL RESEARCH VERSUS OBSERVANT TEACHER |
|---|
|
TOPABSTRACTINTRODUCTIONFORMAL EDUCATIONAL RESEARCH...WHY BIOLOGISTS SHOULD CONDUCT...METHODS OF EDUCATIONAL RESEARCHEXAMPLE OF RESEARCHRESOURCES FOR ADDITIONAL TOOLS...CONCLUSIONAPPENDIXREFERENCES |
|---|
By paying attention to the feedback of a normal course, we can learn a lot about our teaching and our students' learning. However, equating this endeavor with educational research is like saying a child interacting with his or her world is conducting scientific research. The child will function in the physical world very well, but she will still have basic misconceptions that will lead her to erroneous predictions. Feedback that comes from teaching a course can be enlightening, but ultimately its worth is limited. It is anecdotal in nature, not based on rigorous testing of a hypothesis and the elimination of alternatives, and it cannot be generalized. Simply because a method worked well in one course, with a specific group of students, does not mean it will work well in other courses.
Just as a rigorous study of the physical world was required to change scientific thought from an Aristotelian point of view to a Newtonian (and eventually to an Einsteinian) viewpoint, a rigorous study is necessary for us to make true progress in the realm of teaching and learning biology. A study of teaching and learning, based on rigorous research methods, can lead to a worthwhile increase in our ability to help students learn. The results may be surprising and they may not be what we want to find. However, done well, research can provide us with many valuable insights that could not be obtained by simply paying attention to our own teaching.
|
WHY BIOLOGISTS SHOULD CONDUCT EDUCATIONAL RESEARCH |
|---|
|
TOPABSTRACTINTRODUCTIONFORMAL EDUCATIONAL RESEARCH...WHY BIOLOGISTS SHOULD CONDUCT...METHODS OF EDUCATIONAL RESEARCHEXAMPLE OF RESEARCHRESOURCES FOR ADDITIONAL TOOLS...CONCLUSIONAPPENDIXREFERENCES |
|---|
Biologists may not be experts in education, but they are experts in biology. This adds a distinct advantage to research in biology education. To make sense of students' concepts about cell biology, the researcher must have a firm understanding of the concepts. Although the necessity of strong content knowledge is most obvious when student learning in upper-level courses is being researched, it is also important in introductory-level courses. Areas of student difficulty in the introductory level often persist well into graduate school. Put simply, there are many insights into student learning of biology that are best discovered by someone intimately connected with the discipline.
In addition to being experts in biology, teaching biologists are familiar with biology education. An education researcher may know more of the theoretical underpinnings of teaching, but the teaching biologist has spent hours in the biology classroom and has an enormous range of practical experiences to draw on. It is difficult to research teaching and learning in a course that the researcher has never taught. For this reason, we (the authors) argue that understanding of student learning in biology courses must come from biologists.
Biologists also have a practical advantage when it comes to biological education research. Teaching biologists have easy access to potential research opportunities. The easiest type of research to do is on their own courses because it is sometimes difficult to ask someone else for precious class time or student data. Biologists also have the most control in their own courses, which gives them more freedom in design.
Now that we have argued the advantages of biologists' conducting educational research, let us turn the tables a bit with a word of caution. Because biologists are so intimately connected to biology, they are in a position to develop the knowledge base of biology education in unique ways. However, the greatest mistake scientists make when they undertake educational research is to ignore the expertise from the field of education. Although it would be unfathomable for a researcher to begin a study of cell biology without first undertaking a study of what others have done in the area, scientists often conduct educational research, and even formally present results, without ever taking the time to find out what has been done in the past and to learn from others with related interests.
Past research in education has been successful. Many hard-working, intelligent, and talented people have worked to develop the knowledge base of education and the methods of obtaining this knowledge. Let us not underestimate what they have to offer, nor belittle their contributions. Instead, let us learn from them so that we can use our unique knowledge and skills to contribute to biology education.
The Appendix provides resources to be used as an initial gateway into the world of educational research. Following is a list of resources related to the professional activity of evaluating your own classroom.
Carnegie Academy for the Scholarship of Teaching and Learning (CASTL)
This program focuses on supporting and encouraging the scholarship of
teaching and learning. Through the CASTL web site, you will find numerous
thought-provoking and helpful resources:
www.carnegiefoundation.org/CASTL/.
Opening Lines: Approaches to the Scholarship of Teaching and Learning
Edited by P. Hutchings
(2000), this book is a
collection of eight essays. "Each of the eight authors tells the story
of her or his efforts at `opening lines' of inquiry into significant issues in
the teaching and learning of the field" (from book introduction).
Classroom Research: Implementing the Scholarship of Teaching
According to the publisher of this book by Cross and Steadman
(1996), "Classroom
Research details a collaborative process for investigating teaching and
learning issues. This technique engages teachers in problem-based discussions,
integrates their teaching experience with recent research and theory on
learning, and gives examples of Classroom Assessment and Classroom Research
projects that can be carried out in any classroom."
The Journal of Scholarship of Teaching and Learning (JoSoTL)
According to its mission statement, the JoSoTL is "designed to
encourage all instructors to engage in the discussion of the Scholarship of
Teaching and Learning (SoTL), and to become involved in the sharing of
knowledge and learning about the teaching-learning process." It
publishes classroom-based research as well as discussions related to the
scholarship of teaching and learning. Articles are available online at
www.iusb.edu/~josotl/.
|
METHODS OF EDUCATIONAL RESEARCH |
|---|
|
TOPABSTRACTINTRODUCTIONFORMAL EDUCATIONAL RESEARCH...WHY BIOLOGISTS SHOULD CONDUCT...METHODS OF EDUCATIONAL RESEARCHEXAMPLE OF RESEARCHRESOURCES FOR ADDITIONAL TOOLS...CONCLUSIONAPPENDIXREFERENCES |
|---|
Therefore how can research be conducted to give meaning to the results? It is impossible to thoroughly answer this question here. What follows is not intended to be a definitive guide to the educational researcher but rather an introduction to some of the most commonly used methods.
Quantitative versus Qualitative Research Methods
There are essentially two forms of educational research: quantitative (or
statistical) and qualitative. In the past, most research was quantitative in
nature, fashioned after the highly successful hard-science research methods.
Since the 1990s, educational researchers have embraced qualitative research
with the recognition that research on the human mind is fundamentally
different from research on physical systems. Qualitative methods are used for
depth of knowledge and quantitative methods are used for breadth and
generalizability. We cannot emphasize this enough: the best research designs
employ both methods.
Most readers of this article will be familiar and comfortable with the ideas behind quantitative research. Because of this, and because quantitative research methods are well documented (e.g., Hopkins, 1998), we will not attempt a discussion of statistical research methods here. However, many readers will be uncomfortable with qualitative research because it is so different from the formal training of a biologist. Nonetheless, it is a vital component of educational research and should not be overlooked. What follows is a brief discussion of the most useful and common qualitative research methods for education.
Qualitative Research
Statistical research is well suited for drawing generalizable and
repeatable conclusions based on a large sample of students, but it lacks
depth. For example, suppose you use a particular approach in your course that
you believe will lead to a greater understanding of transcription. You design
a test that measures student understanding and give it to numerous students
taught by using your approach and a traditional approach. You find that the
students taught under your new method significantly outperform their peers.
You come to the conclusion that your approach is superior. If your test and
research methods were well designed, your conclusion is valid. You know your
method is superior, but you have no data that tell you why the method
was so successful. You probably have hypotheses about why the method worked so
well (you had some reason to try it in the first place), but you do not know
for sure. To properly answer the question of why, you must engage in
qualitative research.
Qualitative research is also enormously helpful when you are initially investigating a particular area and are in the process of determining what questions might be interesting to study. The data of qualitative research are incredibly rich. Although this richness is an asset, it also makes qualitative data difficult and time consuming to analyze.
The National Science Foundation (NSF) has developed a detailed introduction to qualitative research geared toward the scientist. This introduction, User-Friendly Handbook for Mixed Method Evaluations, is excellent and can be found online at www.nsf.gov/pubsys/ods/getpub.cfm?nsf97153.
In the following subsections, we describe student interviews—a commonly used qualitative research tool. Most of what applies for interviews also applies for other types of qualitative research such as classroom observations and open-ended surveys. You are urged to read the NSF's handbook, mentioned above, for a more complete description of qualitative research.
Interviews. It is difficult to find out exactly what students are thinking. It would be useful if there were a probe that we could connect to the human brain that could give us information directly. Unfortunately, we must gauge thinking indirectly. The interview provides one mechanism through which we can begin to access students' thoughts, although it is certainly not foolproof. Most students are not particularly aware of their own thought processes and therefore have difficulty articulating them. They also have a tendency to filter out thoughts for various reasons. Interviews must be carefully structured to obtain the most information, and the results must be carefully analyzed.
Formal interviews are very different from the kind of casual interactions that occur between an instructor and a student. When a student comes to your office, your mission is to help the student. You often provide him or her with feedback and answer his or her questions. In an interview, your role is different. Your mission is to find out how students are thinking. You do not want to direct their answers, so you must refrain from providing feedback and may not correct their errors.
|
Interviews must be audiotaped and are often videotaped as well (see Figure 1). Such taping is particularly important when students must do something, such as interact with a computer or solve a problem, during the interview. When more than one student is being interviewed at a time, video recordings help determine the owner of each comment. In addition, video captures interesting behaviors such as hand gestures and facial expressions. The quality of audio recordings is always an issue, so be sure to use the best equipment you can obtain and test it before beginning the interview. The recording equipment may seem intrusive and makes most students nervous at first. However, once the interview is under way most people ignore the recording equipment if attention is not drawn to it.
Once the interviews are completed, they need to be transcribed. Transcription is a necessary but tedious task. In our experience, it takes about five times longer to transcribe an interview than it does to conduct it. Because of the long time involved, you may want to hire someone to do the transcription for you. Professional transcription services (whose main clientele are doctors and lawyers) can be expensive. Because transcription is easy to learn and you do not need 100% accuracy (you need to go over the tape yourself anyway), student workers are an adequate, less expensive alternative. If you choose to do the transcription yourself, investment in a transcription machine is a good idea (Figure 2). Such machines are easy to use and the ratio of transcribing time to interview time is closer to 8:1 without one. Voice-recognition software is another alternative, but at this point it does not offer any time savings over manual transcription.
|
|
How many interviews should you conduct and analyze? In statistical research, it is important to have a large sample. A statistical study reporting results based on 10 students would not be given much respect. Qualitative research is different. It is not practical to conduct and analyze 300 interviews, nor is it necessary. When you take statistical data from one student, you get only a small piece of information. In contrast, qualitative data are enormously rich. An hour-long interview with a student will provide you with more information than you will ever be able to fully analyze. There are excellent studies that were done with three or fewer participants. How many you need will depend on your particular project. If you plan to spend half an hour or so with individual students, probing them on their understandings of a narrow topic or their opinions, after about 10-15 interviews you will probably have covered most of the aspects of the topic. As you are doing the interviews, you will reach a point at which you feel as if there is nothing new to be found. At this point you probably have enough data.
Besides the general interview in which you might ask students their perceptions of the course or their learning, there are two specific types of interviews that can be very beneficial for science education: the think-aloud interview and the focus-group interview.
Think-Aloud Interviews. The think-aloud interview is a useful tool for discovering the thought processes of students as they tackle a cognitive task. The idea is to present the student with a task to complete and ask him or her to talk out loud about what he or she is thinking and doing. You should be able to get more information by using this method than you would if you simply asked the students to answer a question and then tell you their thought processes. If you ask students for their thoughts after the fact, they will usually give you a condensed version and leave out dead ends in their thinking. Often it is the process, not the final answer, that you are most interested in.
The think-aloud interview can be stressful for students who are not accustomed to sharing every inner thought they have. It is important to set them at ease at the beginning. This can be done by assuring them that the interview is confidential, that what they say will have no bearing on their grade in a course, and by allowing them to warm up. The warm-up is important because it allows them extra time to forget about the recording equipment and it gets them accustomed to speaking out loud. The warm-up question or task could be anything simple and easy. For example, you might ask them to determine the number of windows in their houses or to complete a very simple version of tasks that they will be given later. Remind them to talk out loud and to say everything they are thinking without trying to judge the correctness of their thoughts. There is some debate about whether or not the interviewer should remind students to keep talking if they fall into silence. Most students will focus on their own thinking and cease to speak at some point. We interject with "What are you thinking now?" to get them talking again. Others believe this approach distracts students. Do whatever seems appropriate to you as long as you mostly stay out of the students' way and do not lead them any more than is necessary.
Focus-Group Interviews. Focus-group interviews involve interviewing more than one student at a time. The dynamics of the interview is very different when multiple students are involved. For example, suppose you recently made major revisions to your course structure. You want to know how well you have done from the students' point of view. In this particular situation you will probably get more information if you interview students in groups. First, the students are likely to be more comfortable expressing criticism of your course when they are not alone in that expression. Second, the students will feed off one another, adding to what others have said and disagreeing at times. Consequently, the focus-group interview can be a powerful tool.
|
EXAMPLE OF RESEARCH |
|---|
|
TOPABSTRACTINTRODUCTIONFORMAL EDUCATIONAL RESEARCH...WHY BIOLOGISTS SHOULD CONDUCT...METHODS OF EDUCATIONAL RESEARCHEXAMPLE OF RESEARCHRESOURCES FOR ADDITIONAL TOOLS...CONCLUSIONAPPENDIXREFERENCES |
|---|
SCALE-UP (Student-Centered Activities for Large Enrollment University Physics; www.ncsu.edu/per/scaleup.htm)
Our first step in this curriculum-development effort was to list what we
were trying to accomplish with the new pedagogy. This list included not only
content-specific learning, but also building problem-solving skills and
opportunities for collaborative learning. Once you have clearly defined
objectives, you can devise ways to measure them. Because we had a wide-ranging
set of desired outcomes, we needed a variety of assessment methods.
To help us ascertain whether students were learning the content, we
utilized a bank of nationally normed conceptual tests. We gave these tests in
a pre/post mode so that we could measure learning from the improvement of
scores during the semester. The pretests were given before any instruction on
the topic was presented in the experimental class. The posttests were taken
after students had been taught the specific content, usually at the end of the
semester. The normalized gain score (Hake,
1998) is used to allow easy comparison of students who may have
underlying differences in preparation. It is the percentage of total possible
gain and is calculated from the following equation:
Classroom observations can be very helpful. The observer needs to remain "outside" the class in the sense that he or she cannot answer students' questions about the material but must instead focus on recording what is happening in the classroom. There are various ways to do this. You can create a protocol in which a "sweep" of the room is conducted at regular time intervals. Alternatively, the observer can focus on a single student or team, or on how the instructor manages the class. Even something as simple as playing a videotape at high speed provides a glimpse of how an instructor moves around the room and what kinds of discussions occur. It is important that someone other than the instructor be involved in qualitative research because students often respond to questions with what they believe the teacher wants to hear.
Careful observations were made of what was occurring in the classroom. After the semester's notes were completed, the approximately 300 pages were reviewed multiple times. An assortment of colored highlighting pens allowed reviewers to quickly categorize observations recorded on photocopies of the original notes. After each reading, the categories were revised as needed. In some cases categories were combined; other times they were split. The purpose of organizing the data this way was to facilitate further analysis. Once a useful scheme was imposed on the data, all the examples fitting into each category (with its own highlighting color) were collected for further review. We were interested in seeing changes that took place during the semester and how these changes were influenced by the instruction and the learning environment. The first and last sets of categories extracted from the data are listed in Table 2. Although they seem similar, the first set is more vague than the second. When creating categories, you must be careful not to make them so broad that they are meaningless or so narrow that they apply to only a few isolated instances in the data.
|
A great deal can be learned from this type of review. For example, most of the first set of observation categories could have been predicted. However, during our note review we were surprised at the frequency of praise coming from the instructor. We created a separate category for esteem building and decided that it was an important component of the learning environment. We followed up on this by reviewing and categorizing the notes left on the classwide electronic bulletin board. We were interested in communication among class members, so we had already planned to analyze the messages in the electronic bulletin board. Reiterating what we saw in the classroom observation notes, 13% of the messages from faculty to students contained some kind of positive reinforcement.
Additional qualitative investigations gave us insight into students' understanding as well as their feelings about the course. We interviewed students individually and in focus groups, as described earlier. We also collected portfolios of student work. Reviewing these documents can be interesting and is ongoing. For example, we are currently comparing group contracts (agreements made between collaborating team members to ensure success) to see what students have learned about how groups function. We are comparing contracts made at the beginning of the semester with those made halfway through, when new groups were assigned. A preliminary finding is that the contracts written after the students experienced group work for 7 weeks incorporated more detail as to how to handle problems that arise. For example, rather than simply stating: "Members will attend group meetings," students also listed consequences of nonattendance. This detail is something the students devised, and, we believe, clearly indicates their increasing ability to make groups function. The research methods utilized in the project continue to evolve as new ideas emerge from our analyses.
|
RESOURCES FOR ADDITIONAL TOOLS AND INSIGHTS |
|---|
|
TOPABSTRACTINTRODUCTIONFORMAL EDUCATIONAL RESEARCH...WHY BIOLOGISTS SHOULD CONDUCT...METHODS OF EDUCATIONAL RESEARCHEXAMPLE OF RESEARCHRESOURCES FOR ADDITIONAL TOOLS...CONCLUSIONAPPENDIXREFERENCES |
|---|
Field-tested Learning Assessment Guide (FLAG; www.wcer.wisc.edu/nise/cl1/flag/)
This web site describes itself as offering "broadly applicable,
self-contained modular classroom assessment techniques (CATs) and
discipline-specific tools for instructors interested in new approaches to
evaluating student learning, attitudes and performance." If you are
looking for a tool to use to assess a particular component of your course,
this is a wonderful place to start.
Classroom Assessment Techniques
This book by Angelo and Cross
(1993) provides an extensive
discussion of classroom assessment and offers many insights that would be
helpful to the teacher assessing his or her classroom. There are also
descriptions of a large number of techniques that can be used to gain
information about students and their learning. It is a valuable resource for
the classroom teacher.
|
CONCLUSION |
|---|
|
TOPABSTRACTINTRODUCTIONFORMAL EDUCATIONAL RESEARCH...WHY BIOLOGISTS SHOULD CONDUCT...METHODS OF EDUCATIONAL RESEARCHEXAMPLE OF RESEARCHRESOURCES FOR ADDITIONAL TOOLS...CONCLUSIONAPPENDIXREFERENCES |
|---|
|
APPENDIX |
|---|
|
TOPABSTRACTINTRODUCTIONFORMAL EDUCATIONAL RESEARCH...WHY BIOLOGISTS SHOULD CONDUCT...METHODS OF EDUCATIONAL RESEARCHEXAMPLE OF RESEARCHRESOURCES FOR ADDITIONAL TOOLS...CONCLUSIONAPPENDIXREFERENCES |
|---|
EDUCATIONAL RESEARCH RESOURCES
Journal Databases
ERIC and Education Abstracts are two searchable databases generally
provided online by your library. They are a great place to start but the
quality of what you will find will vary greatly because a variety of journals
are included in the databases. Below is a list of research journals that we
have found to be useful. It is not meant to be a complete list, just a
starting point.
REDCUBE (Research, Development, and Change in Undergraduate Biology Education; media4.physics.indiana.edu/~redcube/) REDCUBE is a web-based resource for biology education. According to Richard Hake, REDCUBE's author, it "contains 47 biology-educator profiles; 446 references (including 124 relevant to general science-education reform); and 490 hot-linked URL's on (a) Biology Associations, (b) Biology Teacher's Web Sites, (c) Scientific Societies and Projects (not confined to Biology), (d) Higher Education, (e) Cognitive Science and Psychology, (f) U.S. Government, and (g) Searches and Directories." Although he intended this resource to be for nonbiologists, it is so extensive that it is a valuable resource for biologists as well.
Listservs In recent years a large number of listservs dealing with educational issues have come into being. Most of them have searchable archives. Chances are that if you are interested in a particular issue, someone else has recently posted about it. If you do not find anything useful in the archives, you can post your own question. The listserv can be a great tool for finding out the current thought in an area. Publication in a journal is usually a few years behind. Following is a list of a few relevant listservs to get you started.
Web Search Engines It is amazing how much information can be pulled up on a computer in a matter of minutes. Most people have a web page now that describes their work. They generally provide related references, links, and downloads. A simple web search is always a great place to start.
TYPES OF EDUCATIONAL LITERATURE
As you make your way through the educational literature from the various
resources, be aware that two types of articles exist. There are those that are
written primarily for the teacher and those that are research reports. Those
for the teacher focus on how to implement a particular idea into the
classroom. They often lack the details that would allow an individual to make
a judgment regarding the quality of the idea. Many of these articles are
antecdotal and are based on teaching experiences rather than on rigorous
research.
Other articles are research articles. If an article is a research report, it should clearly describe the research method and may or may not be directly applicable to the classroom. The research method should be justified. Simply discussing a classroom innovation with students during office hours does not constitute research. Unfortunately, some of these "studies" are presented as research. Be critical of what you see by setting high standards for others as well as for yourself.
|
ACKNOWLEDGMENTS |
|---|
|
FOOTNOTES |
|---|
Corresponding author. E-mail address:
medancy{at}davidson.edu.
|
REFERENCES |
|---|
|
TOPABSTRACTINTRODUCTIONFORMAL EDUCATIONAL RESEARCH...WHY BIOLOGISTS SHOULD CONDUCT...METHODS OF EDUCATIONAL RESEARCHEXAMPLE OF RESEARCHRESOURCES FOR ADDITIONAL TOOLS...CONCLUSIONAPPENDIXREFERENCES |
|---|
Cross, K.P., and Steadman, M.H. (1996).Classroom Research: Implementing the Scholarship of Teaching , San Francisco: Jossey-Bass.
Hake, R.R. (1998). Interactive-engagement vs. traditional methods: a six-thousand-student survey of mechanics test data for introductory physics courses. Am. J. Phys. 66, 64.
Hopkins, K.D. (1998). Educational and Psychological Measurement and Evaluation, Boston: Allyn & Bacon.
Hutchings, P., ed. (2000). Opening Lines: Approaches to the Scholarship of Teaching and Learning. Menlo Park, CA: Carnegie Foundation for the Advancement of Teaching.
Titus, A.P. (1998). Integrating video and animation with physics problem solving: exercises on the World Wide Web. Ph.D. dissertation; North Carolina State University, Raleigh.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | ARCHIVE | SEARCH | TABLE OF CONTENTS |
