|
|
|||||||
ESSAY |
Department of Biological Sciences, Emporia State University, Emporia, Kansas 66801
Submitted March 29, 2002; Revised April 1, 2002; Accepted April 1, 2002
| ABSTRACT |
|---|
|
|
|---|
Key Words: assessment learning biology education
| INTRODUCTION |
|---|
|
|
|---|
The term "assessment" has a number of different meanings and connotations, but for the present purpose it may be defined as a systematic method to determine if, and to what extent, student learning has occurred. Unlike exams, whose purpose is to assign grades based on students' understanding, the purpose of assessment is to determine the impact of instruction on improving student learning. Although we are usually concerned with the end result of instruction, summative assessment, even more important are evaluations made during the course of instruction. Such formative assessment can guide changes as a course or program proceeds. Both formative and summative assessment can provide useful information about the efficacy of instruction, and today this kind of information is expected in reports and publications describing the activities (Stevens et al., 1993; Frechtling and Westat, 1997).
One of the concerns that must be addressed in assessment is whether it is important to have a control, and what type of control would be appropriate. As scientists, we are accustomed to evidence that fits the natural science model of controlled experiments where procedures can be replicated and measurements reproduced. However, evaluation of the resulting data is interpretative (in fact, a qualitative assessment) and progress in science is the result of debate over these interpretations. Course size and intent of the study also raise some important questions. For instance, in large multi-section courses it is relatively easy to match control and experimental groups, but to do so will require approvals by your campus Human Subjects Committee and individual student permission. With a small course an internal control may not be possible, although data from previous years (see "Database" below) could be used for comparison. In fact, as the literature in biology education accumulates, other studies can provide a baseline for comparison (see for instance, Hake, 2002, for an example from physics education).
There are many different models of assessment techniques, but they can be divided into two basic categories, quantitative and qualitative. Each has important advantages and serious limitations. Therefore, it is important to consider assessment from the beginning of the study so that appropriate tools can be used, as seamlessly as possible, to generate meaningful data. Finally, it is now generally accepted that multiple assessment measures are required to adequately gauge student learning (Siebert and McIntosh, 2001). One source of general assessment information is FLAG (Field-tested Learning Assessment Guide) available online at http://www.wcer.wisc.edu/nise/cl1. Some of the most commonly used assessment techniques are described below.
| QUANTITATIVE TECHNIQUES |
|---|
|
|
|---|
| PRETEST/POSTTEST |
|---|
|
|
|---|
Advantages
Perhaps the main advantage of this type of instrument is that it is most
similar to what we typically give as examinations, particularly in lower level
courses. This provides both a level of comfort and a sense of reliability to
the instructor. Because content is usually addressed, constructing these
tests, and analyzing the results are similar to what we already do to assign
student grades. The numerical scores can be analyzed statistically.
Disadvantages
That these assessments are similar to typical exams has a problem of
"familiarity breeds contempt" both for the instructor and the
student. The similarity to a typical exam makes it difficult to spend the time
necessary to construct questions that adequately address the goals of the
course. It also has the potential for student "abuse." How
reliable are the data being generated? To a large degree, this depends on the
character of the class and the rapport of the instructor. Pretests, especially
in first-semester courses, typically have a higher reliability simply because
of the naivete of the students. This looks like an exam and they tend to take
it seriously. Posttest reliability can be questionable when students know that
it will not affect their grade and a variety of attitudinal factors may
influence their performance. This problem is sometimes addressed by embedding
questions in a normal examination or by rewarding "gain" with
"bonus points" that can raise a course grade. (The trade-off in
the latter case is that students cannot remain anonymous, which therefore
necessitates approval by the Human Subjects Committee if you intend to publish
results.)
| QUESTIONNAIRES |
|---|
|
|
|---|
Advantages
The major advantage is the ease of obtaining information about student
reactions and student perceptions as a result of the course. Another advantage
is the ease of comparing quantitative scores from different sections or
different courses.
Disadvantages
A well-designed instrument will be constructed so that for some questions a
high score is desirable while for other questions the low score is preferred.
The scores must be standardized before comparisons can be made. It is also
important to use multiple questions to address each issue of concern to
establish validity. As with the content-based posttest, one must also be
concerned with how seriously students completed the questionnaire.
| DATABASE |
|---|
|
|
|---|
Advantages
Once established, a database can grow to provide a longitudinal record of
change. Ideally, a number of investigators at different institutions around
the country could be adding to a common database that could provide real power
to analysis of student learning. This has been done, for example, in the
physics community where the Force Concept Inventory is a widely adopted
assessment tool at two- and four-year institutions
(Hake, 2002).
Disadvantages
The strength of uniformity has the disadvantage of not being tailored to a
situation. Even in the same program, as a course or program evolves, it may be
desirable to modify the instrument to reflect goals attained and new
objectives that are defined.
| QUALITATIVE TECHNIQUES |
|---|
|
|
|---|
The two major problems with qualitative techniques are that, first, they tend to be time consuming, and second, most of us have no training in their use. There are several solutions to these problems. Perhaps you have a colleague in the behavioral or social sciences, trained in using qualitative assessment, who would be interested in collaboration. If funds are available, a specialist could be hired, or in some cases commercial instruments with professional readers could be employed. In many cases, the best choice will be to use some of the simpler tools outlined below to gather limited amounts of very specific information.
There are several ways qualitative data can be reported. For example, representative examples of student work could be presented from early, middle, and later portions of a course (Moncada, 1993). Alternatively, representative answers to reflective questions can be used to illustrate common strengths and weaknesses (Udovic et al., 2002). Student responses can often be categorized by type and reported in the form of a table or graph (see Fig. 1 in Wright et al., 1998).
| OBSERVATION |
|---|
|
|
|---|
Advantages
There is little or no imposition on students or the instructor as data are
collected during a class period. An added advantage is that instructor bias is
removed from the assessment.
Disadvantages
This form of assessment is useful only in student-active situations such as
laboratories or small classes where there is ample opportunity for individuals
or student groups to interact with each other and with the instructor. It also
requires that a qualified observer can be found, which in turn may be
expensive.
| INTERVIEWS |
|---|
|
|
|---|
One of the objections to involving the instructor in the interview process is the potential for introducing unintended bias into the assessment. The chemists at the University of Wisconsin (UW) devised a novel strategy to provide independent and unbiased assessment of student learning. Twentyfive faculty volunteers from math, engineering, and other science departments were asked to design their own 30-min oral examination over the chemistry course material. Each assessor interviewed approximately eight students (blindly assigned from two differently taught large lecture chemistry sections and octile ranking from the previous prerequisite course) and ranked the competence of the students they examined (Wright et al., 1998). Independently, researchers from the UWMadison LEAD Center (Learning through Evaluation, Assessment, and Dissemination) used qualitative sociological research methods to assess the same students. There were significant two- and three-way correlations between rank in prerequisite class, grade in this class, and volunteer assessor relative ranking.
Advantages
The primary advantage of this approach is that it allows you to test
thinking skills as opposed to content matter. It also allows you to follow
changes in student thinking during the progress of the course (see
introduction to qualitative assessment above). For the purpose of reporting
results, a major advantage is that it can be used effectively in situations
where there is no control group.
Disadvantages
The major disadvantage is the time commitment required from the
interviewer. Of necessity, this will require scheduling times outside the
normal class period.
| FOCUS GROUPS |
|---|
|
|
|---|
Advantages
Focus groups are good for identifying general patterns of student learning
within a class. Open discussion between students can uncover information
unanticipated by the instructor that can be very valuable in providing new
insights.
Disadvantages
First, there is the logistical problem of finding a suitable time when all
members of the focus group can meet. Second, a great deal of time and effort
goes into planning a successful session and analyzing the results. Finally,
this technique is not useful for uncovering details and specifics about
individual learning.
| CONCEPT MAPS |
|---|
|
|
|---|
Several different approaches have been used to evaluate concept maps. One approach is to establish knowledge categories ranging from common misconceptions about the subject to valid propositions. Initially, this may be based on a sample of 20 to 30 student maps that will become the baseline for a database (see above). Subsequently, individual student maps are matched to a category. Similarly, concept maps have been used to track developmental stage based on Perry's stages of intellectual development (Perry, 1970).
Advantages
One advantage of concept mapping over interviews is that maps can be
generated simultaneously by an entire class. Concept maps also provide a
permanent record of student understanding at a particular time, which is
useful to show changes in student understanding.
Disadvantages
The main disadvantages of using concept maps are that, first, instructors
must learn how to use and teach the technique, and second, students must be
taught how to construct them a process that can take up to a full class
period. In-class use of the technique takes up instructional time.
| JOURNAL WRITES, MINUTE PAPERS |
|---|
|
|
|---|
Advantages
Two advantages are the low impact on class time and the relative immediacy
of feedback to the instructor. The minute paper has the additional advantage
of providing an attendance check in large classes.
Disadvantages
The information provided is narrowly restricted to the specific question
asked and therefore focuses on only a small part of the material covered on
any single day.
| CONCLUSIONS |
|---|
|
|
|---|
Less natural for many of us is to accept that qualitative data can be as rich and informative as the statistical analyses we get from quantitative results. In fact, qualitative data are richer! Because quantitative assessment of necessity focuses on a few specific questions, but over a large number of students, it provides broad, generalized information about the class or program. Qualitative assessment, because of its open-ended nature, produces detailed information, but of a relatively few individuals. Quantitative assessment provides the broad strokes; qualitative assessment fills in the details. Both are needed to produce a good picture of student learning.
Corresponding author. E-mail address: sundberm{at}emporia.edu.
| REFERENCES |
|---|
|
|
|---|
Frechtling, J., and Westat, L.S., Eds. (1997).User-Friendly Handbook for Mixed Method Evaluations (NSF 97-153) , Washington, DC: National Science Foundation.
Hake, R.R. (2002). Lessons from the physics education reform effort. Conserv. Ecol. 5(2), 28. Available on-line at: http://www.consecol.org/vol5/iss2/art28
Johnson, D.W., Johnson, R.T., and Smith, K.A. (1991).Cooperative learning: increasing college faculty instructional productivity , ERICASHE Higher Education Report No. 4, Washington DC: George Washington University.
Mazur, E. (1997). Peer Instruction: A User's Manual, New York: Prentice-Hall. Available online at: http://galileo.harvard.edu
Mintzes, J.L., Wandersee, J.H., and Novak, J.D. (1999). Assessing Science Understanding: A Human Constructivist View, New York: Academic Press.
Moncada, G.J. (1993). Do college investigative laboratories really work? A descriptive study of an investigative biology laboratory in action. M.A. thesis, Louisiana State University, Baton Rouge.
Novak, J.D., and Gowin, D.B. (1984). Learning How to Learn. Cambridge, UK: Cambridge University Press.
Perry, W.G., Jr. (1970). Forms of Intellectual and Ethical Development in the College Years, New York: Holt, Rinehart, and Winston.
Siebert, E.D., and McIntosh, W.J. (2001).College Pathways to the Science Education Standards , Arlington, VA: NSTA Press.
Stevens, F., Lawrenz, F., and Sharp, L., (Frechtling, J., Ed.). (1993). User-Friendly Handbook for Project Evaluation: Science, Mathematics, Engineering and Technology Education (NSF 93-152), Washington, DC: National Science Foundation.
Sundberg, M.D. (1997). Assessing the effectiveness of an investigative laboratory to confront common misconceptions in life sciences. In McNeal, A.P., and D'Avanzo, C., eds., Student-Active Science: Models of Innovation in College Science Teaching, Orlando, FL: Harcourt Brace & Company.
Sundberg, M.D., and Moncada, G.J. (1994). Creating effective investigative laboratories for undergraduates.BioScience 44,698 -704.[CrossRef]
Tashiro, J., and Rowland, P.McD. (1997). What works: empirical approaches to restructuring courses in biology and environmental sciences. In McNeal, A.P., and D'Avanzo, C., eds., Student-Active Science: Models of Innovation in College Science Teaching, Orlando, FL: Harcourt Brace & Company.
Udovic, D., Morris, D., Dickman, A., Postlethwait, J., and Wetherwax, P. (2002). Workshop biology: demonstrating the effectiveness of active learning in an introductory biology course.BioScience 52,272 -281.[CrossRef]
Wright, J.C., Millar, S.B., Kosciuk, S.A., Penberthy, D.L., Williams, P.H., and Wampold, B.E. (1998). A novel strategy for assessing the effects of curriculum reform on student competence. J. Chem. Educ. 75,986 -992.
This article has been cited by other articles:
![]() |
W. B. Wood The Future of CBE-Life Sciences Education CBE Life Sci Educ, December 1, 2005; 4(4): 261 - 261. [Full Text] [PDF] |
||||
![]() |
R. Stevens, D. F. Johnson, and A. Soller Probabilities and Predictions: Modeling the Development of Scientific Problem-Solving Skills CBE Life Sci Educ, March 1, 2005; 4(1): 42 - 57. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. A. Felix, M. D. Hertle, J. G. Conley, L. B. Washington, and P. J. Bruns Assessing Precollege Science Education Outreach Initiatives: A Funder's Perspective CBE Life Sci Educ, September 1, 2004; 3(3): 189 - 195. [Abstract] [Full Text] [PDF] |
||||
![]() |
K. Tanner and D. Allen Approaches to Biology Teaching and Learning: From Assays to Assessments--On Collecting Evidence in Science Teaching CBE Life Sci Educ, June 1, 2004; 3(2): 69 - 74. [Full Text] [PDF] |
||||
![]() |
A. M. Campbell Public Access for Teaching Genomics, Proteomics, and Bioinformatics CBE Life Sci Educ, June 1, 2003; 2(2): 98 - 111. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. K. DebBurman Learning How Scientists Work: Experiential Research Projects to Promote Cell Biology Learning and Scientific Process Skills CBE Life Sci Educ, December 1, 2002; 1(4): 154 - 172. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | ARCHIVE | SEARCH | TABLE OF CONTENTS |