ASCB logo LSE Logo

Osmosis and Diffusion Conceptual Assessment

    Published Online:https://doi.org/10.1187/cbe.11-04-0038

    Abstract

    Biology student mastery regarding the mechanisms of diffusion and osmosis is difficult to achieve. To monitor comprehension of these processes among students at a large public university, we developed and validated an 18-item Osmosis and Diffusion Conceptual Assessment (ODCA). This assessment includes two-tiered items, some adopted or modified from the previously published Diffusion and Osmosis Diagnostic Test (DODT) and some newly developed items. The ODCA, a validated instrument containing fewer items than the DODT and emphasizing different content areas within the realm of osmosis and diffusion, better aligns with our curriculum. Creation of the ODCA involved removal of six DODT item pairs, modification of another six DODT item pairs, and development of three new item pairs addressing basic osmosis and diffusion concepts. Responses to ODCA items testing the same concepts as the DODT were remarkably similar to responses to the DODT collected from students 15 yr earlier, suggesting that student mastery regarding the mechanisms of diffusion and osmosis remains elusive.

    INTRODUCTION

    Fifteen years ago, Odom and Barrow (1995; Odom, 1995) created the Diffusion and Osmosis Diagnostic Test (DODT; available in Odom [1995] and Odom and Barrow [1995]), designed to assess secondary and college biology students’ understanding of osmosis and diffusion. Odom and Barrow reported that performance on the DODT by college biology majors was consistently poor, and scores earned by college non–biology majors and high school students were even lower (Odom, 1995). Although understanding how the fundamental processes of diffusion and osmosis work is essential to comprehending a wide range of biological functions, the research of Odom and Barrow and others has demonstrated that student mastery of osmosis and diffusion is extremely difficult to achieve. Inadequate understanding of osmosis and diffusion has been documented among high school and college students in the United States (e.g., Marek, 1986; Westbrook and Marek, 1991; Marek et al., 1994; Zuckerman, 1998; Christianson and Fisher, 1999) and elsewhere (e.g., Friedler et al., 1987; She, 2004).

    The DODT contains 12 two-tiered, multiple-choice questions. The first tier asks “What happens when …?” (“What?”), and requires students to analyze a situation and/or make a prediction about what will happen in a situation, given specific conditions. The second tier asks the reason (“Why?”) for the first-tier response. Item distracters (incorrect responses) were drawn from 20 prevalent misconceptions held by students, as identified by the DODT authors (Odom, 1995; Odom and Barrow, 1995).

    Twelve years later, Odom and Barrow (2007) examined responses and levels of certainty among 58 high school students who responded to the DODT after completing a week of instruction about osmosis and diffusion. Responses among subjects were remarkably similar to those obtained previously. Further, the authors assessed students’ confidence levels and found that students displayed high levels of confidence that their (incorrect) responses on the DODT were accurate. The extended time frame between studies (12 yr), the similarities in students’ responses across the years, and the students’ certainty about the correctness of their (incorrect) responses all illustrate the persistence and tenacity of students’ misconceptions (i.e., alternative ideas, naive ideas) regarding the processes of osmosis and diffusion.

    In this report, we use the original term, “misconception,” although we recognize that there are other, perhaps more appropriate, ways to refer to the scientifically inaccurate understandings that students have developed about natural phenomena (e.g., Smith et al., 1993; Chi, 2005). The misconception-based distracters in the DODT are valuable. They can reveal to instructors the presence of prevalent misconceptions among students in a given class. A wise instructor can effectively use that information to help students move from maintaining the identified misconceptions to applying accurate scientific reasoning.

    Our project's overarching goal was to develop several validated instruments that could be used for programmatic assessment in order to monitor learning at the college level, and specifically at our university. With these instruments, we examined biology students’ understandings of five fundamental topics, including 1) osmosis and diffusion, 2) natural selection, 3) cell division, 4) photosynthesis and respiration, and 5) nature of science.

    This paper describes our efforts to assess students’ understanding of the first topic area, osmosis and diffusion, by creating a modified version of the DODT that better corresponds to the instructional emphases on our campus, and that would fit within our programmatic assessment plan, in terms of both its overall length and the topics it included (described in Methods). The resulting modified Osmosis and Diffusion Conceptual Assessment (ODCA) was developed, evaluated, refined, and validated through successive administrations to students enrolled in several lower- and upper-division biology courses at a large public university across multiple semesters.

    In response to the growing interest in conceptual assessments in biology (e.g., D’Avanzo, 2008), we provide the completed ODCA (Supplemental Material) as another validated instrument that can be used to assess students’ comprehension (Chi and Roscoe, 2002) in order to improve both instructional effectiveness and curriculum design, as well as to aid programmatic assessment efforts.

    METHODS

    Development and Design

    The ODCA items were developed and modified over a period of several years through 1) administration of developing items each term to a subset of students in multiple classes, 2) reviews by expert faculty, and 3) interviews with both students and faculty about the test items. Incremental revisions were made between each administration of the evolving ODCA and were evaluated after subsequent ODCA administrations. In developing the ODCA, we attempted to focus on examining students’ reasoning about the biological mechanisms of osmosis and diffusion, which involves relatively high-level cognitive skills.

    To align with our programmatic learning goals, which emphasize different content areas within the realm of osmosis and diffusion than are addressed in the DODT, we began with the DODT, and deleted some items, modified others, and added items (Table 1; see Supplemental Material for ODCA items). In addition, we wanted to reduce the length of the instrument to encourage more faculty, even those teaching large college classes, to use it to inform their teaching, and to encourage students to pay close attention and read all items carefully and reflect on their answers. With our modifications, the validated ODCA is composed of eight paired items (16 questions), while the DODT has 12 paired items (24 questions). To use examples that are more relevant to essential curriculum in our state, we retained items relating to Odom and Barrow's (1995) categories of “kinetic energy of matter” and “membranes” (DODT items 7 and 12 [ODCA items 9/10 and 1/2], respectively). To shorten the test, we reduced the total number of items assessing concepts of “the process of diffusion,” “the particulate and random nature of matter,” and “the process of osmosis,” by omitting DODT items 1a,b; 3a,b; and 10a,b. Since concepts of “tonicity and concentrations” and “the influence of life forces on diffusion and osmosis” are not specifically addressed in our learning outcomes, we omitted those items to keep the ODCA brief (DODT items 4a,b and 9a,b, and items 11a,b, respectively). New items were added to the ODCA regarding diffusion of particles, and solute and solvent movement through a membrane (Tables 1 and 2). We also modified one item into a question about human cells, rather than plant cells, so it would be more appropriate for our human physiology courses, where students might not have strong preparation in plant biology and cell structure. We modified the language of many of the response choices during iterations of face validation by expert faculty to: 1) reduce the length of some response choices, especially when the correct choice was the longest option; 2) increase the number of responses that would allow us to diagnose students’ conceptual understanding of diffusion and osmosis, rather than testing students’ ability to define terms; and 3) attempt to reduce the number of concepts being tested in any one response choice by shortening some responses. Lastly, we reordered response choices to ensure correct responses were more evenly distributed among all response letters (a–d). These changes produced a shorter, validated test that addressed the processes and scientific reasoning we want our graduates to apply accurately.

    Table 1. Similar and unique items in DODTa and ODCAb

    Six DODT item pairs modified in ODCA
    Six DODT item pairs omitted from ODCADODTODCA3 New item pairs in ODCA
    1a,b2a,b3/411/12
    3a,b5a,b5/6c13/14
    4a,b6a,b15/1617/18
    9a,b7a,b9/10
    10a,b8a,b7/8
    11a,b12a,b1/2

    aOdom and Barrow (1995).

    bSimilar item pairs address the same topic and often use similar drawings, but the wording of the items differs to various degrees.

    cOnly one word was changed in 5/6.

    Table 2. Specification table summarizing seven scientifically correct ideas examined in the ODCAa

    Seven scientifically correct ideasResponse(s)
    Category A: Dissolving and solutions
      1. When a soluble substance is placed in water undisturbed, it will dissolve in water spontaneously, assuming its concentration is below the saturation point of the solution.5b
    Category B: Solute and solvent movement through a membrane
      2. Cell membranes are semipermeable in both directions, allowing some substances to pass through but not others.1a, 2d, 12a
      3. When separated by a semipermeable membrane through which only water can pass, water particles will move from the higher concentration of water (low solute concentration) into the lower concentration of water (high solute concentration). This can raise the water level on the side to which the water migrates.7a, 8c, 14d, 17a, 18c
      4. If water moves into an animal cell, the cell may swell and burst because water moves toward regions where there is more solute, so if a blood cell is placed in pure water, the cell will swell and eventually burst.13b
    Category C: Diffusion of particles
      5. Solutes and solvents move from higher to lower concentrations.3a, 5b, 6d, 11a
      6. Component particles of all phases of matter are moving; molecules continuously move due to Brownian motion; molecules become evenly distributed throughout their container, and continue to move.4b, 5b, 15b, 16c
      7. The higher the temperatures, the faster the rate of diffusion, all other things being equal, because the individual molecules are moving faster.9b, 10c

    aOrganized by category and expressed in the item responses identified in the second column.

    From interviews, we realized that some students responded to questions based on their recognition of specific biological terms and their recall of definitions—a low-level and often short-lived cognitive skill that sometimes led them to select incorrect responses reflecting inappropriate scientific reasoning. Thus, as much as possible, we avoided using scientific terminology in the ODCA to avoid triggering recall responses and to encourage thinking about mechanisms. We also eliminated or modified responses with low student response rates to improve the attractiveness of all distracters. Ultimately, six DODT item pairs were omitted, another six DODT item pairs were modified, and three new item pairs were created as a result of the revision process (Table 1). The two-tiered ODCA items are numbered consecutively (instead of using 1a and 1b, as Odom and Barrow did).

    In creating the ODCA, we did not aim to identify new misconceptions held by students. Rather, like the DODT, the ODCA assesses 1) students’ abilities to identify scientifically accurate interpretations of osmosis and diffusion events and 2) their attraction to well-known misconceptions regarding the processes of osmosis and diffusion.

    Specification Tables

    Specification tables were developed and modified along with the ODCA. These tables were used to track the scientifically correct ideas (Table 2) and misconceptions (Table 3) captured in the various item responses to allow comparison of students’ responses to similar ideas in different contexts. The tables are similar in construct to those used in the development of the Conceptual Inventory of Natural Selection (CINS; Anderson et al., 2002; Anderson, 2003) and to the lists of propositions and misconceptions identified in the development of the DODT (Odom, 1995; Odom and Barrow, 1995). ODCA concepts are organized into three categories (Tables 2 and 3): “Dissolving and solutions,” “Solute and solvent movement through a membrane,” and “Diffusion of particles,” and the item responses associated with those concepts are shown in the right columns of Tables 2 and 3. Table 2 lists the seven scientifically correct ideas examined in the ODCA and Table 3 summarizes the 20 misconceptions or other types of errors presented in the ODCA alternative responses.

    Table 3. Specification table that lists 20 prevalent misconceptions and other common errors, organized by category

    Twenty misconceptions and other errorsODCA responses
    Category A: Dissolving and solutions
      1. Dissolved substances will eventually settle out of solution.5a, 6a, 6c, 16b
      2. Solute particles will not dissolve in water spontaneously.6b
      3. Conflation of solvent and solute concentration and their interactions with solutions.7c, 17b, 17c, 18b
    Category B: Solute and solvent movement through a membrane
      4. Semipermeability is unidirectional.2b
      5. Only beneficial materials pass through a cell membrane.2c
      6. All small materials can pass through a cell membrane.1b, 2a
      7. Water moves from high to low solute concentration.8a, 13a, 14a
      8. Solutes do not cross a semipermeable membrane.11c
      9. When solutes in water-based solutions cross a semi-permeable membrane, they can alter the heights of the liquids on each side; movement of water particles across the membrane does not warrant consideration.7b, 11b, 12c
      10. In a system open to atmospheric pressure, water always equilibrates the levels on both sides of a membrane.8b, 12d, 18a
      11a. An animal cell can maintain itself in any environment.13c, 14b
      11b. An animal cell cannot survive outside the body.14c
      12. At a given temperature, solute and solvent particles move across a membrane at different speeds.12b
    Category C: Diffusion of particles
      13. Solvent particles will move from lower concentration of that solvent to higher concentration of that solvent.3b, 7b, 8d
      14. Coolness facilitates movement.9a, 10b
      15. Dye tends to break down in solution.10a
      16. As the temperature changes, individual particles (atoms, molecules, ions) expand or contract.10d
      17. Particles move through solutions or gases only until they are evenly distributed, then they stop.4c, 15a, 16a
      18. Particles move by repelling one another.4d
      19. Particles actively seek (want) isolation or more room.4a
      20. Particles in gases and liquids are moving; particles in solids are not.16d

    Initial Refinement and Validation

    As noted previously, initial versions of the ODCA were administered to biology students for several semesters to improve instrument validity. During this period, incremental changes were made to increase clarity and effectiveness of items, and ensure that all responses were attractive to some students. Face validation was obtained when 56 biology instructors taking part in scoring the AP Biology Examination voluntarily completed the ODCA. Of those who responded, 33 taught biology at the high school level and 23 taught biology at the community college or university level (San Diego State University [SDSU] Institutional Review Board [IRB] Approval 113073). Each of the participants stated that they had taught osmosis and diffusion to students in the past year. In addition to taking the ODCA, these instructors were asked to provide feedback concerning the wording of the instrument items and questions, and this feedback was taken into consideration during development of the conceptual assessment. The instructors were also invited to describe the specific areas in which their students tend to experience the most difficulty when trying to comprehend diffusion and osmosis.

    In addition, 16 undergraduate students (10 biology majors, four kinesiology majors, and two nursing majors) were invited to participate in semistructured interviews (Bernard, 1988). The interviews focused on students’ thinking about osmosis and diffusion and were conducted by two graduate students. Each interviewed student was asked to read ODCA items aloud and provide explanations for why he or she selected or rejected each response. Students were also encouraged to suggest an alternative response if they were not satisfied with the fixed choices offered. Additionally, the interviewer asked students to respond to follow-up probes designed to further explore their reasons for answering the way they did. Manipulatives, demonstrations, and/or drawings were occasionally used to elicit additional information regarding participants’ ideas about osmosis and diffusion.

    To determine construct validity, we calculated the standard difficulty index (p: proportion of students who answered a test item correctly) and discrimination index (d: index that refers to how well an item differentiates between high and low scorers), and evaluated the instrument's reliability with Cronbach's alpha (Cronbach, 1951), using PASW Statistics 17, Release Version 17.0.2 (IBM SPSS Statistics [2009], Chicago, IL). Values from the ODCA were then compared with those reported for the DODT (Odom, 1995; Odom and Barrow, 1995).

    ODCA Administration

    Once ODCA design was finalized, student performance was assessed in four successive semesters during two academic years, 2007–2008 and 2008–2009 (Table 4; IRB Approval 278). This study compares the performance of three groups of university students: upper-division biology majors, lower-division biology majors, and lower-division allied health majors. The upper-division biology majors were students enrolled in four upper-division biology courses (“core” courses of the curriculum) taken by most biology majors. The lower-division allied health majors were non–biology majors enrolled in a lower-division microbiology course. These two groups were surveyed for four semesters. The lower-division biology majors consisted of students enrolled in a lower-division introductory biology course required for all biology majors; they were surveyed during the 2008–2009 year.

    Table 4. Number of upper-division biology majors, lower-division biology majors, and nonmajors invited to complete the ODCA online via Blackboarda

    UD majorsLD majorsNM
    Semester and yearInvitedTestedInvitedTestedInvitedTested
    Fall 2007 6840 (59%) 5033 (66%)
    Spring 2008 6439 (61%) 4326 (60%)
    Fall 2008 6644 (67%)10245 (44%) 4626 (57%)
    Spring 2009 8052 (64%)10162 (61%) 5042 (84%)
    Total278174 (63%)203107 (53%)189127 (67%)

    aNumber and percentage of students who actually completed the ODCA are broken out by semester. UD, upper-division biology majors; LD, lower-division biology majors; NM, nonmajors.

    A random subset of students in each course was invited each semester to complete the ODCA, using the course management system, Blackboard Academic Suite. Students were asked to complete the ODCA during a 10-d window after course registration ended and enrollments were finalized. Students were encouraged to complete the ODCA as their contribution to the biology department's annual programmatic assessment effort; three participation points could be earned by completing a “survey.” We used the term “survey” to distinguish the ODCA and other conceptual assessments from the graded tests taken by students. Approximately 20% of the students enrolled in each of the courses were randomly invited to complete the ODCA each semester (Table 4). The remaining students received one of four other conceptual assessments on different topics examined for programmatic assessment. None of the courses directly addressed topics of osmosis and diffusion before the ODCA was administered, as these processes were taught explicitly in previous courses.

    RESULTS

    Variation among Courses

    Of the 670 students invited to complete the ODCA over the four semesters, 408 (61%) responded and gave consent to use their data (Table 4). Within each level (upper-division biology major, lower-division biology major, and nonmajor), there was considerable within-class variation in student performance. Consequently, no significant differences were found in student performance across semesters within the upper-division courses, lower-division courses, or nonmajor courses (two-way analysis of variance (ANOVA), P > 0.05). Thus, for each level (upper division, lower division, nonmajor), results from all semesters were combined for these analyses.

    Difficulty and Discrimination Indices

    Difficulty indices (p) of the items ranged from 0.27 to 0.98 (Table 5), with a mean of 0.65, providing a wide range of item difficulty, similar to values reported for the DODT (Odom and Barrow, 1995). The discrimination indices (d) ranged from 0.07 to 0.67, with a mean of 0.44. The discrimination index refers to how well an item differentiates between high and low scorers; it is a basic measure of the validity of an item. On ordinary tests, a low or negative discrimination index value generally indicates that the item does not measure what other items on the instrument are measuring. However, in the case of the paired items on this instrument, the first-tier “What happens?” item is often quite easy, and provides the necessary context for the more difficult second-tier “Why does this happen?” item. This can produce a low discrimination index for some of the first (odd-numbered) items, but a high discrimination index for the paired (even-numbered) items. For example, three odd-numbered ODCA “What?” items had discrimination indices of 0.07, 0.08, and 0.16—below the value of 0.20 that is typically considered as a minimum (Odom and Barrow, 1995), and the subsequent even-numbered “Why?” items had discrimination values over 0.20. Thus, on this two-tiered assessment, a correct answer on the first-tier item (content) did not predict performance on the second-tier item (reason). For purposes of this project, all ODCA test items were considered acceptable, since the item pair (and particularly the “Why?” response) is the element of interest.

    Table 5. Number of ODCA items within each range of difficulty and discriminationa

    Difficulty (No. of items)Discrimination (No. of items)
    0.20–0.29 (1)0.00–0.09 (2)
    0.30–0.39 (1)0.10–0.19 (1)
    0.40–0.49 (3)0.20–0.29 (2)
    0.50–0.59 (2)0.30–0.39 (1)
    0.60–0.69 (5)0.40–0.49 (2)
    0.70–0.79 (1)0.50–0.59 (7)
    0.80–0.89 (2)0.60–0.69 (3)
    0.90–0.99 (3)

    a See text for further details.

    For evaluating the difficulty of an item, it is helpful to consider the likelihood of guessing the correct answer. For a multiple-choice item with four possible responses, there is a 25% chance of guessing the correct response. A two-tier item with two selections for the first tier and four selections for the second tier provides a 12.5% chance of guessing the correct answer combination. In the ODCA, there are two or three response options in the first tier, and three or four in the second tier. This results in chances of guessing correct answers at 8.3% to 12.5%.

    For the first tier of the ODCA, the percentage of students responding with correct answers ranged from means of 48.3% to 97.8% for upper-division biology major classes, 45.0% to 95.4% for lower-division biology major classes, and 35.6% to 97.3% for nonmajor classes (Table 6 and Figure 1). The mean percentage of combined content–reason responses on the ODCA that were correct ranged from 28.2% to 92.3% for upper-division biology major classes, 19.9% to 74.7% for lower-division biology majors, and 16.8% to 85.3% for nonmajors. Results from college students taking the DODT and ODCA were strikingly similar (Table 6), especially for the highest and lowest scoring-item combinations (Figure 1).

    Figure 1.

    Figure 1. Percentages of college students who selected the correct content response (pale bars) and the correct content plus reason combination (darker bars) on (A) ODCA items 3 and 4 compared with DODT items 2a,b, and on ODCA items 15 and 16 compared with DODT items 6a,b, and (B) ODCA items 5 and 6 compared to DODT items 5a,b, and ODCA items 7 and 8 compared with DODT 8a,b. ODCA items are shown in green, while DODT items are shown in blue. NM, nonmajors; LD, lower-division biology majors; UD, upper-division biology majors; Values represent means + 1 SE. Sample sizes appear in Table 4.

    Table 6. Percentages of college biology students who selected the correct tier 1 “What?” (content) responses and the first- and second-tier combined content–reason (“Comb”) responses on the ODCA and the DODT

    Non–biology majorsBiology majors
    ODCA Item no. (DODT no.)DODTDODTODCAODCALD DODTLD DODTLD ODCALD ODCAUD ODCAUD ODCA
    Item pairTier 1CombTier 1CombTier 1CombTier 1CombTier 1Comb
    1/2 (12)95.181.397.385.395.792.395.474.797.892.3
    3/4 (2)90.22294.224.889.723.190.723.097.343.6
    5/6 (5)37.434.144.239.537.636.845.043.148.345.8
    7/8 (8)51.214.635.616.864.128.246.319.949.328.2
    9/10 (7)97.695.989.650.09492.384.865.295.072.3
    11/1255.334.158.044.674.058.0
    13/1464.847.662.246.177.558.4
    15/16 (6)88.665.984.464.19476.188.858.391.772.3
    17/1868.158.663.152.863.153.7

    a The first column shows the ODCA item pairs and the corresponding DODT item numbers in parentheses. UD, upper-division courses; LD, lower-division courses. ODCA results, in bold, represent the student performance across all courses and semesters, by course level. ODCA sample sizes appear in Table 4.

    Test reliability was at least 0.70 each semester using Cronbach's alpha calculation (Fall 2007: 0.70; Spring 2008: 0.74; Fall 2008: 0.73; Spring 2009: 0.70), which is considered to be an acceptable level. Test completion time ranged from 5 to 20 min per student and was not correlated with performance (r2 > 0.05; n = 71, 72, and 83 for three semesters, Fall 2007–Fall 2008).

    Expert Performance

    We found no difference between high school and college instructors’ performance on the ODCA; thus, all 56 instructors were grouped together for data analysis. As expected, these biology experts performed exceedingly well on the ODCA, earning a mean score of 16.5 out of the total possible 18 (SE = 0.2). These data provide evidence that the test items accurately represent the processes of osmosis and diffusion.

    Student Performance

    The most striking result is the overall consistency of students’ response patterns across levels, semesters, and years (Figure 2). Students generally performed better on first-tier “What?” items (odd numbers) than on second-tier reason “Why?” items (even numbers) on both the ODCA and the DODT (Figures 1 and 2 and Table 6), indicating that they often can predict the outcome but have less understanding about the underlying mechanisms.

    Figure 2A.

    Figure 2A. Radar graph shows the percentage of nonmajors (NM), by course and by semester, who selected the correct response for each item on the ODCA. Items are grouped into the three conceptual categories described in the text. Note the similarity of performance across semesters. S, spring semester; F, fall semester. Sample sizes appear in Table 4.

    Figure 2B.

    Figure 2B. Radar graph shows the percentage of lower-division biology majors (LD), by course and by semester, who selected the correct response for each item on the ODCA. Items are grouped into the three conceptual categories described in the text. Note the similarity of performance across semesters. S, spring semester; F, fall semester. Sample sizes appear in Table 4.

    Figure 2C.

    Figure 2C. Radar graph shows the percentage of upper-division biology majors (UD), by course and by semester, who selected the correct response for each item on the ODCA. Items are grouped into the three conceptual categories described in the text. Note the similarity of performance across semesters. S, spring semester; F, fall semester. Sample sizes appear in Table 4.

    Of the nine ODCA question pairs, 3/4, 5/6, and 7/8 yielded the lowest combined content–reason values (Figure 1 and Table 6), and thus reflected the most prevalent misconceptions among our students. Excerpts from these three item pairs are shown in Table 7, along with frequencies of student responses and variance among semesters in parentheses.

    More than 90% of all students correctly indicated that particles (dissolved substances) would move from areas of high to low concentration (response 3a; Figure 1A and Table 6). However, when asked to provide a reason for their answer, many faltered (Table 6). The correct combined content–reason ODCA responses, 3a and 4b, were selected by < 25% of nonbiology majors and lower-division biology students (Table 6). More than one-fourth (27–44%) of students at each course level chose response 4a, “crowded particles want to move to an area with more room” (Table 3, misconception #19; Table 7). That is, a substantial proportion of our undergraduates attributed anthropomorphic qualities to substances in place of scientifically accurate alternatives. Interviews corroborated these tendencies. When an interviewee was asked why she selected 4a (“… crowded particles want to move …”) rather than 4b (“the random motion of particles suspended in a fluid results in their uniform distribution”), she stated that 4a seemed more understandable and consistent with her conceptualization of the diffusion process. “I just think of diffusion as…there's bunch of particles here and there aren't any there, so it's crowded. So [the particles] just move to the other side because…they want to even out” (italicized word indicates emphasis by the speaker). When asked why she didn't choose answer 4b, the student replied that while she knew that the particles moved randomly, she didn't quite understand how such motion would result in uniform distribution.

    Table 7. Misconceptions chosen by at least 10% of students are identified below for non–biology majors, lower-division biology majors, and upper-division biology majorsa

    Prevalent misconceptionsNMLDUD
    Category C: Diffusion of particlesb
      3a: “During the process of diffusion, particles will generally move from an area of (a) high to low particle concentration …”c
      4a. because crowded particles want to move to an area with more room.d0.31 (0.05)0.44 (0.01)0.27 (0.06)
      4c. because the particles tend to keep moving until they are uniformly distributed and then they stop moving.d0.37 (0.02)0.24 (0.02)0.23 (0.02)
    Category A: Dissolving and solutionb
      5a: “If a small amount of table salt (1 tsp) is added to a large container of water) and allowed to set for several days without stirring … the salt molecules will (a) be more concentrated on the bottom of the water …”c
      6a. because salt is heavier than water and will sink.d0.25 (0.04)0.24 (0.07)0.28 (0.02)
      6c. because there will be more time for settling.d0.10 (0.02)0.24 (0.07)0.28 (0.02)
    Category B: Solute and solvent movement through a membraneb
      7a: “In Figure 1, … the water level in Side 1 will be higher than in Side 2 …”c
      8a. because water will move from high to low solute concentration.d0.11 (0.04)0.19 (0.02)0.12 (0.02)
      7b: “In Figure 1;, … the water level in Side 1 will be lower than in Side 2 …”c
      8a. because water will move from high to low solute concentration.d0.19 (0.04)0.12 (0.02)0.09 (0.02)
      7c: “In Figure 1, … the water level in Side 1 will be the same height as in Side 2 …”c
      8b. because water flows freely and maintains equal levels on both sides.d0.29 (0.07)0.27 (0.06)0.26 (0.04)

    a Values represent proportions of students who selected combined content–reason responses, averaged across four semesters for nonmajors (NM) and upper-division biology majors (UD), and across two semesters for lower-division biology majors (LD). Frequencies represent mean proportions of students selecting that response by course, with variance among semesters (SE) shown in parentheses. Sample sizes appear in Table 4.

    bConceptual knowledge categories of misconceptions (from Tables 1 and 2).

    cFirst-tier (odd-numbered) item stem.

    dCommonly chosen (incorrect) second-tier (even-numbered) responses, along with response frequencies for each student group.

    Surveyed biology instructors similarly identified anthropomorphism as a problem for students. For example, a college instructor with 10 years of experience noted that students think molecules “want to diffuse.” Another instructor suggested that it is difficult for students to eliminate the ideas of “‘vitalism’ and ‘volition’ on the part of the particles.”

    One-fourth of our students indicated that particles tend to keep moving until they are uniformly distributed and then they stop moving (response 4c). Some of the students’ follow-up statements during interviews seemed to suggest that the students conceived of particles moving around only if there was a need to do so (e.g., a concentration gradient is present). When there was not such a need, they thought that particles would stop moving and remain stationary. Surveyed biology instructors offered similar observations. For example, a high school instructor with 20 years of experience stated: “[Students have trouble with the idea] that molecules have constant motion which can vary with conditions (temperature, pressure, etc.).” Similarly, a college instructor with 15 years of experience noted that, “[Students exhibit] no prior knowledge of kinetic energy/Brownian movement.”

    Regarding question pair 5 and 6 (Figure 1B; and Table 7), illustrating misconception 1 (Table 3), more than 50% of all students (including upper-division biology majors) in most semesters indicated that if a small amount of salt is added to a large volume of water and then allowed to sit still for several days, salt molecules would be concentrated at the bottom of the container. In some semesters, more than 50% of students chose this content response (Figure 2 and Table 6). Of those students, 24–28% chose reason 6a, asserting that salt would sink, because salt is heavier than water, and 10–28% chose reason 6c, indicating that salt would sink because there will be more time for settling (Table 7).

    In interviews with test takers, students who selected content response 5a combined with reason response 6a or 6c were confident that, given enough time, solute particles would be concentrated on the bottom. For example, when asked why she selected her response, one senior biology major stated, “I do think that salt is heavier…When you put it in a container, it goes to the bottom.” Interviewees stated that this would be the case both for solid solutes (e.g., salt and sugar) and for liquid solutes (e.g., food coloring or dye). Thus, in the minds of many undergraduate students, even after considerable training in biology, gravity appears to be a major factor affecting dissolved substances, even when only a small amount of solute is introduced into a solvent.

    Item pair 7/8 (Figure 1B, Table 7, and Supplemental Material) presents an illustration and description of a container divided by a semipermeable membrane, with dye plus water on the left and pure water on the right. The problem states that only water can move through the membrane; the dye cannot pass through. In some semesters, half of our students selected content response 7c, indicating that water will maintain at equal levels on both sides of the membrane. Of those selecting response 7c, 26–29% of students across all semesters choose the reason response 8b, that water flows freely and maintains equal levels on both sides of the membrane (Table 7).

    Among upper-division students who were interviewed, those who selected responses 7c and 8b indicated that atmospheric pressure overrides water's ability to move through the membrane during osmosis. In the words of a senior biology major, “I’m assuming that water's not going to rush in and…go against pressure to…dilute the dye side of things…Atmospheric pressure is going to override the system, as far as water's desire to dilute that.” Thus, students apparently have difficulty weighing and integrating the different factors affecting the movement of molecules in solution. Their intuition about the effects of atmospheric pressure tends to override the less familiar concept of osmotic pressure.

    About 10–20% of students across course levels who thought that water levels would be different (correct content response, 7a: “Side 1 is higher than Side 2”) chose reason 8a (incorrect): “Water will move from high to low solute concentration.” These results indicate a major lack of understanding of the basic principles of osmosis and diffusion.

    DISCUSSION

    Comprehending Osmosis and Diffusion

    The data collected from students and biology experts indicate that the ODCA is a valid and reliable tool for assessing students’ understandings of osmosis and diffusion. The consistent response patterns exhibited by random samples of three student groups (nonmajors, lower-division biology majors, and upper-division biology majors), sampled across four successive semesters, reveal that students continue to have difficulty with reasoning scientifically about the basic processes of osmosis and diffusion, topics typically taught in high school biology.

    Although the ODCA items are somewhat different from the DODT items, the response patterns from our 2007–2009 student groups were very similar to those seen among the midwestern students studied in 1995 and 2007 (Odom, 1995; Odom and Barrow, 1995, 2007). Only a fraction of college students in the studies fully understood the mechanisms of diffusion and osmosis, reaffirming our initial claim that student mastery of those concepts is difficult to achieve. Furthermore, biology majors in this study did not perform better than nonmajors. This was not surprising, given that others have shown that performance of nonmajors in biology courses can surpass the performance of majors in terms of content comprehension (Sundberg and Dini, 1993; Sundberg et al., 1994).

    The challenge remains: Why is it so difficult to successfully teach and learn about osmosis and diffusion? Part of the challenge may be due to the fact that these processes result from the constant, random motion of invisible particles, and a significant fraction of students struggle to comprehend such abstract ideas. Indeed, Garvin-Doxas and Klymkowsky (2008) found that while students understood there to be a random component to biological processes (e.g., diffusion), students were unable to link this “randomness” to emergent systematic behaviors (e.g., net movement of particles through a membrane). Another obstacle to students’ developing a deeper understanding of diffusion and osmosis may be that instructors still tend to rely on more traditional, didactic-style methods of teaching these abstract concepts.

    The lecture method has prevailed in college classrooms for decades. Yet there is a growing body of research suggesting that successful scientific understanding is rarely achieved through lecture teaching and passive student listening (e.g., Donovan et al., 1999; National Research Council, 2000; Brewer et al., 2011). Instructional methods that involve dialogue with fellow students (Lemke, 1990), active learning (Ebert-May et al., 1997), problem solving (Mintzes et al., 1998), animation (Sanger et al., 2001), and/or modeling knowledge (Fisher et al., 2000; Odom and Kelly, 2001; Tekkaya, 2003) may lead to significant improvements in understanding (Freeman et al., 2007). Furthermore, research has shown that when instructors employ formative assessment techniques to gather data regarding their students’ knowledge as they teach, they are more capable of designing additional instruction to move their students toward deeper understanding of the material (Black and Wiliam, 1998).

    Conceptual Assessments

    There are many ways to use reliable, validated conceptual assessments such as the ODCA and the DODT. Many (but not all) conceptual assessments, including the DODT and the ODCA, employ students’ prevalent misconceptions as distracters. When such a conceptual assessment is administered at the beginning of a course, it can provide valuable insight to the instructor regarding students’ prior knowledge and beliefs. When an instructor can build from students’ initial conceptions and demonstrate why or how scientific reasoning is better able to explain observations, students may be more prepared to grasp the scientific concepts.

    The project described here focused on creating an appropriate, validated conceptual assessment and establishing baseline data for an academic program. Once available, there are many ways to use such reliable, validated conceptual assessments, and two strategies are described below.

    The first obvious use of Conceptual Assessments in Biology is to assess pre- to postlearning gains within a course or curriculum. When students complete a conceptual assessment at both the beginning and end of a course, it is possible to assess learning gains (or losses) within a single semester or to compare different groups of students across semesters to improve course and curriculum designs. This approach is exemplified with the use of CINS (Anderson et al., 2002; Anderson, 2003), and three other genetics assessments: the Genetics Literacy Assessment Instrument (Bowling et al., 2008a,b), the Genetics Concept Assessment (Smith et al., 2008), and the Genetics Two-Tier Diagnostic Instrument (Tsui and Treagust, 2009). This strategy was also used with the Introductory Molecular and Cell Biology Assessment (Shi et al., 2010), the Photosynthesis and Respiration Assessment (Haslam and Treagust, 1987), and Energy and Matter Diagnostic Question Clusters (D’Avanzo et al. 2010; Hartley et al., 2011), as well as the DODT (Odom, 1995; Odom and Barrow, 1995; Christianson and Fisher, 1999). Several authors have demonstrated the use of conceptual assessments for broader programmatic improvement, including Garvin-Doxas and Klymkowsky (2008) using the Biology Conceptual Inventory, Marbach-Ad et al., (2009, 2010) using the Host Pathogen Interactions Concept Inventory, and Wilson et al. (2007) using the Energy and Matter Diagnostic Question Clusters.

    When conceptual assessment tools are not used for assessing instruction or measuring student learning gains, instructors may use individual items from those assessments as “concept questions” to generate in-class discussions and facilitate biology learning, as encouraged by Klymkowsky et al. (2003), D’Avanzo (2008), and Shi et al. (2010). This method has been used effectively for promoting learning of a variety of topics, including, for example, physics (Lawson, 1988), geoscience (McConnell et al., 2006), and biology (Meltzer and Manivannan, 2002; Tessier, 2004; Tanner and Allen, 2005; Michael, 2006). Thus, conceptual assessments in biology are resources that allow instructors to plan teaching strategies and help their students to move toward more scientifically appropriate reasoning (Sundberg, 2002).

    Validated conceptual assessments such as the ODCA, the DODT, and others (D’Avanzo, 2008; Fisher and Williams, 2011), hold the potential for helping students gain the capacity to talk and think using scientifically principled reasoning, and prepare for solving problems of the twenty-first century (Wood, 2009; Adams and Wieman, 2011). These easily administered instruments afford biology faculty data they can utilize to generate instruction tailored to the current conceptual understandings of their students. When instructors are aware of their students’ misunderstandings (Smith and Tanner, 2010) and utilize interactive instructional strategies (Woodin et al., 2009, 2010; Brewer et al., 2011), students’ comprehension of abstract scientific concepts can be improved.

    ACKNOWLEDGMENTS

    We are grateful to the students who voluntarily participated in our study. We thank Janessa Molinari-Gruby for her assistance in interviewing students. We thank the SDSU College of Sciences and Department of Biology for supporting our project. We are grateful to the SDSU Instructional Technology Services staff for their help with online deployment of the ODCA. We thank anonymous reviewers for helpful comments of an earlier version of this report. We are also deeply indebted to Louis Odom and L. H. Barrow, on whose shoulders we stand, for their thorough explication of students’ misconceptions about osmosis and diffusion and for their pioneering creation of the DODT.

    REFERENCES

  • Adams WK, Wieman CE (2011). Development and validation of instruments to measure learning of expert-like thinking. Int J Sci Educ 33, 1289-1312. Google Scholar
  • Anderson DL (2003). Natural selection theory in non-majors biology: instruction, assessment, and conceptual difficulty PhD Dissertation San Diego and San Diego State University, San Diego, CA University of California. Google Scholar
  • Anderson DL, Fisher KM, Norman GJ (2002). Development and evaluation of the conceptual inventory of natural selection. J Res Sci Teach 39, 952-978. Google Scholar
  • Bernard HR (1988). Research Methods in Cultural Anthropology, Newbury Park, CA: Sage. Google Scholar
  • Black P, Wiliam D (1998). Inside the black box: raising standards through classroom assessment. Phi Delta Kappan 80, 139-148. Google Scholar
  • Bowling BV, Acra EE, Wang L, Myers MF, Dean GE, Markle GC, Moskalik CL, Huether CA (2008a). Development and evaluation of a genetics literacy assessment instrument for undergraduates. Genetics 178, 15-22. MedlineGoogle Scholar
  • Bowling BV, Huether CA, Wang L, Myers MF, Markle GC, Dean GE, Acra EE, Wray FP, Jacob GA (2008b). Genetic literacy of undergraduate non-science majors and the impact of introductory biology and genetics courses. BioScience 58, 654-660. Google Scholar
  • Brewer CA, et al. (2011). Vision and change in undergraduate biology education: A call to action, Washington DC: American Association for the Advancement of Science, http://visionandchange.org/files/2011/02/Vision-and-Change-low-res.pdf (accessed 20 February 2011). Google Scholar
  • Chi M, Roscoe R (2002, Ed. M LimónL Mason, The processes and challenges of conceptual change In: Reconsidering Conceptual Change: Issues in Theory and Practice, Dordrecht, The Netherlands: Kluwer Academic, 3-27. Google Scholar
  • Chi MT (2005). Commonsense conceptions of emergent processes: why some misconceptions are robust. J Learn Sci 14, 161-199. Google Scholar
  • Christianson RG, Fisher KM (1999). Comparison of student learning about diffusion and osmosis in constructivist and traditional classrooms. Int J Sci Educ 21, 687-698. Google Scholar
  • Cronbach LJ (1951). Coefficient alpha and the internal structure of tests. Psychometrika 16, 297-334. Google Scholar
  • D’Avanzo C (2008). Biology concept inventories: overview, status, and next steps. BioScience 58, 11079-1085. Google Scholar
  • D’Avanzo C, Anderson CW, Griffith A, Merrill J (2010). Thinking like a biologist: using diagnostic questions to help students reason with biological principles In: www.biodqc.org (accessed 17 January 2010). Google Scholar
  • Donovan MS, Bransford JD, Pellegrino JW (1999). How People Learn: Bridging Research and Practice, Washington, DC: National Academies Press. Google Scholar
  • Ebert-May D, Brewer C, Allred S (1997). Innovation in large lectures—teaching for active learning. Bioscience 47, 601-607. Google Scholar
  • Fisher KM, Wandersee JH, Moody DE (2000). Mapping Biology Knowledge, Dordrecht, The Netherlands: Kluwer Academic. Google Scholar
  • Fisher KM, Williams KS (2011). Concept inventories and conceptual assessments in biology (CABs): An annotated list In: www.sci.sdsu.edu/CRMSE/files/Concept_Inventories_in_Biology_20110325.pdf (accessed 17 April 2011). Google Scholar
  • Freeman S, O’Connor E, Parks JW, Cunningham M, Hurley D, Haak D, Dirks C, Wenderoth MP (2007). Prescribed active learning increases performance in introductory biology. CBE Life Sci Educ 6, 132-139. LinkGoogle Scholar
  • Friedler Y, Amir R, Tamir P (1987). High school students’ difficulties in understanding osmosis. Int J Sci Educ 9, 541-551. Google Scholar
  • Garvin-Doxas K, Klymkowsky MW (2008). Understanding randomness and its impact on student learning: lessons learned from building the Biology Concept Inventory (BCI). CBE Life Sci Educ 7, 227-233. LinkGoogle Scholar
  • Hartley LM, Wilke BJ, Schramm JW, D’Avanzo C, Anderson CW (2011). College students’ understanding of the carbon cycle: contrasting principle-based and informal reasoning. BioScience 61, 65-75. Google Scholar
  • Haslam F, Treagust DF (1987). Diagnosing secondary students’ misconceptions of photosynthesis and respiration in plants using a two-tier multiple choice instrument. J Biol Educ 21, 203-211. Google Scholar
  • Klymkowsky MW, Garvin-Doxas K, Zeilik M (2003). Bioliteracy and teaching efficacy: what biologists can learn from physicists. Cell Biol Educ 2, 155-161. LinkGoogle Scholar
  • Lawson AE (1988). A better way to teach biology. Am Biol Teach 50, 266-278. Google Scholar
  • Lemke J (1990). Talking Science: Language, Learning, and Values, New York: Ablex. Google Scholar
  • Marbach-Ad G, et al. (2009). Assessing student understanding of host pathogen interactions using a concept inventory. J Microbiol Educ 10, 43-50. MedlineGoogle Scholar
  • Marbach-Ad G, et al. (2010). A model for using a concept inventory as a tool for students’ assessment and faculty professional development. CBE Life Sci Educ 9, 408-416. LinkGoogle Scholar
  • Marek E (1986). Understandings and misunderstandings of biology concepts. Am Biol Teach 48, 37-40. Google Scholar
  • Marek EA, Cowan CC, Cavallo AML (1994). Students’ misconceptions about diffusion: how can they be eliminated. Am Biol Teach 56, 74-77. Google Scholar
  • McConnell DA, Steer DN, Owens KD, Knott JR, Van Horn S, Borowski W, Dick J, Foos A, Malone M, McGrew H (2006). Using conceptests to assess and improve student conceptual understanding in introductory geoscience courses. J Geosci Educ 54, 61-68. Google Scholar
  • Meltzer DE, Manivannan K (2002). Transforming the lecture-hall environment: the fully interactive physics lecture. Am J Phys 70, 639. Google Scholar
  • Michael J (2006). Where's the evidence that active learning works?. Advan Physiol Edu 30, 159-167. MedlineGoogle Scholar
  • Mintzes JJ, Wandersee JH, Novak JD (1998). Teaching Science for Understanding: A Human Constructivist View, San Diego, CA: Academic Press. Google Scholar
  • National Research Council  (2000). Bransford JDBrown LLCocking RR How People Learn: Brain, Mind, Experience, and School, expanded ed. In: Committee on Developments in the Science of Learning, Committee on Learning Research and Educational Practice, and Commission on Behavioral and Social Sciences and Education, Washington, DC: National Academies Press. Google Scholar
  • Odom AL (1995). Secondary and college biology students’ misconceptions about diffusion and osmosis. Am Biol Teach 57, 409-415. Google Scholar
  • Odom AL, Barrow LH (1995). Development and application of a two-tier diagnostic test measuring college biology students’ understanding of diffusion and osmosis after a course of instruction. J Res Sci Teach 32, 45-61. Google Scholar
  • Odom AL, Barrow LH (2007). High school biology students’ knowledge and certainty about diffusion and osmosis concepts. Sch Sci Math 107, 94-101. Google Scholar
  • Odom AL, Kelly PV (2001). Integrating concept mapping and the learning cycle to teach diffusion and osmosis concepts to high school biology students. Sci Educ 85, 615-635. Google Scholar
  • Sanger MJ, Brecheisen DM, Hynek BM (2001). Can computer animations affect college biology students’ conceptions about diffusion & osmosis. Am Biol Teach 63, 104-109. Google Scholar
  • She H-C (2004). Facilitating changes in ninth grade students’ understanding of dissolution and diffusion through DSLM instruction. Res Sci Educ 34, 503-525. Google Scholar
  • Shi J, Wood WB, Martin JM, Guild NA, Vicens Q, Knight JK (2010). Diagnostic assessment for introductory molecular and cell biology. CBE Life Sci Educ 9, 453-461. LinkGoogle Scholar
  • Smith JI, Tanner K (2010). The problem of revealing how students think: concept inventories and beyond. CBE Life Sci Educ 9, 1-5. LinkGoogle Scholar
  • Smith JP III, di Sessa AA, Roschelle J (1993). Misconceptions reconceived: a constructivist analysis of knowledge in transition. J Learn Sci 3, 115-163. Google Scholar
  • Smith MK, Wood WB, Knight JK (2008). The genetics concept assessment: a new concept inventory for gauging student understanding of genetics. CBE Life Sci Educ 7, 422-430. LinkGoogle Scholar
  • Sundberg M (2002). Assessing student learning. Cell Biol Educ 1, 11-15. LinkGoogle Scholar
  • Sundberg MD, Dini ML (1993). Science majors vs. nonmajors: is there a difference?. J Coll Sci Teach 22, 299-304. Google Scholar
  • Sundberg MD, Dini ML, Li E (1994). Decreasing course content improves student comprehension of science and attitudes towards science in freshman biology. J Res Sci Teach 31, 679-693. Google Scholar
  • Tanner K, Allen D (2005). Approaches to biology teaching and learning: understanding the wrong answers—teaching toward conceptual change. Cell Biol Educ 4, 112-117. LinkGoogle Scholar
  • Tekkaya C (2003). Remediating high school students’ misconceptions concerning diffusion and osmosis through concept mapping and conceptual change text. Res Sci Technol Educ 21, 5-16. Google Scholar
  • Tessier J (2004). Using peer teaching to promote learning in biology. J Coll Sci Teach 33, 16-19. Google Scholar
  • Tsui CY, Treagust D (2009). Evaluating secondary students’ scientific reasoning in genetics using a two-tier diagnostic instrument. Int J Sci Educ 32, 1073-1098. Google Scholar
  • Westbrook SL, Marek EA (1991). A cross-age study of student understanding of the concept of diffusion. J Res Sci Teach 28, 649-660. Google Scholar
  • Wilson CD, Anderson CW, Heidemann M, Merrill JE, Merritt BW, Richmond G, Sibley DF, Parker JM (2007). Assessing students’ ability to trace matter in dynamic systems in cell biology. CBE Life Sci Educ 5, 323-331. Google Scholar
  • Wood WB (2009). Innovations in teaching undergraduate biology and why we need them. Annu Rev Cell Dev Biol 25, 92-112. Google Scholar
  • Woodin T, Carter VC, Fletcher L (2010). Vision and change in biology undergraduate education, a call for action-initial responses. CBE Life Sci Educ 9, 71-73. LinkGoogle Scholar
  • Woodin T, Smith D, Allen D (2009). Transforming undergraduate biology education for all students: an action plan for the twenty-first century. CBE Life Sci Educ 8, 271-273. LinkGoogle Scholar
  • Zuckerman JT (1998). Representations of an osmosis problem. Am Biol Teach 60, 27-30. Google Scholar