One challenge in science education assessment is that students often focus on surface features of questions rather than the underlying scientific principles. We investigated how student written responses to constructed-response questions about photosynthesis vary based on two surface features of the question: the species of plant and the order of two question prompts. We asked four versions of the question with different combinations of the two plant species and order of prompts in an introductory cell biology course. We found that there was not a significant difference in the content of student responses to versions of the question stem with different species or order of prompts, using both computerized lexical analysis and expert scoring. We conducted 20 face-to-face interviews with students to further probe the effects of question wording on student responses. During the interviews, we found that students thought that the plant species was neither relevant nor confusing when answering the question. Students identified the prompts as both relevant and confusing. However, this confusion was not specific to a single version.

Vol. 14, No. 2

June 01, 2015

Supplemental Materials

Metrics

Downloads & Citations

Downloads: 350

Citations: 31

History

Submitted: 3 July 2014

Revised: 17 January 2015

Accepted: 27 January 2015

Information

© 2015 M. Weston et al. CBE—Life Sciences Education © 2015 The American Society for Cell Biology. This article is distributed by The American Society for Cell Biology under license from the author(s). It is available to the public under an Attribution–Noncommercial–Share Alike 3.0 Unported Creative Commons License (http://creativecommons.org/licenses/by-nc-sa/3.0).

ACKNOWLEDGMENTS

The authors thank and acknowledge the contributions of Joyce Parker (for help with item creation, rubric development, and interview protocol), Alexander Lyford and Jennifer Kaplan (for help with statistical testing and discussions about results), and Emily Norton-Henry and Matthew Berry (for help in expert scoring and conducting interviews). The work presented here was supported by the National Science Foundation (DUE 1022653). Any opinions, findings and conclusions expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. Data collection for this project was approved by the Michigan State University Institutional Review Board (x10-577). Analytic resources used for lexical analysis that are described in this report, as well as information about the Automated Analysis of Constructed Response research project and help with TA, can be found at: www.msu.edu/∼aacr.

PDF download