FIRST—What's Next?

The Faculty Institutes for Reforming Science Teaching (FIRST II) ( http://www.first2.org) network was established in 1998 by ecologists accustomed to conducting their research in the field. The original impetus behind establishing FIRST was to engage biology faculty in active, inquiry-based science teaching, first in the field, an ideal environment for cooperative learning, and then in their classes. The impact of FIRST is seen in the activities of faculty now from all areas of biology who are change agents for the improvement of undergraduate science education, but major challenges toward implementing change still loom. Like many proponents of change, we are flush with new course and curricular materials and active-learning instructional strategies, but short on substantive research to back our claims that the “new” teaching approaches promote student learning better than traditional approaches. Many faculty remain unconvinced of the need for change because they have not been confronted with enough assessment data demonstrating that the new instructional methods have a positive impact on student learning—it is the “show me the data” dilemma. In fact, many faculty do have substantive assessment data, and others would like to collect such data. The challenge is learning what kinds of questions to ask; how to gather and analyze assessment data; and, ultimately, how to disseminate the results to an appropriate target audience.

One way to meet these challenges has been through publication of assessment results in a peer-reviewed educational journal such as CBE—Life Sciences Education, whose registered subscriber base is growing at a rapid rate. However, it is difficult for such a journal to provide sufficient guidance in assessment strategies and techniques to potential authors new to educational research. We present the case here for a somewhat different approach toward meeting these challenges: the creation of an assessment database in biology education.

In 1998, the National Science Foundation supported prototype projects that led to the establishment of the National Science Digital Library. This online library provides organized access to resources and tools that support innovative instruction and research in science, technology, engineering, and mathematics at all levels. Resources include images, animations, laboratory activities, and classroom materials to assist faculty and enhance student learning. We are developing the FIRST assessment database so the community of life scientists can begin to determine the effectiveness of nationally available course and curriculum materials, and their own teaching and learning activities. This database is an aggregation of assessment data about student learning in undergraduate biology courses. It will be a useful tool to facilitate and encourage faculty from all fields of biology to conduct research in biology education.

The fundamental principles for the FIRST database are modeled after those described for the Long-Term Ecological Research Network ( http://lternet.lternet.edu/DTOC/) database and GenBank ( http://www.ncbi.nlm.nih.gov/Genbank/index.html). To be successful, such a database must contain data desired by and useful to a large group of users, data must be easily added, the level of technological sophistication needed to use the system must match that of the users, and there must be a mechanism for submitting and dealing with queries that identify user needs (Michener et al., 1997).

Open source programs associated with the FIRST database will provide faculty with easy, efficient ways to document and code their data for uploading to the database. Faculty can upload student response data and download coded student data in a variety of file formats designed to work with programs such as Excel, Access, Filemaker Pro, SAS, and SPSS. As faculty gain experience using the database, they can extract data from different courses and institutions to conduct comparative analyses. For example, faculty can search and download questions and student responses from assessments on evolution that include multiple-choice instruments and extended response questions. Investigators may request restricted access to their data for a limited time of up to 3 yr, to encourage use of the database while giving them time to publish. After the release date, data will be available to all other database users. Documentation for all data retrieved from the database will contain source metadata including the names, e-mails, and institutions of the submitted data and links to any publications associated with the data.

The database accepts a variety of assessment types, including multiple choice; “clicker data”; short answer; original text of essays; coded responses of essays; computer-generated files, including scanned student drawings; and concept maps. The database is dynamic, which means that as new assessment formats are developed, new schema files (similar to Endnote connection files) can be generated, ensuring future database compatibility. Before data are uploaded, information that could identify an individual student will be replaced with a random number to “deidentify” data in compliance with Institutional Review Board regulations. Only faculty will maintain the ability to decode student identifiers locally.

Currently, we are field-testing the database by uploading and downloading multiple types of assessments and students' responses. For example, we have added the Concept Inventory of Natural Selection (Anderson et al., 2002) and assessments for the carbon cycle (Ebert-May et al., 2003) to the database, with metadata describing questions, foils (for multiple choice), and extended responses. The metadata show the specific content area for each question, for example, 1) populations evolve, not individuals; and 2) genetic traits are inherited, not acquired. Eventually, investigators may download files and tables of student responses that are classified according to these concepts and then import formatted data into Excel or other software for analysis.

Faculty can also mine raw student responses from many courses and institutions and recode the data to meet their research objective. As a hypothetical example, suppose an instructor predicts that understanding DNA replication and protein synthesis is vital to learning evolution by natural selection. To test the hypothesis, she needs to match assessments of DNA replication and protein synthesis to assessments of evolution. First, the instructor queries the database and finds all courses that include assessments for both concept areas; 50 courses from 20 institutions are available. After examining the course and instrument metadata, the instructor narrows the data set further to majors' courses that used pre- and posttests, thereby reducing the data set to 30 courses from 10 institutions. When the instructor downloads the data for analysis, the query is saved with an accession number. This number acts as a record that can be used when double-checking results and can be cited in published articles.

The FIRST database provides instruments and protocols for conducting classroom research, as well as data for cross-institutional studies of student learning and faculty teaching. However, the question remains, “If we build it, will they come?” We are currently studying the nature, extent, and effects of faculty change among a large number of faculty who participated in two different professional development programs. If faculty do change to improve their teaching practice based on learning theory and tested pedagogical principles, will student learning in fact improve? The FIRST assessment database will help to explore that question. It will also provide a useful source of assessment methods, data, and analyses to faculty who want to improve their teaching and measure the results, or simply to convince their colleagues that improving teaching can significantly affect student learning. The prototype for the database is in place; if the project is funded, a public version will be available within 12 mo. In the meantime, a subsample of faculty from the American Society for Cell Biology were surveyed about their interest in using the database. We welcome any feedback and will solicit input from individuals about the database, so together we can improve undergraduate life science education.

FOOTNOTES

ACKNOWLEDGMENTS