Education at the National Academies: Three Recent Reports on Improving Science Education

INTRODUCTION

The federal No Child Left Behind Act (NCLB), with its requirements for increased testing, accountability, and teacher quality, coupled with concerns about the performance of U.S. students on national and international examinations and the future of the scientific workforce, are resulting in a greater level of scrutiny of the performance of education in both the secondary grades and higher education. This increased attention has focused especially on the quality and level of mathematics and science education that students are receiving during their high school and college years.

This is a good news/bad news scenario. The good news is that concerns about science education are becoming especially focused and rising to a level of concern in the Congress and other parts of the federal government that rarely have been seen before. This increased attention is due in part to lackluster performance on national and international examinations, which becomes more glaring as U.S. students progress through their precollege years. It is also related to the realization by education officials in states and local school districts, who have worked since the inception of NCLB to improve student performance in reading and mathematics, that they now must also demonstrate similar improvements in science beginning in the 2007–2008 school year. There's nothing quite like testing coupled with accountability to get people's attention!

The potentially bad news is that if science is tested poorly, the messages that those statewide tests convey to teachers, parents, and school administrators about what is important for students to know and be able to do in science could result in a setback for the kinds of science education that have been promoted for precollege and undergraduate education during the past decade by the American Association for the Advancement of Science (AAAS; 1993, 1997, 2001; see also Rutherford and Ahlgren, 1990), National Research Council (NRC; 1996, 1997, 1999, 2000, 2001a, b), National Science Foundation (NSF; e.g., 1996), and other organizations (National Science Teachers Association, 2000, 2001, 2003).

The problem becomes especially thorny when trying to decide whether and how to assess students' knowledge and skills that are learned in science laboratories. If the adage that what is valuable is that which is measured, then science assessments that do not require or emphasize specific laboratory skills could result in a scaling back or even the demise of a component of science courses that many see as too expensive, potentially dangerous, and of dubious educational value, although others see laboratory experience as the core of any science program. But what kinds of knowledge and skills should be tested? Is there any consensus about what the goals and outcomes of teaching laboratories should be for high school and college students, especially for those students who do not aspire to careers in science, mathematics, or engineering? Can the why and how of testing laboratory skills be informed by a body of research?

This confluence of events has resulted in requests from members of Congress or the National Science Foundation for the National Academies¹ to investigate these critical issues, leading to the issuance of three reports during fall/winter 2005, which I describe in this article. These reports are as follows:

• Rising Above the Gathering Storm: Energizing and Employing America for a Brighter Economic Future (National Academy of Sciences [NAS] et al., 2005)

• Systems for State Science Assessment (NRC, 2005b)

• America's Lab Report: Investigations in High School Science (NRC, 2005a)

1The National Academies are a private, nonprofit organization whose charter is to assist the federal government and the American people in analyzing pressing science and technology policy issues. The National Academies include three honorific societies: the National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. The National Research Council serves as the operating and research arm. Much of the work of the National Academies is performed by leading scientists, mathematicians, engineers, social scientists, and policy experts who provide pro bono service to the National Academies and the nation.

OVERVIEW OF REPORTS

Rising Above the Gathering Storm

Rising Above the Gathering Storm (Figure 1) was produced by the Committee on Science, Engineering, and Public Policy of the National Academy of Sciences, National Academy of Engineering, and the Institute of Medicine. The report was produced at the request of Senators Lamar Alexander (Tennessee) and Jeff Bingaman (New Mexico), and later of Representatives Sherwood Boehlert (New York) and Bart Gordon (Tennessee) from the House Committee on Science, to assist members of Congress in their deliberations about maintaining the nation's competitiveness in an increasingly global economy. The study committee included presidents from major universities, Nobel laureates, and chief executive officers of Fortune 100 corporations. They were asked to respond to the following questions:

• What are the top 10 actions, in priority order, that federal policy makers could take to enhance the science and technology enterprise so the United States can compete successfully, prosper, and be secure in the global community of the twenty-first century?

• What implementation strategy, with several concrete steps, could be used to implement each of those actions?

Following the committee's hearings and deliberations, the members offered four recommendations: 1) increasing America's talent pool by vastly improving K–12 science and mathematics education; 2) sustaining and strengthening the nation's traditional commitment to long-term basic research that has the potential to be transformational to maintain the flow of new ideas that fuel the economy, provide security, and enhance the quality of life; 3) making the United States the most attractive setting in which to study and perform research to develop, recruit, and retain the best and brightest students, scientists, and engineers from within the United States and throughout the world; and 4) ensuring that the United States is the premier place in the world to innovate by investing in downstream activities such as manufacturing and marketing and by creating high-paying jobs that are based on innovation. These goals should be accomplished by modernizing the patent system, realigning tax policies to encourage innovation, and ensuring affordable broadband.

These recommendations were supported by a total of 20 action items for implementation. Action items for K–12 and undergraduate education are presented below.²

2Rising Above the Gathering Storm was available only as a prepublication at the time that this article was accepted for publication. The report is subject to editing prior to releasing the final version, so the recommendations and action items listed here may not reflect the exact language in the final report.

Recommendations for K–12 education from the Report Rising Above the Gathering Storm:

Recommendations for higher education from the report Rising Above the Gathering Storm:

3The controls governed by the federal Export Administration Act extend to the transfer of technology. Technology includes “specific information necessary for the `development,' `production,' or `use' of a product.” Providing information that is subject to export controls to a foreign national within the United States may be “deemed” an export, and that transfer requires an export license.

To date, the report has been discussed with members of Congress in both the House and Senate, several cabinet secretaries, and many others in the federal government. It can be downloaded in its entirety without cost at http://books.nap.edu/catalog/11463.html.

Systems for State Science Assessment

In response both to the imminent requirements for testing of science knowledge that is mandated by NCLB and the more general goal of promoting high-quality assessments of science, the NSF asked the NRC's Board on Testing and Assessment to study what is currently known about science assessments for grades K–12 and to offer recommendations about ways to improve assessment of students' knowledge and skills in science.

The study committee reiterated a fundamental position of the National Science Education Standards (NRC, 1996), stating that scientific literacy should be the primary goal for K–12 science education. The committee noted that an essential element of science literacy is a strong foundation in the content knowledge of the life, physical, earth, and space sciences. They also emphasized that it is critically important for students to understand science as a specific way of knowing and to develop the skills necessary to both understand and appropriately apply the strategies of scientific inquiry.

The resulting report, Systems for State Science Assessment (NRC, 2005b; Figure 2), examines the ideas and tools that are needed to assess science learning at the state level.⁴ This book provides a detailed examination of K–12 science assessment, looking specifically at what should be measured and how to measure it. It builds on earlier work on assessment of learning that was reported by NRC (2001c). An important conclusion of this report is that states and the designers of assessments need to incorporate these fundamental principles of science literacy when designing science assessments for NCLB.

4As for all subjects that are required to be tested (currently mathematics and reading), NCLB assigns responsibilities to the states to assess and report data on student learning.

The study committee recognized that because each state has its own goals for science education and assessment, any guidance that they offered had to not only be specific enough to address the important issues raised by the NCLB science requirements but also general enough to be adaptable to a wide range of contexts. For example, as the committee noted in the Executive Summary to its report, one state might choose to develop a single hybrid test in which students take a core assessment that provides individual results, along with an assessment with a matrix sampling design⁵ that provides information about the achievement of groups of students across some area of scientific content. Another state might elect to combine standardized classroom assessments that provide diagnostic, descriptive, and interpretive information with an external assessment to judge the progress that all students are making toward achieving state standards for program evaluation. A third state may decide to eschew a statewide test and opt instead for one of many different models in which results from local, district, or state assessments are combined, aggregated, and reported for specific purposes.

5In a matrix sampling design, the group being tested collectively responds to the entire set of questions being asked, but each test taker answers only a subset of questions. Thus, the purpose of a matrix sampling model is to measure whether some group of people (e.g., students in a school or an entire school district) has mastered the corpus of information or set of concepts being tested without assigning individual achievement scores to each person. For additional information, see http://www.ericdigests.org/2005-1/matrix.htm.

The committee makes clear that a single assessment strategy cannot provide all of the information that education decision makers in classrooms, schools, school districts, and states need to support student learning. Teachers need ongoing information about their students' learning so that they can target instruction; students need timely feedback on how they are meeting expectations so that they can adjust their learning strategies; districts need information on the effectiveness of their programs; and policy makers need to know how well their policies are working and where resources might best be targeted. Addressing all of these needs for assessment-based information requires multiple assessment strategies, each designed to serve its own specific purpose. These multiple assessment strategies should be designed from the beginning to function as part of a coherent system of assessment.

The committee's advice to states is offered in the form of 10 recommendations and 35 questions that all those responsible for designing and implementing state assessment programs should consider as they develop science assessments. These recommendations and questions are intended to focus state decision makers on important issues that need to be addressed as assessments are developed, implemented, and used. The recommendations and a compilation of all of the questions that are raised throughout the report can be found in Chapter 9. Although most of the recommendations are directed at federal and state agencies and departments of education, one recommendation is addressed to higher education faculty and institutions:

“Recommendation 6: Postsecondary institutions that prepare science teachers should require that preservice science teachers have appropriate knowledge and skills regarding effective science assessment practices. Such knowledge includes the use of assessment results in promoting student learning and making decisions about instruction, developing and using sound assessments, and understanding the limitations of various types of assessment practices and results. Accomplishing this requires that preservice teachers have a deep understanding of the science they teach” (NRC, 2005a, page 9-4).

This report can be read online in its entirety without cost at http://books.nap.edu/catalog/11312.html.

America's Lab Report: Investigations in High School Science

Laboratory experiences have been a part of U.S. high school science curricula for decades. Since the late nineteenth century, high school students in the United States have carried out laboratory investigations as part of their science classes. Today, on average, students enrolled in science classes spend about one class period per week in laboratory investigations such as observing and comparing different cell types under a microscope in biology class or adding a solution of known acidity to a solution of unknown alkalinity in chemistry class.

Educators and policy makers have periodically debated the value of laboratory experiences in helping students understand science, but little research has been done to inform those debates or to guide the design of laboratory education. Laboratory activities commonly used in high school (and I would contend, in many college settings) rarely have been examined carefully or critically. What purposes do they serve and how much do they contribute to science learning? What might laboratory time contribute to science learning if it were structured differently?

At the request of the NSF, the NRC study committee that authored this report (Figure 3) examined a range of questions about how laboratory experiences fit into U.S. high school science courses, including the following:

• What is effective laboratory teaching?

• What does research tell us about learning in high school science laboratories?

• How should student learning in laboratory experiences be assessed?

• Do all students have access to laboratory experiences?

• What changes need to be made to improve laboratory experiences for high school students?

• How can school organization contribute to effective laboratory teaching?

In response, the study committee first undertook an extensive review of the literature on science laboratories and identified a number of learning goals that traditionally have been associated with science laboratory experiences (see column 1 of Table 1). As a result of this literature review, the committee reached an important conclusion:

Given this apparent state of disagreement, the committee adopted its own definition of laboratory experiences:

The results of their findings about the effectiveness of “typical laboratory experiences” (i.e., those that traditionally have been disconnected in time or content from the flow of classroom activities) versus“ integrated instructional units” (i.e., laboratory activities that are connected with other types of science learning such as lectures, readings, and discussion) are summarized in columns 2 and 3, respectively, of Table 1.

Importantly, the committee found that evidence for the efficacy of both typical laboratory experiences and integrated instructional units is currently lacking for a number of the higher order learning goals that instructors claim they are trying to achieve.

The committee framed its other findings as a set of conclusions about high school labs (reported here verbatim from the report's Executive Summary):

In suggesting next steps for improving high school science laboratories, the committee posed a series of questions for curriculum developers, implementers, and school administrators. The audiences for this report include science educators, school administrators, policy makers, and parents, who would benefit from a better understanding of the need for laboratory experiences to be an integral part of the science curriculum and of how those objectives might be accomplished. This report can be read online in its entirety without cost at http://books.nap.edu/catalog/11311.html.

WHAT DO REPORTS ON LARGE-SCALE, STATE SCIENCE ASSESSMENTS AND HIGH SCHOOL SCIENCE LABORATORIES HAVE TO DO WITH HIGHER EDUCATION?

Although the connection between the nation's competitiveness in science and technology and the role of higher education is apparent, readers may be asking why this column also is devoting so much space to NRC reports that are focused on K–12 education. There are several reasons. First, there is a fine line (both educationally and with regard to students' development and maturity) between a graduating high school senior and an entering college student. The kinds of experiences (or lack thereof) in science that students encounter during their K–12 years will have direct consequences on what college-level instructors will be able to accomplish in their own classrooms and teaching laboratories. Being aware of the issues that K–12 education faces in their own state and nationally will allow readers to better understand and appreciate the kinds of experiences in science that students bring with them to college courses and allow instructors to better serve the educational needs of those students.

Second, although deliberately broad, the issues and questions that are posed in these reports on state assessment of science and on high school science laboratories are also applicable on a smaller scale to individual college courses and departmental curricula in the natural sciences. For example, college faculty also should reflect on whether the assessments that they use in their own courses or as indicators at the end of a student's career in a science department are really measuring the kind of learning that they claim to value. They also might ask themselves whether the laboratory experiences that they provide to students in their courses, especially at the introductory level or for students who will not major in one of the sciences, meet the definition for quality laboratory experiences that the NRC study committee has developed.

Third, institutions of higher education can do much more to assist local schools and school districts to improve their science education programs. By understanding the issues and challenges that teachers and administrators face every day, college and university faculty can make substantive contributions to improving science education. These reports can help inform that process.