Providing Undergraduate Science Partners for Elementary Teachers: Benefits and Challenges

INTRODUCTION

Reports from high-level panels representing the scientific, engineering, and corporate communities have emerged with increasing frequency and urgency for at least the past 25 yr, warning that the country is not producing a sufficient supply of scientific and technical talent (National Commission on Excellence in Education, 1983; Glenn Commission Report, 2000; National Research Council [NRC], 2003). One recent expert panel concluded that “Laying a foundation for a scientifically literate workforce begins with developing outstanding K–12 teachers in science and mathematics …[who have a] thorough knowledge of content, solid pedagogical skills, motivational abilities, and career-long opportunities for continuing education” (NRC, 2006).

In its 1990 Science Report Card, the National Assessment of Educational Progress (National Center for Educational Statistics, 1992) cited major flaws in science instruction in American elementary schools: Teachers often had inadequate training in math and science content and were uncomfortable teaching those subjects (Barr, 1994); science instruction was given to most students only a few minutes each day, whereas 28% received only a few minutes a week or none at all (Alberts, 1993); and focus was generally on passive learning of factual information rather than on science as a way of asking questions or of increasing problem-solving skills. These realizations led the NRC, under the leadership of Bruce Alberts, then president of the National Academy of Sciences, to undertake the development of a set of national science education standards (Alberts, 1994) that could be used to guide the professional development of K–12 teachers in science.

To help provide funds to support improvements in science teaching, the National Science Foundation (NSF) established the Local Systemic Change (LSC) Initiative in 1994. The initiative's primary goal was to improve instruction in science, mathematics, and technology through teacher professional development. A consensus was emerging in the education community at that time, regarding “best practices” in professional development that could guide LSC project leaders (Loucks-Horsley and Hergert, 1985; Showers et al., 1987; Loucks-Horsley et al., 1998; see also Wayne et al., 2008). Evidence was accumulating in the psychological and pedagogical literature that the most effective learning occurs through hands-on, guided-inquiry instruction (Duckworth et al., 1990; Tobin et al., 1994), an idea since recognized as important at all educational levels (DeHaan, 2005; Perkins and Wieman, 2008). In an inquiry-based lesson, the teacher skillfully guides learners to actively “discover” important ideas on their own, rather than having them passively receive facts and concepts. Moreover, it was found that sustained professional development that was located in the teachers' classroom and that focused on the content of the subject that teachers teach, was essential to improve classroom instruction and student achievement (Loucks-Horsley and Hergert, 1985; Loucks-Horsley et al., 1998; Garet et al., 2001). Against this background, NSF funded the first cohort of LSC projects in 1994 and an additional cohort of projects each year until 2002 (for a total of 88 projects funded).

One of the goals of the LSC program was to bring school personnel into working partnerships with members of the scientific community, an idea that has since led to a burgeoning variety of “outreach” efforts (Bruce et al., 1997; Tanner et al., 2003; Dolan and Tanner, 2005; Moreno, 2005; Laursen et al., 2007). In the mid-1990s, the plan was based on the view of many in the emerging national education reform movement (Smith, 1994) that scientists visiting teachers in their classrooms could bring their expertise, their sense of confidence about the subject, and their enthusiasm for the scientific enterprise (Alberts, 1993; Bruce et al., 1997). Another LSC goal differed from those of prior NSF-supported teacher enhancement efforts by explicitly promoting teacher empowerment programs to build a supportive community environment for improving science, mathematics, and technology instruction. This focus on creating comprehensive, long-term, in-depth experiences for teachers arose from the insight that the traditional brief “one-shot” professional development workshops for teachers were largely ineffective, an insight confirmed by later program evaluation studies (Supovitz and Turner, 2000; Raghavan et al., 2001; Wayne et al., 2008).

In the present article, we describe elements of an LSC project, the Elementary Science Education Partnership (ESEP). We focus primarily on the professional development (PD) effort we designed for the classroom teachers and on the impact of the program on the undergraduate college students who served as their “science partners.” Further details of the chronology and history of the ESEP program are available at www.nas.edu/rise/examp65.htm and from doctoral dissertations (May, 2000; Falkenberg, 2002; Blackmon, 2003; Goebel, 2008) and publications describing early elements of the project (DeHaan, 1995; Kozaitis, 1997; Weinburgh, 2003). Banilower et al. (2006) have reported on the results of the LSC program nationally.

Location and History of the ESEP Program

Origins. The ESEP program began in 1993–1994 as a student-inspired pilot project at Emory University, informed by school–university partnerships in other parts of the country (Alberts, 1993; Bower, 1994; Yure et al., 1997; Comfort, 2001; Raghavan et al., 2001).¹ In the initial phase, we arranged to place science-literate undergraduates as “para-professionals” with elementary teachers to help them teach science. Both teachers and undergraduates were highly enthusiastic about their experiences with this pilot effort, and these positive reports gained us a subvention from the Emory University Provost and a small Fund for the Improvement of Postsecondary Education grant. These funds allowed two of the authors (DeHaan, Umoja) and the project co-organizers (Drs. L. Vernon Allwood and Molly Weinburgh) to participate in a Strategic Planning Institute at the National Science Resources Center (Washington, DC) during summer 1995. As a result of this experience, the aims of the ESEP program became more sophisticated. We realized that it would be necessary to provide training for the college students in the fundamentals of inquiry science instruction and in how to assist elementary teachers; we also would need to offer PD for teachers. The goals cited in an NSF LSC ESEP grant application, submitted early in 1995, were to provide college science students with an intensive teaching experience to enhance teachers' skills in inquiry-based science instruction, and to improve science education at the K–5 level. The goals and philosophy of ESEP were later outlined on our website (www.college.emory.edu/esep/index2.html) as follows:

The modified plan outlined in the grant proposal was to provide professional orientation seminars for college students, based on principles of inquiry-based science instruction as promoted by the National Science Resources Center (www.nsrconline.org) and described in prepublication editions of the National Science Education Standards (NRC, 1996). These undergraduates were to be paired with teachers to serve as science partners in their classrooms (Alberts, 1993; Pelaez and Gonzalez, 2002). NSF funding in September 1995 (grant ESI-9552864) provided resources to hire experienced professional developers and an administrator and to expand the pilot project. Instead of informal arrangements to place students from Emory with teachers in a few schools, ESEP grew into a formal partnership between a consortium of seven urban Atlanta universities² and an entire school district.

The Public School System. The school district that became ESEP's partner, which for purposes of anonymity we are calling LPS, is one of several large urban systems in the Atlanta area. In 1995, LPS included 78 elementary schools (K–5) that used >1500 classroom teachers serving approximately 32,000 children. Eighty-three percent of the instructional staff and 91% of the students were African-American, whereas 76% of the latter qualified for free or reduced cost lunch. Results of a 1994 statewide assessment in grades 3 and 5 confirmed that the system had problems in science instruction. Only 23% of LPS third graders and 21% of the fifth graders achieved “acceptable” scores on the science section of the “curriculum-based assessment” in use by the district at that time.

Organization of the ESEP Professional Development Scheme

With NSF funds, ESEP was able to hire a staff of seven educators to be primarily responsible for the PD offered to a cadre of LPS mentor teachers, the college students, and the larger population of classroom teachers. In addition, we brought in a team of experienced professional developers from the Alabama Hands-on Activities Science Program at the University of Alabama, Huntsville (Comfort, 2001) to further enhance the PD skills of the ESEP administrators and staff. Professional developers from ESEP and the Alabama group worked together during the first year to share expertise and implement the initial PD sessions.

To provide PD in inquiry science instruction for the large number of teachers in the LPS, we developed four main elements: 1) a cadre of mentor teachers to train the bulk of the classroom teachers in inquiry-based science instruction, 2) an orientation and recruitment program for training and supplying the undergraduate science partners for the classroom teachers to augment and support their professional development, 3) a strategy called “participatory reform” designed to empower and engage the teachers; and 4) a curriculum based on classroom materials (science “kits”) for experiential problem solving. The roll-out of the PD effort in 1995–1996 began with teachers from a limited number of schools in grades 2–4. By the end of that year, the district superintendent asked that we modify the plan for 1996–1997 to recruit teachers district-wide but from only one grade level at a time, beginning with fifth grade.

Preparing Mentor Teachers. Each year, teachers were recruited to become Science, Knowledge, Inquiry, Leadership (SKIL) mentor teachers. The selection process required teachers to provide two letters of recommendation and an essay describing their goals. One of the letters had to be from their school principal agreeing to support the applicants' 10 d out-of-class commitment to provide PD to other teachers (6 d) and receive 4 d of their own PD if they were selected. They were required to complete 2 yr of training, including attendance at two SKIL summer institutes and to lead mentored professional development workshops for classroom teachers. Each SKIL institute provided 80 h of intensive study in inquiry instruction with the science modules. To become a SKIL teacher was considered an honor by teachers and school administrators. Sixty teachers applied for the SKIL teacher program the first year; 24 were selected. During the following years, teachers continued to compete for the distinction of becoming a SKIL teacher. Over the 5 yr of the grant period, ESEP recruited and trained a cadre of 54 SKIL mentor teachers who provided most of the professional development in inquiry-science instruction for the bulk of the 1400+ classroom teachers.

The SKIL institutes had two goals: 1) to provide the PD skills and knowledge for a cadre of lead teachers to enable them to facilitate inquiry-science workshops and to serve as mentor teachers for other classroom teachers; and 2) to promote group cohesion, acceptance of the new instructional strategies, and a sense of empowerment among the participants that we realized would be crucial to their development (Sweetland and Hoy, 2000; Hirsch and Emerick, 2006). The strategy to achieve the latter goal was to incorporate the principles of the participatory reform program (described below) into the SKIL institute. Each participating SKIL teacher attended two or three ESEP PD workshops from which they gained additional knowledge about an assigned, grade-appropriate science kit that was to be taught at their grade level that year (see below). These workshops focused on helping the SKIL teachers learn both the science content and the pedagogical strategies required to teach the kit, including training in inquiry learning strategies; the 5-E learning cycle (NRC, 1996, 2005; Trowbridge et al., 2004); professional development design; and authentic assessment (Loucks-Horsley et al., 1998). The sessions also served as models of how to organize a teacher-training workshop and as opportunities to engage the SKIL teachers in elements of participatory reform. A detailed program of a SKIL training institute can be accessed at http://lsc-net.terc.edu/do.cfm/report/7009/show.

After the first summer SKIL institute, participants began serving as mentor teachers by co-leading kit-based training sessions for 15–20 classroom teachers with an ESEP staff professional developer. After the first 2 yr, as SKIL teachers grew more confident and were able to train their colleagues, ESEP staff was replaced with an experienced SKIL teacher to cofacilitate the training sessions. After each session, SKIL teachers participated in a group debriefing or an individual reflection session with the ESEP staff. As a result, the PD workshops for classroom teachers became additional PD experiences for the SKIL teachers themselves, as well as opportunities for qualitative data collection.

For the first 4 yr of ESEP's tenure, the LPS administration agreed to release SKIL teachers from their classrooms for the requested total of 10 d of PD. Four days were devoted to learning inquiry teaching strategies and group empowerment activities. The remaining six of those days were spent facilitating PD for other teachers under the guidance of ESEP staff or a more experienced SKIL teacher. During the last year of the program, the LPS administration became concerned about program costs after the grant ended and felt under increasing pressure to observe federal mandates for teacher in-class time. This led to an alteration of the goals of the SKIL institutes to include both mathematics and science (calling the resultant trainees MASKIL teachers) and to a reduction in the number of out-of-class days for PD.

Curricular Materials. Following the lead of other programs (e.g., Consuegra, 1994), we used commercially available science modules, or kits, from two publishers to provide science content and pedagogical strategies for the PD workshops. These kits were available from Science and Technology for Children produced by the National Science Resources Center, and Full Options Science Systems produced by the Lawrence Hall of Science (see www.delta-education.com/science/foss/index.shtml and www.carolinacurriculum.com/stc/index.asp). Each kit was designed to engage children in active learning about a specific science topic such as human body systems, insect development, or rocks and minerals. A matrix of 14 kits was selected (see www.college.emory.edu/esep/resources/scienceCurriculum.html) by a team composed of LPS teachers and administrators, and ESEP scientists and educators, to be integrated into the LPS curriculum.

During the pilot phase of the project, we purchased a few kits and stored them in the ESEP office. But soon after obtaining NSF support, the LPS made a commitment to establish a science materials support center (SMSC) to house and distribute kits and to refurbish them after they were used for PD purposes or by teachers in their classrooms. Operational in late 1996 and staffed by a director, two assistants, and a driver, the SMSC delivered >1000 kits designed for grade 1–5 teachers in three 6- to 8-wk cycles each school year, and then they retrieved and refurbished each kit for delivery to a different teacher in the next cycle. Information about how to organize such a center can be obtained from the Association for Science Materials Centers (www.kitsupport.org/).

Participatory Reform. Most LPS teachers were accustomed to teaching science (if at all) by using traditional textbook-based instruction. According to studies by Everett Rogers and others, such teachers are more likely to accept new inquiry-based instructional strategies if they perceive the new approaches as desirable, and if they believe they have the power to choose such strategies, rather than if the teachers perceive new approaches as being imposed from an outside agency (Rogers, 1995; Kozaitis, 1997). To offer a sense of empowerment (Sweetland and Hoy, 2000; Hirsch and Emerick, 2006) and to produce a lasting impact on teachers' belief systems (Haney et al., 1996; Nietfeld and Enders, 2003; Klentschy, 2005; Mundry, 2005), ESEP called on the expertise of Dr. Kathryn Kozaitis, a cultural anthropologist at Georgia State University, for assistance with the design and implementation of a PD strategy that she termed participatory reform.

Participatory reform had three elements that were interwoven into all ESEP PD activities: 1) ensuring that LPS teachers and administrators became vested in the reform effort: 2) actively engaging the teachers in the design and implementation of the training, and 3) providing culturally sensitive experiences that offered a sense of empowerment and group cohesion (Kozaitis, 1997). Parallel elements were included in the orientation and training of the science partners in an effort to ensure that their classroom behavior would reinforce the features of both inquiry-based instruction and participatory reform.

Science Partner Recruitment and Orientation. The benefits of placing scientists as adjuncts with elementary teachers, even for brief visits, have been well established (Otero et al., 2006; Laursen et al., 2007). The ESEP science partners were predominantly science- or premed-major undergraduates from the seven institutions who, typically, had no prior teaching experience. They were trained at their institutions by participating ESEP PD staff or faculty in the inquiry approach to learning science (NRC, 1996). Although there was variation in recruitment and orientation methods among the seven campuses, we found at Emory that the most effective strategy for instructing the undergraduates in how to work with elementary teachers in ways that integrated them into the overall PD program was to have science partners enroll in a 2 credit-hour, one-semester reflective service learning course (Eyler, 2002). At the beginning of each semester, students received 12 h of instruction on inquiry-based approaches to science learning, constructivist theory, science pedagogy, cultural change, and age-appropriate teaching methods. The course syllabus can be accessed at www.college.emory.edu/esep/resources/courseSyllabus.pdf. This course also provided the opportunity for students to be introduced to representative FOSS and STC science modules. An especially effective (and popular) part of the training were “mock” classrooms in which an experienced teacher brought a small number of his or her pupils (usually five to eight fourth- or fifth-graders) to a Saturday science partner orientation session and taught a sample lesson for the children while the undergraduates observed. The science partners program was feasible because each of the seven college campuses was within easy driving distance of a subset of the 78 LPS schools. The training was designed to help the science partners provide content knowledge, enthusiasm, and a supportive atmosphere during approximately 30 h of science co-instruction for an entire semester.

After orientation the undergraduates were assisted in selecting a classroom teacher who had applied for a science partner. Partners and teachers agreed to commit 3–4 h per week (on average) of joint work in the classroom. Each partner discussed weekly schedules and curricular materials with his/her teacher, and jointly planned the partners' classroom visits in advance. In the classroom, the partners were instructed to work as science paraprofessionals with the teachers, helping to lead classes in hands-on activities from the assigned kit or in other exercises. The intent was that the teacher and science partner would facilitate science lessons together. However, in practice teachers would often hand off responsibility for lessons to the science partner, remaining in the room merely for classroom management purposes. Such behavior may have revealed resistance of some teachers to engaging directly in the inquiry-based teaching strategies that ESEP was promoting. However, according to analysis of teachers' comments, the science partners were seen mainly to serve as role models; they helped children develop skills in problem solving and critical thinking and an understanding of the scientific method. They aided teachers in making science fun, exciting, accurate, and relevant. Each semester we recruited and trained approximately 200 college students from the seven campuses for a semester-long partnership.

Science partners were required to engage in intensive reflection (Peltier et al., 2006) and to keep reflective journals on their observations in the elementary school classroom. One evening per week, they met in groups of eight to 10 for 1-h reflection sessions. Leading or attending each reflection session was an experienced teacher or an ESEP staff member. Three times each semester a consulting cultural anthropologist led discussions on differences in children's learning styles and cultural aspects of both the children's and undergraduates' backgrounds that could affect classroom interactions.

METHODS

Program Evaluation and Data Collection

Like other LSC programs, ESEP was designed as a teacher-enhancement effort in science education reform, not as an experimental study. Nonetheless, we were able to assess the degree of success or failure of components of the program by using data from two sources. One source was the collection of surveys, interviews, and observations implemented by the authors and other ESEP staff who acquired data for the doctoral dissertations cited above, and for publications describing various elements of the project such as the participatory reform program (Kozaitis, 1997) and attitudinal changes toward science among children in ESEP classrooms (Weinburgh and Steele, 2000; Weinburgh, 2003).

The second body of evaluative data included qualitative research surveys, interviews, focus groups, and classroom observations collected by a professional evaluator.³ In 1995, NSF contracted with Horizon Research (HRI) in Chapel Hill, NC, to develop a data collection framework to be used by all of the LSC projects. The goals were to assess individual projects and to permit aggregation of data across projects (Banilower et al., 2006). Each project was required to hire a professional evaluator, whose duties would include the collection of baseline data during an LSC's first year of funding and a range of data collection activities during subsequent years. Evaluators were asked to provide comprehensive annual core evaluation reports driven by a set of core questions aimed at assessing the quality of the PD, its impact on teacher attitudes and pedagogical skills in mathematics and science teaching, and the degree of institutionalization of program features.

Data collection activities completed by the program evaluator annually included five to eight observations of PD activities, observations of a sample of teachers who were randomly selected by HRI, teacher questionnaires and interviews that probed respondents' attitudes and instructional practices, and questionnaires for principals intended to delve into their opinions about mathematics and science instruction and their school's progress in implementing LSC reforms. The evaluator's report was submitted each year to HRI and to the project principal investigator. Below, we distinguish data collected and interpreted by the evaluator from that collected by us or by other ESEP staff.

Evaluating ESEP Professional Development

Data Collected from Classroom Teachers and SKIL Teachers by the Authors and Other ESEP Staff. At the end of each PD workshop, ESEP staff required classroom teachers to complete a 22-item survey to assess indicators such as their sense of their own understanding of inquiry pedagogy, their ability to engage students in inquiry-based learning, and their knowledge about the ESEP kit materials, and to determine how often they engage the students in their own classrooms in various inquiry-based instructional activities (Hughley, 2000). To capture the teachers' views of the science partner program, one of us (Goebel) conducted surveys and interviews of partner–teachers, made classroom observations, and conducted teacher focus groups. The teachers being prepared as professional developers (SKIL teachers) completed even more intensive surveys, to assess their experience in terms of learning gains, new insights, and achievements as leaders and professional developers, and to obtain suggestions for future PD institutes. Protocols and results of some of these interview studies are available in two master's theses (Hughley, 2000; Jester, 2000) and four doctoral dissertations (May, 2000; Falkenberg, 2002; Blackmon, 2003; Goebel, 2008).

Data Collected from Classroom Teachers and SKIL Teachers by the Program Evaluator. In addition to requiring teacher questionnaires and interviews, each year HRI randomly selected 10 teachers from the LPS teacher file for classroom observations by the evaluator (teachers with science partners were not included). Similar classroom observations were made on SKIL teachers in their own classrooms, to assess improvements in their instructional skills as teachers, and while conducting a PD workshop to evaluate them as professional developers.

Data Collected from Science Partners by Us and Other ESEP Staff. The online journal entries and end-of-semester experience essays collected from every science partner produced rich sources of data. Each student was expected each week to engage in intensive reflection (Peltier et al., 2006) and to write a paragraph or two about his or her experiences, focusing on such issues as: significant learning events that happened in their classroom, their own role and that of their partner–teacher, changes noted in either the children's or teacher's behavior during the course of the semester, and their own attitudes and beliefs. Reflection sessions provided opportunities for session leaders to evoke discussions of issues that arose during the week and to become aware of any problems arising between undergraduates and their partner-teachers.

For an in-depth analysis case studies were conducted over two consecutive semesters with four science partners (pseudonyms: Anna, Badra, Chikara, and Dawei) by one of us (Goebel) using qualitative research methods. The case-study undergraduates were partnered with teachers in first, second, third, and fifth grades at different schools. In addition, data from 84 student-generated documents, 24 student and teacher interviews, and 32 classroom observations, totaling 140 documents, were collected and analyzed. At the start of the fall semester 2000, there were 39 Emory college students enrolled in the ESEP course. Of seven available novice (i.e., first-time) science–partner and teacher–partner pairs, the first four undergraduates who arrived at the first Emory ESEP class were selected for the case studies. These included one white and three Asian Americans from lower-, middle-, and upper-class families (self-reported). The participant undergraduates were interviewed using a semistructured protocol (Merriam, 1998) during their fourth, 11th, 20th, and 23rd weeks of participation. The interviews each took approximately 1 h to cover a protocol that included eight to 11 questions. Similar questions were repeated in each interview, e.g., “What evidence do you have that your [pupils] are learning science?” and “How has that knowledge affected your understanding of them as learners?” (see Supplemental Material SM-1 for 23-wk interview protocol). Their partner-teachers were each interviewed at the start and end of the partnership. The use of the semistructured open-question format was chosen to allow appropriate responses to the emergent views of the participants (Merriam, 1998). To establish validity by reducing the risk of chance associations and biases, the 140 documents represented a variety of data sources, individuals, and settings, allowing ample triangulation on the results (Maxwell, 1996). Transcriptions were compared with the audiotapes and checked with the study participants. To add to the internal generalizability of the study, four of the reflective focus-group discussions involving a combined total of 20 non–case-study science partners were analyzed after completing all work on the case participants.

All interviews, college partner documents, observations, and focus group data were made anonymous by using pseudonyms and then analyzed for emergent themes with data-grounded coding (Maxwell, 1996; Boyatzis, 1998) by using NVIVO 2.0 software (QSR International, 2002). Observations of classroom teaching and partner-teacher interviews were also descriptively quantified and used to contextualize and verify the primary data sources. To check the consistency of agreement on the codes and establish reliability, samples from eight primary data sources (interviews from weeks 4, 11, 20, and 23, and fall and spring journal and summary documents) were recoded by two colleagues experienced in qualitative research. Intercoder agreement on the presence of the code themes was 80–98%, substantially greater than the 70% generally considered adequate for qualitative analysis (Boyatzis, 1998, p. 155). The data were then re-examined, summarized, reviewed, and recoded iteratively. This procedure revealed clustering and changes in student attitudes. The interpretation of the data was an ongoing process assisted by 73 code definitions that documented the rules for the decisions made (Miles and Huberman, 1994). The themes that emerged were used to build categories for each case. Coded belief expressions included categories such as pupil ability to learn science (ABS/ability), pupil learning difficulties (ABS/difficulty), pupil willingness to learn (ABS/willingness/general), and pupil willingness to learn science (ABS/willingness/science). Finally, the expressions of all four cases were compared for commonalities, patterns, and dissimilarities. Separately, the focus group discussions were coded for patterns relevant to the study questions. The findings from the focus group science partners and the four case-study participants were then compared (Schensul et al., 1999). The beliefs expressed by one or more focus group participants that resembled those of the case students provided a useful point of reference from which to determine how representative the four case-study students' expressions were of the larger group. After this analysis, representative quotations were selected to fairly portray the changes in expressions of beliefs by the college science partners after two semesters of ESEP experience.

RESULTS

As detailed below, based upon qualitative analyses of the documents and sources described in Methods, the program had measurable impact in three areas: 1) modifying SKIL teachers' and classroom teachers' attitudes about inquiry science instruction, 2) providing for the development of a cohesive cadre of capable SKIL mentor teachers, and 3) altering science partners' beliefs.

Classroom Teachers' Attitudes Regarding Inquiry Science Instruction

During the 1993–1994 pilot project, our experience, corroborated by teacher self-reports (Blackmon, 2003, p. 91) and statements from school principals, was that most LPS teachers we worked with tended to avoid teaching science. Within a year or two after NSF funding of ESEP, surveys and interviews by us and other ESEP staff revealed substantial changes in those attitudes in teachers' self-reports about their instructional strategies. When teachers (grades 2–5; n = 207) were asked at the end of the third year about the impact of the PD classes they had attended on their science content knowledge and their ability to apply inquiry science instructional strategies to engage their pupils in conceptual learning, their responses (on a 5-point scale) averaged 4.72 (adapted from Hughley, 2000), suggesting that many teachers had undergone substantial improvements in their self-confidence and their attitudes about teaching science. Three reasons were most often cited by teachers for their improved views of science instruction: support from SKIL teachers and science partners, attendance at kit-based PD sessions, and convenient access to science materials in the kits.

The program evaluator's reports confirmed these attitudinal changes. Referring to pre-1995 conditions, she notes, “Before ESEP, most teachers and principals believed that there was a policy that mandated 45 min of each subject per day … Some teachers reported that they could not do hands-on [science] because 45 min was not enough time to accomplish the lesson so they did not even try” (Davis, 1999, p. 23). Once the program began, teacher interviews and questionnaires led the program evaluator independently to recognize that many teachers began to voice positive opinions about their ability to deliver accurate and well-implemented lessons in science. Teachers reported that they were teaching science on a more regular basis and for longer periods than before ESEP was instituted. She noted that this was “partly because of a greater consciousness of the importance of the subject, and partly because the regular visits of the science partners ensured that a science lesson would be taught approximately three hours each week for 10 weeks of each semester (Davis, 1999, p. 31).

Despite these positive attitudinal effects of the ESEP PD, the program evaluator found no such positive impact on indicators of actual quality of classroom instruction among the small number of randomly selected teachers whom she observed. For example, in the spring of year 2 (1996–1997) of the grant period, nine randomly selected teachers were observed by the program evaluator, four of whom (in the interests of space-saving) are shown in Table 1. The column headed ”Topic“ refers to the kit that the teacher was using for that class; ”Treated?“ refers to the number of hours of PD the teacher (identified by ID number) has had. The rating indicators answer the following questions on a 1–5 scale (1, poor; 5, excellent): How well did the teacher design the lesson? How well did the teacher implement the lesson? Did the teacher understand and convey the important scientific content? Was the pedagogy used by the teacher sensitive to cultural issues of diversity and equity? At the time of observation, this cohort of teachers had had 24–80 h of PD, and their evaluator ratings were highly variable, ranging from 2 to 5. When classroom observation data on eight to 10 teachers each year from 1996 to 2000 were combined and analyzed, no significant correlation was found between hours of PD (0–144 h) and ratings on any of the indicators. Pearson r values ranged from 0.115 to 0.321, but with the small samples, significance never reached even the 95% level.

Developing the Capacities of SKIL Teachers

Each SKIL teacher received 80 h of training in each of two summer SKIL institutes and facilitated training sessions for classroom teachers under the tutelage of an ESEP staff professional developer (which also counted toward their own hours of PD). Most also worked with a science partner for at least one semester. Each year the program evaluator observed a sample of SKIL teachers teaching in their own classrooms and a sample of professional development sessions that they implemented for other teachers.

When the program evaluator observed and rated SKIL teachers' instructional abilities in their own classrooms, using the same instructional indicators as those applied to the classroom teachers, she found striking differences. Her combined random samples of classroom teachers, when observed had had 0–144 h of PD, whereas the range for SKIL teachers was 80–266 h. As shown in Table 2, rating scores for the SKIL teachers were almost a full point higher for all indicators, suggesting that the SKIL teachers had internalized the knowledge and skills gained in their training workshops and were implementing them in their own classrooms. They were better able to design and implement lessons, more skilled at delivering the scientific content, and more sensitive to cultural and equity issues among their students, than the classroom teachers she observed. However, it must be noted that SKIL teachers were chosen by an intensive recruitment and selection process requiring high levels of motivation and early evidence of pedagogical skills. Again, because of the limited sample sizes, no significant correlation could be seen between hours of PD (80–266 h) and any of the indicators of teaching skills among the SKIL teachers.

Each year during the grant period, the program evaluator also rated eight to 10 PD sessions led by SKIL teachers, each one attended by 12–30 classroom teachers. She used the same rating indicators to judge these sessions as those applied to classroom observations plus an additional indicator that focused explicitly on pedagogical content knowledge of the SKIL teacher. According to the evaluator, over the 5-yr tenure of the program, professional development by the SKIL teachers came to be of high quality. Indeed, she ranked it as one of the strongest elements of the ESEP effort, giving ratings of 3–4 to eight of the 10 SKIL teachers she observed in the final months of the program. To quote from her 1999–2000 report:

While rough at the edges at first, over time most SKIL teachers have emerged as good deliverers of professional development … As a group, these lead teachers have shown remarkable growth. The lead teachers represent one of the most sustainable components of the program (Davis, 2001, p. 11).

Forming a Community of SKIL Teachers

By applying the tenets of Kozaitis' participatory reform program (Kozaitis, 1997), ESEP was able to unite the group of 54 SKIL teachers into a professional learning community. Judging by comments recorded during PD workshops and interviews, these teachers became vested in their role of promoting inquiry-based science. Their sense of empowerment also was evidenced by their willingness to expend extra time and effort for the LSC, and by other expressions of support. The SKIL teachers not only trained other teachers; they clearly believed that they were (and in fact, succeeded in becoming) change agents involved in participatory reform. The contribution to their sense of empowerment by the ESEP staff was acknowledged in a representative statement during an interview by a fifth grade SKIL teacher:

The SKIL teachers served as ambassadors for the ESEP program both within the school system and in the larger community. Moreover, their recorded expressions indicated a growing sense of enthusiasm, confidence in their science content knowledge, pedagogical knowledge, and abilities as professional developers. In response to a teacher survey question during year 3 of the program that asked, ”Have you noted differences with respect to teaching science since ESEP began?“, another fifth-grade SKIL teacher responded:

The Science Partners

Over 5 yr, the science partner program established a total of 1498 semester-long partnerships between classroom teachers and college students from the seven participating universities (Table 3). One sign of the success of this element of ESEP was the continuing demand. Every semester, the number of teachers applying for a partner was greater than the number of students available. Equally telling was the large number of teachers who applied for a second science partner after their first experience (78% of 406 teachers surveyed over six semesters). Further judgments were based upon teachers' expressed views of the undergraduates with whom they were partnered and by the opinions expressed by the science partners themselves about their own experiences. For example, interviews with teachers confirmed that the science partners supplemented and amplified what the teachers learned in their PD workshop about how to motivate and teach science to children. The teacher of one of the case-study partners addressed this by saying, “Well, they are all little scientists now. I think that having Anna here just naturally made [the children] more inquisitive about science” (Goebel, 2008, p. 95). Focus groups of teachers revealed that those who had a science partner were more accepting of the inquiry science approaches than those who did not. Teachers routinely reported that their science partners were able to help them with matters of science content. They typically used terms such as “enthusiastic,” “bright ideas,” “imaginative,” and “caring” to describe science partner contributions (Goebel, 2008).

When a fifth-grade teacher was asked to talk about the benefits to her pupils of having an ESEP science partner, she explained that the children had the opportunity to work with an experienced teacher, a young and enthusiastic scientist role model [the science partner], and the science kits, doing fun science three hours a week. In her own words: “Science used to be a subject that was textbook driven and wedged into the curriculum. It was required, boring and frustrating for many children … now [with my science partner] I have ventured outside of the book [and] use other resources.” And a typical comment from an LPS school principal confirms that her teachers had positive attitudes about the science partners:

The science partners themselves also expressed strong positive opinions regarding the benefits they had gained from participating in the program. Among the most striking were changes in their beliefs about the learning capacity of the children in their predominantly African-American classrooms (Goebel, 2008). There were no prepartnership beliefs expressed by the case students regarding their initial views of the learning capacities of their pupils. However, in subsequent comments recorded in later documents, two of these partners detailed their initial beliefs as they remembered them, revealing through their statements consistently low expectations of the pupils. These recollected belief reflections are listed in Supplemental Material SM-2 under the Initial beliefs column. These data from the students' own memories are consistent with the data collected in the fourth-week interviews, revealing preconceptions that were based mainly on deficit-model assumptions that the children had innate learning deficiencies or suffered from “underdeveloped skills.” A comment from Anna was revealing: “I expected to find poor, lower class, unintelligent, misbehaved, uninterested [pupils] …” (Goebel, 2008, p. 84). A typical opinion from Dawei's initial interview in the fourth week of her experience was that only a few of the children could readily grasp the material she was presenting. “I think sometimes these children need to have the information slowed down so they can digest it …” She found that she had to present material “really slowly” and was surprised when she “had a hard time getting [the children] to describe what they were seeing. Many of them needed prompting, like asking them what it looked like, what color was it. etc … ” (Goebel, 2008, p. 134). Similarly, Badra expressed the belief in her initial interview that the students would be “slow learners with poor memories” (Goebel, 2008, p. 103).

Later in their experience, interviews during the 11th or 20th week revealed profound changes in the science partners' views. For example, Dawei wrote in her journal of her excitement when she realized that the children were making conceptual connections.

To my amazement, they said (with little prompting) that it [organic matter] would undergo decomposition! And when asked what kind of organisms helped decompose dirt, they answered worms and mushrooms. That made me really excited because I thought that they would not make the connection at all … They are very bright … It is unfortunate that they didn't get to finish my chemistry lesson (Goebel, 2008, p. 137).

This is especially significant because Dawei had agreed with her partner-teacher in the fall that the pupils were not competent to work on “experiments” independently in groups. But, in her 20-wk interview, she commented, “Maybe we underestimated the [children] in how they would get organized and work in groups” (Goebel, 2008, p. 140).

In general, the undergraduates' early perceptions of their pupils' inabilities to observe, compare, describe, and conceptualize gave way, after working in their classes for several more weeks, to the view that the children were able to think critically about what they learned in science and to communicate their ideas. Over the course of their partnerships, the initial beliefs of the four case-study participants about children's learning difficulties became much more positive, although Anna tenaciously retained the idea that they were disinterested in learning and would not succeed overall. By the end of their ESEP experience, journal entries and interview comments like the following examples were common: “Well, they're great learners! They absorb materials like sponges and maybe they don't hold onto it for five years … but that's okay … (Badra; Goebel, 2008, p. 106); and ”I have a great deal of respect for them. Like they are all like each one has probably done something or explained something great once and surprised me … but they all can do it. I was really proud.“ (Chikara; Goebel, 2008, p. 121).

By the end of their partnerships, judging by comments from their interviews and other sources, all four case participants had distinctly or partially reduced their deficit-type beliefs, and all four had modified the social factors that they perceived to affect the abilities of the elementary pupils to learn science. Moreover, they explicitly attributed these new attitudes to their ESEP experience. These changes were corroborated by similar shifts in perspective expressed by members of the two science partner focus groups conducted at the end of the fall and spring terms (Table 4). Because it is not possible to note accurately the number of participants in a focus group who agree with a given comment, we list in the table the number of beliefs expressed by one or more participants. Of 20 comments made by one or more of the focus group participants that were coded as being relevant, four of these expressions corroborated the shift in thinking of the case students about their pupils' ability to learn science, and 16 resembled the case students' modified views of the factors affecting their pupils' ability to learn science. One focus group participant made a comment indicating that although the pupils learned science, their learning was probably short-term due to their general indifference to learning. This was similar to Anna's expression of her reservations about the pupils' overall disinterest in learning. However, it was clear that a majority of the focus group members were in agreement that their pupils had the ability to learn and were learning science. Moreover, it should be noted that none of the focus group participants said anything that would contradict our finding that college students generally gained much more positive beliefs about the learning ability of the pupils in their classes from their ESEP experience.

A postexperience (23rd week) interview statement by Dawei was representative of the views of many of the science partners:

For a summary of the changes in views of the four case-study partners, see Supplemental Material SM-2.

Another important effect of the ESEP experience was on the science partners' concept of science and how to learn it. In the intensive interviews with the case-study partners, three of the four made statements indicating recognition of gains in their own understanding of science as a result of teaching basic and very simple elements of the subject to the children in their ESEP classrooms. Typical was the following quote from Chikara:

Sustainability of ESEP

When the NSF grant terminated in 2000, the LPS administration began to modify the four major elements of the ESEP program. PD of mentor teachers, as administered by the LPS, was altered to include both mathematics and science and the new participants were termed MASKIL teachers, whereas the time devoted to PD was substantially reduced. Some of the original cadre of SKIL teachers were incorporated into that group; many soon left LPS to take positions at other school districts. The SMSC continued to function at a reduced level for several years, but kit distribution was voluntary; teachers could request a kit for a specific lesson but were under no obligation to do so. In 2008, the SMSC director retired and the facility was closed down. Students at Emory continue to be recruited into a much reduced science partner program through the Division of Educational Studies. The original ESEP orientation has been converted into a course called Introduction to Teaching Math and Science. Fifteen to 20 students each year receive excellent grounding in educational theory and pedagogy in that course, and each is placed as a science partner by individual arrangement with a willing teacher in a nearby school district.

DISCUSSION AND CONCLUSIONS

In recent years, as experience has accumulated with LSC and other federally funded teacher enhancement programs, it has become evident that carefully planned teacher PD that is sustained, intensive, focused on content, and occurs in the classroom can be successful. If delivered by experts, such PD can have a positive impact on instruction, which in turn may result in improved student achievement (Garet et al., 2001; Wayne et al., 2008). However, it is also agreed that the best way to demonstrate these effects is with random assignment study designs that allow causal inference by including an experimental and a control group and by analyzing results using both quantitative and qualitative research methods. Despite their importance, rigorous experiments of this type are rare (for review, see Wayne et al., 2008). As detailed by Laursen et al. (2007), they are notoriously difficult to organize and evaluate in real school settings. Furthermore, funds for implementing such study designs have generally not been included in the budgets of teacher enhancement programs. Despite these barriers, the positive impact of PD on elementary teachers has been documented using qualitative research methods, especially when that PD has included interventions that have brought members of the scientific community into contact with teachers, even for brief interventions (Bruce et al., 1997; Laursen et al., 2007).

Deficiencies in ESEP Planning and Implementation

The affective changes in classroom teachers' beliefs and self-confidence that are documented above may have been accompanied by improvements in their actual instructional skills, but no such changes could be measured from the surveys and interviews implemented by ESEP staff or from the yearly small random samples assigned for evaluator classroom observation. Classroom observation data from four years of the program (1996, 1997, 1999, and 2000) showed no significant correlation between hours of PD attended by the teachers (0–144 h) and the rating scores of instructional indicators given by the program evaluator during her classroom observations. Was this failure to observe significant improvements in classroom teaching a statistical problem: the lack of a power analysis that led to insufficient sampling? Or was it that the PD program that we designed was inadequate to the task of altering the instructional approaches of a large number of the LPS personnel? Moreno (2005) discusses common pitfalls that can cause partnerships to fail, wisely noting that the specifics will likely differ in each case. Although the ESEP PD was well-accepted by LPS teachers, we recognize several major challenges and design flaws that should have been foreseen by the authors or by the original developers of the national LSC effort.

• There was an unexpected turnover of teachers from year to year (up to 30% in some grades), that resulted from attrition and grade level reassignments. The result was that many teachers had to return for new grade-appropriate PD, thereby draining funds and consuming staff time.

• We designed and implemented workshops and other opportunities to familiarize LPS administrators with the work of the ESEP staff and SKIL teachers. However, only a small fraction of these officials took advantage of the sessions or gained an appreciation of the ESEP PD program. After the original district superintendent left to take a different position, a new superintendent was hired in the middle of the third year, whose major focus—influenced strongly by the Reading Excellence Act of 1998 and state mandates—was literacy. Without her strong support for an emphasis on science teaching, enthusiasm from the rest of the district administration waned throughout the final 2 yr. As a result, only a few vestiges of the project were institutionalized in the district.

• We were unable to deploy sufficient resources to engage parents or parent organizations. If ESEP had the capacity to involve what Lueder calls the ”missing families“ (Lueder, 1998, p. 4) in partnerships among parents, school personnel, and community organizations, we might have created more lasting support for the reform effort. However, by the time the program was operational, staff overload prevented us from reaching out to families.

• A systematic plan for sustainability should have been built into the ESEP LSC at its inception, including a step-by-step strategy to guarantee a gradual year-by-year transfer of funding responsibility from the NSF grant to the district, and identification of LPS personnel to take over the PD duties performed by ESEP staff. If this had been done, the cost and work force burdens to the LPS during and beyond the tenure of the NSF grant could have been clearly anticipated and agreed upon at the beginning.

• More funds should have been allocated to design and administer a systematic process of program evaluation and research that included collection of both qualitative and quantitative data. The ESEP LSC was designed as a teacher enhancement intervention, applying many of the ”best practices“ known in the mid-1990s. However, little attention was devoted to creating a sophisticated experimental design, according to tenets more recently advocated (e.g., Wayne et al., 2008) aimed at demonstrating or quantifying progress annually. To repeat one glaring example: the number of yearly classroom observations assigned by HRI to the program evaluators of each LSC should have been determined by a power analysis rather than by arbitrary criteria.

• The indicators of success in improving inquiry-based science instruction for teachers should have included more formal assessments of teacher knowledge and pupils' achievement scores in science.

Components of ESEP That Worked Well

Analysis of the variety of data sources described above revealed striking consistencies among the expressions of the different constituencies (classroom teachers, SKIL teachers, college students, and school principals) collected independently by ESEP staff and by the program evaluator, thereby increasing the reliability of the findings. It seems that the ESEP program had a positive impact on three of those groups: LPS classroom teachers, the cadre of specially trained SKIL teachers, and the undergraduates who participated as science partners.

Classroom Teachers. The importance of teachers' motivation and belief systems to the success of any reform effort has been well documented (Haney et al., 1996; Sweetland and Hoy, 2000). Effects on LPS classroom teachers could be judged from surveys and from opinions expressed by the teachers themselves and statements by the school principals and by the science partners. These sources indicated that participating in the ESEP PD sessions and having access to a science partner succeeded in

• improving the teachers' beliefs about their own abilities to teach science,

• enhancing the importance of science in their minds, and

• increasing the amount of time and effort they devoted in their classrooms to the subject.

SKIL Teachers. The impact of the program on the cadre of 54 SKIL teachers was greater than that on the classroom teachers in at least two respects: they came to believe that they were better teachers, and observations indicated that they actually were better teachers. In surveys, interviews, and written documents collected by ESEP staff and the program evaluator over the tenure of the program, we found SKIL teachers gradually expressing a much greater sense of empowerment, motivation, and group cohesion than classroom teachers. We conclude that the extended PD that incorporated Kozaitis' participatory reform program was especially successful in changing the attitudes of this cadre of mentor teachers. By ensuring that the SKIL teachers and their administrators became invested and motivated to participate in the reform effort, by actively engaging them in the design and implementation of the training, and by providing culturally sensitive experiences that offered a sense of empowerment and group cohesion, the evidence suggests that we succeeded in making an impact on their beliefs about teaching and pedagogy, and about their own abilities to teach.

Not only did the SKIL teachers' attitudes toward their teaching abilities change but also classroom observation ratings by the program evaluator for their ability to design and implement lessons and to help children in their own classrooms understand the scientific content were almost a full point above those for classroom teachers (Table 2). Because pre- and postassessments were not done, and no significant correlation of ratings with hours of training was evident, this difference could reflect the stringent selection requirements to become a SKIL teacher. However, judging by the SKIL teachers' own expressions in interviews and surveys, they believed that their enhanced abilities resulted primarily from the intensive training in inquiry science pedagogy they received during 2 yr of SKIL institutes. Most important, according to their many comments, were the sessions that offered in-depth exploration of the kits, the work with their science partners, and especially the opportunities they had to design and implement PD sessions for other teachers.

Science Partners. It is important to distinguish two aspects of the science partner program: the effects of the teaching experience on the science partners themselves; and the roles that these undergraduates played in the PD effort, i.e., their effects on the teachers with whom they were paired and on the children in their classrooms. The impact of the science partners on their partner–teachers was documented in statements of the teachers. Science partners brought science content knowledge, enthusiasm, and a sense of the importance of the subject to the teachers. We also found that with the orientation and training that they received, and because they were a sustained presence (∼30 h/semester), the science partners became important extensions of the broader PD effort into the classroom, reinforcing concepts of inquiry science pedagogy and participatory reform.

Regarding the reciprocal aspect of the partnership, much has been written about the beneficial effects on college students of becoming engaged with a local community through participation in a reflective service learning course (Eyler, 2002; Goebel, 2008). The ESEP experience had three important, documentable effects on the undergraduate science partners. It caused many of them to 1) alter their beliefs about the abilities of the children in their classes to learn, 2) develop insight into the lives of the children that were not based on deficit-model thinking, and (3) gain a greater understanding of science as a result of teaching basic and very simple elements of the subject to the children in their classrooms. These findings should serve as confirmation of the emerging view (Tanner et al., 2003; Dolan and Tanner, 2005; Laursen et al., 2007) that partnerships between members of the scientific and school communities may not only promote ”reform“ of K–12 science education, but such partnerships also can bring benefits that are bidirectional: both groups can profit.

FOOTNOTES

REFERENCES