Using Invention to Change How Students Tackle Problems

INTRODUCTION

Problem solving is arguably one of the most important skills for science students to develop. Like other university educators, we have often heard professors state that improved problem solving is the ultimate goal of their courses (White, 2009; DeHaan, 2009). However, we observe that these courses generally do not actively provide students with an opportunity to learn or practice these skills. In particular, science courses rarely develop the skills associated with approaching novel or unfamiliar problems: sense-making, problem analysis, and innovation. This is in striking contrast to the fact that these are skills that are routinely used by scientists when tackling novel problems.

In recent years there has been an emphasis on the development of teaching techniques and technologies aimed at engaging students and providing a much more tangible opportunity to practice problem solving. For example, the use of clickers (Freeman et al., 2007; Mayer et al., 2009; Addison et al., 2009) and Just in Time Teaching (Novak et al., 1999) are ways of increasing student engagement in the material. Techniques such as Problem Based Learning (PBL) and Process Oriented Guided Inquiry Learning (POGIL) use small collaborative groups as the primary means of instruction and learning (Prince and Felder, 2007; Eberlein et al., 2008). One feature all of these methods have in common is that they increase the amount of time students spend discussing questions and solving problems.

One method that has interested us is the use of “invention activities.” These activities are based upon the work of Daniel Schwartz involving what he termed “Inventing to Prepare for Learning” tasks (Schwartz and Martin, 2004). These types of activities ask students to solve problems that are seemingly unrelated to the class material and that are based on concepts that have not yet been covered. By working through these types of problems the students construct a mental framework that allows them to better understand the subsequent class material and to succeed in future learning and transfer tasks. Other successes using invention-type activities have also been reported (Roll et al., 2009; Kapur and Lee, 2009; Belenky and Nokes, 2009; Day et al., 2010).

In this study, we explored the impact of invention activities on student exam performance, learning, and problem-solving skills in a first-year biology class. We explored the effect of these types of activities on recall and understanding of the relevant biology concepts, as well as on students' ability to tackle unfamiliar biology-based problems. To measure student performance we used both near-transfer tests and medium/far-transfer tests (Barnett and Ceci, 2002). The tests included performance on exams, recall questions, and a problem requiring innovative thinking posed in an interview setting.

METHODS

Course Description

Biology 112: Unicellular Life is a first-year biology course at the University of British Columbia that serves as a first-year prerequisite for second-year cell biology and introductory biochemistry, which themselves serve as prerequisites to a large number of upper-level courses in biology, microbiology, and biochemistry. Close to 2000 students take Biology 112 each year. The course introduces students to a range of cell biology and microbiology concepts using mainly bacterial cells as the model system. Topics include cell membranes and transport, protein structure and function, DNA replication, transcription, translation, operons and gene regulation, mutations, gene transfer, and metabolism. The course is typically taught as a lecture course in a large class (>270 students), with three lecture sections running in the September and January semesters, and one lecture section during the summer semester. The course has no formal lab or tutorial component. The use of in-class questions and clickers was recently introduced into Biology 112. Multiple instructors are involved in teaching the course. Usually each section is taught by a single instructor.

Study Design

Students in the January 2009 semester of Biology 112 were invited to participate in either invention activities (IAs) or structured problem-solving activities (SPSAs) on a volunteer basis with the following conditions. 1) They would volunteer to participate and would be randomly assigned to attend either an IA session or a SPSA session. 2) The students would meet in classes of 24 once a week for 8 wk (over one semester), and the type of activity they completed would be consistent. 3) They would automatically receive points toward their quiz/assignment grade (in lieu of completing some of the assignments) providing they attended all activity sessions.

More than 50% of all students in the three lecture sections volunteered to participate in the activities. Students were randomly selected from the volunteer pool and assigned to one of the activity sessions. This ensured that the demographics of each test group (year, gender, and program) were a close match to the overall student population in the course. The final participant counts of those who started and completed the activities over the entire semester were as follows: 170 IA students, 170 SPSA students, and 91 students who volunteered but could not attend due to space limitations (control group). Eight IA sessions were held over Monday and Tuesday, and eight SPSA sections were held over Wednesday and Thursday. This facilitated our goal of having the students complete the IAs before seeing the related material during the lectures.

Within each activity session the students worked together on the activities in groups of three or four. Students chose their own groups during the first sessions and maintained those groups for the entire semester. Both the IA and SPSA sessions used a similar 50-min format. During the first 30 min the students solved the problems as a group and wrote their solution (or solutions) on a large piece of flip-chart paper. Typically each small group would spend the beginning of each IA discussing various possible solutions and then decide upon one or two solutions that they would develop. After the 30 min, there were 10 min of presentations by each group to the other students. The final 10 min were dedicated to a wrap-up talk. In the IA sessions, this wrap-up involved a mini-lesson introducing the related material the students would see in lecture and explaining how their inventions were analogous to the material. In the SPSAs, the wrap-up involved the facilitator presenting the correct solutions for the problem-set questions.

The facilitators for the each of the sessions were one researcher (J.L.T. or K.M.S.) and one graduate student. The graduate students were trained on a weekly basis for each of the IA or SPSA activities. Except in the event of a rare scheduling conflict, each session had the same facilitators for the duration of the semester. The facilitators were available to the students for discussion about possible solutions, although care was taken to provide only vague guidance whenever possible so that all solutions would be solely attributable to the each group's effort.

Activity Design

The IAs were designed to allow students to work through problems that are analogous to problems encountered by living cells, although on the surface the problems appear to have little to do with the related course material. These are generally based on the work of Daniel Schwartz (Schwartz and Martin, 2004). Prototype versions of the invention activities were tested with a group of students (87 volunteers from the three sections of the Sept 2008 Biology 112 class), and the lessons learned during the trial run were used to redesign three of the prototype activities and design five new activities for the January 2009 semester. We used the following criteria to design these activities:

• The activity should have a clear goal that can be accomplished by a group within 30 min.

• The context should be simple and familiar but should not appear related to lecture/course material.

• The activity should be engaging for the students.

• The activity should avoid scientific or confusing jargon.

• The activity should have multiple possible solutions (i.e., no single correct answer).

• The activity should use contrasting cases to focus students on the relevant elements and to introduce new features or structures in the system to which the activity relates.

• The activity should be challenging so that a group of students is likely to have significant conversations in arriving at a reasonable solution.

One critical lesson learned during development of the IAs was the importance of perceived context. The initial IAs developed used biology as the context, and students were asked to invent how a living cell might solve a problem. However, many students balked at the tasks. When asked why, the students responded that they didn't know enough biology to solve the problem. Often they assumed that they had forgotten critical information from their previous biology courses and therefore simply could not suggest a solution. This reluctance resulted in many groups being extremely slow to engage in the tasks, and even then only with coaching from the facilitators.

Guided by our initial experiences, the redesigned IAs used problems that did not appear to be related to the course material. These problems included such contexts as zoo exhibits, factory machines, and carnival games. Our experience is that an effective IA is analogous to a biological case but at the same time is a situation that does not depend on domain-specific knowledge. We found these activities to be far more successful than those used during the previous year in engaging the students quickly and without much coaxing.

The need for the students to approach the activity from an unbiased perspective implied that the timing of the IA with respect to the rest of the class was important. IAs were meant to be used as an introduction to the topics. Not only are students solving problems analogous to those cells face, but the students are doing so before seeing the relevant information in lecture. To accomplish this linkage, we timed the sessions so that they would precede the related lecture material by no more than a week.

As an example, our IA that related to membrane function and transport across membranes involved designing zoo exhibits for small rodents. Students were shown images of a squirrel, chipmunk, and a mouse along with the approximate mass and dimensions of each rodent. The students were then shown the diagrams of two exhibits, as shown in Figure 1, and tasked with designing a separating wall that could fulfill the requirements for each exhibit. In both cases, the design of the wall needed to be selective in which type of rodent could pass from side to side. Students were usually quick to realize that the wall in Exhibit 1 could simply use holes big enough for the mice but too small for the squirrels. Exhibit 2, however, required a much more involved design to allow the chipmunks to cross while keeping the mice on one side. Ideas for Exhibit 2 varied, but typically students would use differences in mass, length of the chipmunks (to reach higher), or jumping ability to allow the chipmunks to cross. More novel ideas, such as a tail scanner, were also developed. This activity would conclude with a mini-lesson on membrane structure and the need for transport proteins to selectively allow certain molecules to cross a cell membrane. The most recent versions of the IAs and a guide for using them are available on the Instructor Guidance resource page of the Carl Wieman Science Educative Initiative website (www.cwsei.ubc.ca/resources/instructor_guidance.htm).

As a comparison treatment, we also designed activities that required students to work through biological-based problems directly related to the course material. These SPSAs allowed students to apply material presented in lecture to more standard problems (similar to problems found in many textbooks). Whenever possible the SPSAs were designed to cover the same material that was used to design the IAs. When designing the SPSAs, we also made every effort to avoid presenting them as a series of questions. Rather, we presented the problems as a group of cases and asked one or two broad questions requiring the students to analyze each case or all cases together.

For example, the SPSA activity that paralleled the rodent zoo exhibit IA activity was designed to have students directly explore the structure of cell membranes and their function as a semipermeable barrier. The students were shown several graphs of experimental data (based on Grunwald, 1968) involving the escape of betanin from beet cells as shown in Figure 2. The students were instructed to consider the chemical structures of cell membranes and the relevant molecules in the experiments and to interpret the given data based on these structures. The students were permitted to use their textbooks and class notes whenever needed. The activity concluded with a summary review of membrane structure and permeable versus nonpermeable molecules.

Assessment Design

Students from the IA and SPSA sessions were compared by monitoring their engagement during the activity sessions and examining their responses on an opinion survey about the activities. In addition, students from the three different groups (IA, SPSA, and control) were assessed in two ways to test for differences in their learning. First, performances on the midterm and final exam for the overall course were compared, and second, think-aloud biology-based problem solving interviews were carried out with students from each group.

Engagement Monitoring

Session facilitators monitored each group to track the time off-task activity during the 30-min problem-solving portion of the activity session for six sessions. With two facilitators monitoring six small groups, we were able to approximate time off-task to the nearest half minute. We could judge when a group was off-task by simply listening to the conversation (we observed that our presence next to each table was rarely a signal for disengaged groups to return to the task, and many times a disengaged group would continue being off-task even with a facilitator nearby). Each group that disengaged from the activity was timed until they returned to the task. The students were not informed about this engagement monitoring.

Opinion Survey

Students completed a series of short-answer questions about the course and their experiences in the activity sessions. The responses were coded by two of the researchers, with greater than 95% interrater reliability for all categories.

Think-Aloud Interview

Students from the IA, SPSA, and control groups were randomly invited to participate in individual think-aloud interviews. A total of 10 IA students, 10 SPSA students, and 7 control students participated in the interviews. Care was taken to ensure that the students in these samples had a similar distribution of midterm scores.

The interviews were conducted by one researcher (J.L.T.), and each was recorded for subsequent transcription. Students were instructed to think aloud as they worked through the interview questions. The students were informed that although each question would require ≈10–15 min, they were free to use as little or as much time as they wanted to answer the questions.

In the interviews students were asked to answer two questions. The first was a question directly based on class material about the lactose operon. Students were shown a glucose-lactose diauxic growth curve and an unlabeled diagram of the lactose operon regulation and asked to recall as many details as possible. This question was meant as a warm-up to make the students comfortable with the think-aloud process and to provide a baseline for their incoming knowledge.

The second question was introduced by providing a brief review of the process of attenuation in the Gram-negative bacterial tryptophan operon that they had seen in class. They were then presented with a situation they had not previously seen, attenuation in Gram-positive cells, and asked how this process might work. Gram-positive bacteria are known to use different mechanisms for attenuating the tryptophan operon transcription (Gutiérrez-Preciado et al., 2007). It is highly unlikely that these mechanisms would have been encountered by a student taking first-year biology. In essence, this question is “impossible” because the students would have no way of knowing the correct answer and thus they would need to construct a plausible solution. A reasonable answer would be one which incorporated and applied the appropriate biological concepts. We expected that they might apply the general concepts discussed in Biology 112.

Transcripts of the second question were analyzed by three of the researchers using a six-point rubric for the quality of the response. The rubric was divided into three categories with up to two points per category and represented the different elements we felt a suitable answer should include (Table 1). When applying the rubric, we divided the ideas generated by the students into “solution threads.” We defined a solution thread as an uninterrupted flow of related ideas that the student applied to a solution and that contained a minimum of one of the concepts listed in the rubric. If a student back-tracked to a previous point in their solution (or returned to the question to start over) and tried a previous approach or a new approach, they were credited with a new solution thread. Each solution thread was scored separately using the rubric. Solution threads that involved applying unrealistic or unrelated concepts or fabricated new ill-defined terms were not included in the score of solution threads.

Exam Performance

Both the midterm and the final exam used multiple-choice questions exclusively. The average performances of students in the IA, SPSA, and control groups were calculated and compared.

RESULTS

Opinion Survey

Overall, our experience in using IAs as a teaching tool was positive, both for the students and for ourselves as teachers. From the perspective of the students, the activities were both useful and engaging. In responding to the question “What were the top three benefits of participating in the activities?” many of the students in both the IA and SPSA groups gave thoughtful answers about how the activities affected their learning and skills. As shown in Figure 3, the most common responses related to how the activities helped them improve their teamwork skills and helped them study the class material. The percent of responses in these categories was essentially the same for the IA and SPSA groups. Interestingly, the two groups differed in two categories. The IA students made significantly more comments relating to creativity, problem solving, and thinking skills (Pearson Chi-square, χ²(1) = 9.28, p < 0.01). The IA students also made significantly more comments relating to application of knowledge and making knowledge connections (Pearson Chi-square, χ²(1) = 9.29, p < 0.01).

Engagement Monitoring

It was clear that there was remarkably high engagement during the sessions, especially for the IA groups. The IA groups were off-task an average of 2.0 ± 0.1 min, which was significantly less than the 4.3 ± 0.5 min spent off-task, on average, by the SPSA groups (independent t test with nonequal variances, t(85.0) = 4.61, p < 0.01). In addition to the differences in time lost, we observed differences as to when the students were off-task. Typically, the IA students would be off-task at the very beginning of the activity (i.e., a delay in engaging with the activity); however, once they started the activity they generally remained on-task. In contrast, the SPSA groups tended to disengage multiple times during an activity, which we believe to be a result of the SPSA format. The nature of the SPSA activities meant that they could be broken down into specific tasks. We observed that disengagement often occurred whenever students completed one specific task in the activity. Because the invention activities could not be broken down into clear tasks, the students had less opportunity to find a point to disengage and “take a break.” We feel that this is a characteristic that should be kept in mind when designing future activities of either type.

Exam Performance

Our initial interest in using invention activities was to help the students develop a mental framework within which to incorporate the knowledge presented to them in class. It was our hope that this would translate into an increased performance on the exams. However, ANOVA comparison of the means of the exam scores revealed no significant differences in exam performance between any of the three test groups (overall midterm average = 67.5%; overall final exam average = 62.0%). After the midterm exam during the January 2009 semester, we became concerned that what the students experienced during the IAs (low-pressure group work) may not transfer over to a testing environment (higher-pressure individual work) and the type of questions being asked. For this reason, the individual interviews were used to test the students in a different setting.

Think-Aloud Interviews

The first question in the interview was simply scored on how many details of the lactose operon the students could recall (of a possible 33 that we looked for), and their percentage score was compared with their midterm scores. We found a strong correlation (0.75) between Question 1 scores and the midterm scores, indicating that Question 1 was a good measure of recall of the course material (at least as indicated by midterm scores).

The transcripts of the second question from the think- aloud interviews were analyzed for four components: the number of solution threads, the quality scores of the solution threads (based on the rubric), the time needed to start the first solution thread, and the time spent per solution thread.

The average number of solution threads generated by the students varied substantially between the IA, SPSA, and control groups (Figure 4). ANOVA comparison of the mean values and group contrasts showed the number of solution threads produced by the IA students to be significantly greater than either the control or SPSA groups (Table 2). We observed that students belonging to the control and SPSA groups tended to explore only one or two ideas, and once they had found a satisfactory idea they tended to remain fixed on that idea. In contrast, students from the IA group tended to use multiple solution threads. These students were much more likely to explore an idea and then return to an earlier point and explore another avenue, resulting in the proposal and development of multiple possible mechanisms. We noted that in some cases, the solution threads represented a return to a previous idea. In these cases the students seemed to be evaluating and comparing ideas in an attempt to judge which idea was better.

In terms of the quality of each solution thread, we found very little difference, on average, between the IA, SPSA, and control groups. The quality score for each student was taken to be the average of the rubric scores for his/her solution threads. The average quality scores for the three groups were 2.7 ± 0.3 for the IA students, 2.5 ± 0.4 for the SPSA students, and 2.3 ± 0.3 for the control students (ANOVA comparison of means showed no significant differences between the three groups). It is important to note that while the students from the IA group tended to use multiple solution threads, the quality of their threads was comparable to the average quality score of both the SPSA group and control group. The distributions of quality scores for all of the solution threads generated by the three test groups show that the IA students generated many lower quality threads but also more higher quality threads (Figure 5).

We also observed a difference between the IA, SPSA, and control groups in two time-based measures. We measured the average time required to begin generating the solution threads (from the moment the students finished reading the question) and the average amount of time spent on each solution thread (Figures 6 and 7). ANOVA comparison of the means and group contrasts showed the average time to start and the average time spent per solution thread to be significantly lower for the IA students as compared with the students in the other groups (Table 2). The total time spent by each student working on this problem was similar. The average durations were 11.6 ± 0.9 min for the IA students, 10.8 ± 1.2 min for the SPSA students, and 13.0 ± 1.6 min for the control students. Therefore, not only were IA students generating more solutions of equal quality, they were able to do so more rapidly than the students in the other groups.

DISCUSSION

We began this study to test the effectiveness of IAs in promoting recall and understanding of biology concepts, as well as their ability to tackle unfamiliar biology problems. The results suggest that the use of IAs affected the students, although not in all areas. While we did not observe an increase in general domain knowledge as measured by the exams, we did observe a difference in the way the students initiated the investigation of the medium/far-transfer biology problem. The IA students were more competent at engaging with the unfamiliar problem and quickly generating ideas.

We can offer three general hypotheses about the effects on the IA students:

1. The IA students are simply more persistent or are more motivated to attempt to solve an unfamiliar problem.

2. The IA students tend to be more adept in making connections between concepts, so that thinking of one related idea triggers connections to other related concepts that can be offered as possible solutions.

3. The IA experience makes students more comfortable in solving problems. They have practiced making sense of unfamiliar problems and being innovative, and they are more likely to apply this process to new problems.

Hypothesis 1 is likely to be the least significant as we did not observe an increase in persistence during either of the problems presented during the interviews. The three groups of students spent, on average, the same amount of time on the second interview question. The amount of time required to start generating their first idea may be related to their motivation, but this could also be explained by hypotheses 2 and 3. Neither hypothesis 2 nor 3 can be ruled out by the results shown here, and both may contribute the effect on IA students. Future work needs to be undertaken to determine which modes of thinking are being observed.

What is clear is that the IA students showed a different behavior in the way they think about and approach problems that are unfamiliar to them and to which there is no correct answer that they are expected to know. When faced with such a problem, the IA students seemed much better at being innovative and generating plausible solutions based on their working knowledge of biology. Because these solutions are based upon other ideas and concepts in biology, the students were transferring knowledge learned previously to the current problem.

In terms of the quality of the proposed solutions, there were no significant differences between the three groups, although in light of the IA and SPSA designs, this was not surprising. These activities provided an opportunity to practice solution generation but did not emphasize evaluating the proposed solutions for quality. This is especially true of the IAs, which encouraged the students to quickly develop novel solutions. These activities taught the students to use their existing knowledge to generate plausible solutions to unfamiliar problems, which is what we observed them doing during the interviews. Adding an evaluation component to the activities would give future students an opportunity to practice evaluating the quality of their proposed solutions.

With regard to the time required to generate the solutions threads, the literature suggests that improved problem-solving skills are generally not accompanied by a decrease in time taken. Indeed, when comparing novices to experts solving well-structured problems, experts spend more time analyzing and planning before generating a final solution (Schoenfeld, 1992; Martin and Schwartz, 2009). However, Dunbar investigated the behavior of scientists as they solved problems in real-world situations (Dunbar, 2000). He noted that scientists often generate hypotheses and solutions to problems very quickly using what he described as “analogical reasoning.” This type of problem solving involves drawing analogies from their knowledge of other biological systems to generate plausible hypotheses and solutions. In essence, scientists can quickly transfer their knowledge and apply it to the problem at hand. This is very similar to what we observed in this study with the IA students. When engaging with a realistic problem regarding an unfamiliar system, the IA students quickly generated plausible solutions based on their own knowledge of biology. This behavior is more expert-like than the behavior observed with the control and SPSA students.

Schoenfeld observed that novices tend to choose one solution strategy and focus on it for the entire time, even if it is unproductive (Schoenfeld, 1992). Our observations of the SPSA and control groups match this, as these students also tended to focus on a single solution thread without any increase in solution quality. In contrast, the IA student tended to engage, and generated possible solutions, more quickly. We argue that the decrease in time to engage with a problem is indicative of improvement and is more like the expert behavior observed by Dunbar. This difference in novice versus expert behavior is also something we observe in academic research groups. For example, a first-year graduate student would be slower in engaging with a problem and generating hypotheses than a senior researcher who would quickly engage to generate hypotheses that could then be tested.

Currently we are testing the use of IAs in large lecture courses (>250 students) with the students working on the activities during lecture time. This has worked well in that the students have been highly engaged and have generated the same ideas as those generated in the small section settings. We do not yet have measures of learning for IAs used in this setting but will collect this data in the near future. We are also carrying out work to further investigate the process by which IAs change the way students think about problem solving.

CONCLUSION

The goal of this study was to measure the impact of IAs on student learning and problem solving. In terms of biology-specific knowledge, the results of the exam performances and the interviews showed that students who participated in the IAs could recall and understand essentially the same amount of class-related knowledge as other students. However, what emerged during the interviews was that the IAs appear to improve the ability of students to be innovative in quickly generating quality solutions to a complex problem as compared with other students. As all other aspects of the course were identical for all of the students, our conclusion is that the IAs reshaped how these students approached unfamiliar problems.

Analyzing unfamiliar problems and generating multiple potential ideas is an essential process for research scientists in approaching unsolved problems and is also a valuable life skill for graduates facing an increasing complex society. This study provides evidence that early-phase problem-solving skills (sense-making, problem analysis, and innovation) can be taught through the use of IAs. When properly designed, IAs show a remarkably high level of student engagement and provide them with an opportunity to practice innovative thinking. While this work exclusively used first-year biology as the context, it is likely that IAs could be used successfully in other disciplines.

REFERENCES