Recent Research in Science Teaching and Learning

WHAT IS RELEVANCE?

Priniski, S. J., Hecht, C. A., & Harackiewicz, J. M. (2018). Making learning personally meaningful: A new framework for relevance research. Journal of Experimental Education, 86(1), 11–29.

Relevance is an important concept in biology education research. Relevance is a critical design element of course-based undergraduate research experiences (Corwin et al., 2015), and short relevance interventions increase student performance in biology courses (Tibbetts et al., 2016). Moreover, cultural relevance of biology course content (Chamany et al., 2008; Siritunga et al., 2011) and of research experiences (Jackson et al., 2016) is a growing area of interest. But what is “relevance” and is the way relevance is used and defined across biology education and other educational contexts the same? In this synthesis article, Priniski and colleague establish a unifying framework for relevance in the context of motivation and motivational research. They then explore this framework’s utility for explaining why relevance effectively increases motivation in three common motivational theories and in several common educational interventions. This unifying framework can help researchers and instructors design and deploy more efficacious relevance interventions.

Priniski and colleagues define relevance as simply “a personally meaningful connection to the individual.” They highlight two elements of relevance. First relevance is personal; it is a subjective construct that will vary in the degree the stimulus (such as a particular course topic) is connected to individuals. Next, relevance is meaningful, and the meaning can vary between people or even within a person over time. This variation allowed the researchers to create a spectrum of relevance, defining three distinct but overlapping regions on the spectrum: personal association, personal usefulness, and personal identification. They propose that these regions can be used to characterize the type of relevance elicited by an intervention or course topics. These regions vary in how personally meaningful the relevance is, and thus the strength of motivation the relevance elicits should vary as well.

According to Priniski and colleagues, the least meaningful form of relevance is personal association. This form of relevance involves the stimulus being related to something that a student values. For example, if a student has a strong interest in sharks, then talking about the sinking of the SS Indianapolis in a military history class may have a personal association to the student, because many of the shipwrecked sailors lost their lives to sharks and a monologue about the event was featured in the shark movie Jaws. The next proposed level on the relevance spectrum is personal usefulness. This form involves the stimulus being perceived to help a person achieve an important goal. A student who loves sharks may find a marine biology course to be relevant, because it will help him or her become a shark biologist. The final level of relevance is personal identification. At this level of relevance, instead of seeing a stimulus as connected to oneself, it is seen as part of one’s identity. For someone who identifies as a shark lover, being able to identify different shark species is an important part of his or her identity, so learning that skill in a class would have high personal meaning. Personal identification should lead to the greatest impact on motivation. Thus, instructors interested in increasing motivation should promote increasingly personally meaningful types of relevance, and the paper reviews existing interventions that do just that.

INSTRUCTOR MINDSET AND PERFORMANCE GAPS

Canning, E. A., Muenks, K., Green, D. J., & Murphy, M. C. (2019). STEM faculty who believe ability is fixed have larger racial achievement gaps and inspire less student motivation in their classes. ScienceAdvances, 5(2), eaau4734.

Mindset is a person’s beliefs about how fixed or malleable a particular human characteristic is (Dweck, 2008). It is most commonly used to describe beliefs about intelligence. People with fixed views of intelligence believe it is an innate quality that cannot be changed. People with growth views of intelligence believe that it can change and develop. Work has been done relating the impact of student mindset on performance and persistence in science, technology, engineering, and mathematics (STEM) fields (Cutts et al., 2010; Degol et al., 2018), but the mindset of their teachers and mentors may also impact these important outcomes. Initial work on the mindsets of instructors suggests that those with fixed mindsets are more likely to offer well-intended but demotivating advice to struggling students (Rattan et al., 2012). Faculty mindset and its potential impact on students may be particularly problematic for students from historically underserved backgrounds because of larger societal stereotypes about who is intelligent and who belongs in STEM. In the current paper, Canning and colleagues test this hypothesis by exploring the impact of faculty mindset about intelligence on achievement gaps in their college science classes.

Researchers collected data on 2 years of STEM courses across 13 departments serving more than 15,000 undergraduates. All of these courses were conducted at one university, so the context of this study likely matters: the university was a selective public school with predominantly white students (only ∼11% of students in this study were classified as underrepresented minorities [URMs]). The mindsets of the instructors in this study were identified through use of a reduced form of a mindset survey that consisted of two items addressing beliefs about intelligence. Using multilevel models to account for students being nested in courses, researchers explored the connection of faculty beliefs to student course performance.

Canning and colleagues found that all students performed worse in courses taught by faculty with more fixed mindsets about intelligence. This effect was more extreme for URM students. The URM–white achievement gap was almost twice as large in the classes of faculty with fixed mindsets compared with faculty with growth mindsets. To further explore how faculty mindset could produce student performance differences, the researchers explored student responses on end-of-course surveys and found several differences. Students in courses with fixed-mindset faculty felt less motivated to “do their best work” and reported that these faculty were less likely to use teaching methods that promoted “learning and development.” The use of these demotivating teaching practices mediated the impact of faculty mindset on performance gaps. Although Canning and colleagues could not conclude specifically what students meant by these practices, it could be that these faculty use less active learning. This finding was observed in another study on faculty mindset that demonstrated that life science faculty with a fixed mindset were less likely to believe the evidence for active learning and less likely to employ it (Aragón et al., 2018).

AN INTERVENTION TO INCREASE ENGAGEMENT BY REDUCING PERCEIVED COSTS

Rosenzweig, E. Q., Wigfield, A., & Hulleman, C. S. (2019). More useful or not so bad? Examining the effects of utility value and cost reduction interventions in college physics. Journal of Educational Psychology (advance online publication). https://doi.org/10.1037/edu0000370

Short psychosocial interventions are becoming increasing popular across STEM disciplines (Miyake et al., 2010; Jordt et al., 2017; Canning et al., 2018). These interventions are frequently developed based on various psychological theories of motivation. One of the most well-researched interventions in biology is the utility-value intervention. This intervention was developed from expectancy-value theory. Very briefly, expectancy-value theory posits that humans engage in a task when they believe they can be successful at it and value it, and when that value is greater than the costs they perceive for their engagement (Wigfield and Eccles, 2000). Thus, there are three aspects that an intervention could focus on to increase engagement with a task: self-efficacy, value (as the utility-value intervention does), and cost. In the current paper, Rosenzweig and colleagues develop and test the effectiveness of a short cost intervention for increasing course and exam performance in a physics context. In addition, they compare the cost intervention’s effectiveness with the effectiveness of the more common utility-value intervention.

The short cost intervention involved students reading quotes from other students, reflecting on these quotes, and then writing their own messages to future students taking their current course. To develop these quotes, researchers administered an open-ended survey to students in physics classes to identify challenges they had experienced. Researchers selected and modified quotes focused on costs. They then piloted these modified quotes with physics students to identify anything that was boring, inaccurate, or did not sound like something a student would say. Quotes were also piloted with experts in motivational theory to be sure they accurately reflected the construct of cost from expectancy-value theory.

Although the study has a fairly small sample size per treatment group, overall it is well designed. First, both interventions (n = 52 and 48) and the control (n = 48) were given to different subsets of students in the same physics class, which controls for environmental factors that might impact performance (exams, instructor, course content, etc.). Next, researchers measured student responses on three constructs that are known to influence motivation and thus were predicted to be influenced by the interventions: students’ belief that they can do well in the course (competency beliefs), the usefulness students perceive in the course (utility value), and the costs students perceive from engaging in the course. By measuring these, researchers can evaluate whether any were influenced by the interventions and also test whether it was changes in these constructs that caused any changes observed in exam or course performance. Finally, the timing of the interventions allowed researchers to use student performance on the first exam as a control for a student’s demonstrated ability in physics. These elements allowed the researchers to tell a fairly complete story about the cost intervention.

The two interventions (utility and cost) were each delivered twice in the semester. The first dose occurred a week after the first exam. The second was a week after the second exam. After this second dose, students also completed a survey measuring their competency beliefs, utility value, and perceived costs. These perceptions were measured again on the last homework assignment of the semester. After the semester was over, researchers also collected final course grades and exam scores.

Researchers found that both interventions increased student exam and course performance over the control condition. Interestingly, they found no difference in impact between the two interventions: both increased student exam performance by ∼8% and course performance by ∼11%. The small sample size in this study makes it challenging to evaluate whether the lack of difference between interventions was real or just a product of the small sample size. Both interventions had the largest effects for students with lower performances on the first exam.

Surprisingly, the researchers did not find that the utility-value intervention increased the utility students perceived in the course nor did the cost intervention reduce the cost, but both did increase competency beliefs. When researchers tested whether the increase in competency beliefs explained the increase in student performance, they found no significant results. However, for initially lower-performing students, the increase in subsequent exam scores was partially explained by their competency beliefs and their perceptions of cost.

Together, these results suggest that developing interventions based on different aspects of a theoretical framework can be effective; in this case, the value and cost aspects of expectancy-value theory. The study also shows that interventions may function through mechanisms that researchers do not initially predict. The utility and cost interventions were designed to impact utility beliefs and costs, respectively, but actually seemed to impact competency beliefs. Thus, collecting data on the constructs a researcher believes will change is useful for understanding why an intervention works.

REFERENCES