Student Learning Dispositions: Multidimensional Profiles Highlight Important Differences among Undergraduate STEM Honors Thesis Writers

INTRODUCTION

There is increasing evidence that pedagogical practices that incorporate writing are effective in developing students’ conceptual and procedural knowledge (Gerdeman et al., 2007) and communication (Clase et al., 2010), scientific reasoning (Reynolds et al., 2012), and critical-thinking skills (Quitadamo and Kurtz, 2007). One function of writing-to-learn (WTL) assignments in science, technology, engineering, and mathematics (STEM) disciplines is to promote students’ shift from “knowledge telling” to “knowledge transforming” (Bereiter and Scardamalia, 1987) in the service of building their own knowledge. Furthermore, writing affords one of the most effective means for making thinking visible (Reynolds et al., 2012) and learning how to “think like” and “write like” disciplinary experts (Meizlish et al., 2013). These multiple and intersecting functions of teaching through writing come together when we focus on students and their development as scientific writers.

More broadly, shifting from a one-size-fits-all model to more tailored teaching strategies may allow us to better attend to the diversity of student populations. Research shows that students respond differently to writing-focused teaching practices, and the effectiveness of such practices both varies across STEM disciplinary contexts and depends upon learning objectives (Rivard, 1994; Anderson et al., 2015). Average changes in a population give us some information about individual students, but they can mask important variability in effectiveness of teaching practices among subpopulations. As a result, research is now focusing on identifying the effective ingredients of writing interventions to leverage their differential impact (Klein, 2015). These ingredients include both mediators (i.e., the intervention influences psychological or social processes that in turn affect learning) and moderators (i.e., personal characteristics or instructional practices increase or decrease the effects of the intervention on learning; Klein, 2015).

Research on tailored teaching practices, both writing focused and more generally, is beginning to concentrate on student characteristics beyond demographics (gender, race and ethnicity, socioeconomic status, etc.) and prior academic preparation, as exemplified by a recent report of the National Academies of Sciences, Engineering, and Medicine (2017). The report examined the role of personal competencies in student success. Competencies were defined as a range of attitudes, behaviors, beliefs, and dispositions that reside within the individual student and that may also be influenced by college environments and contexts. Promising evidence suggests that a number of intrapersonal competencies—including conscientiousness, a sense of belonging, a growth mind-set, utility goals and values, intrinsic goals and interest, and academic self-efficacy—are related to college success and are malleable in response to interventions. It was recommended that future research focus on the mechanisms through which these intrapersonal competencies operate to improve college success.

Individual dimensions of students’ intrapersonal competencies—particularly motivation, self-efficacy beliefs, and epistemic beliefs—have been shown to change in response to teaching and affect student learning (reviewed in detail in the following respective sections). What has been lacking, however, are a conceptual framework of the intersection of these intrapersonal competencies and methodological approaches to consider their combined influence on students’ learning and development. The emergence of the construct of dispositions—characteristic behavioral tendencies that are not innate but acquired—provides a conceptual, integrative framework positioning relatively modifiable personal characteristics in the middle of the continuum from fixed traits (e.g., the way someone one might be described as gentle or caring) to more momentary, state-related behaviors (e.g., the way someone might be miserable or elated).

Dispositions refer to a relatively enduring tendency to behave in a certain way (Shum and Crick, 2012). Recently, some intrapersonal competencies related to educational success have been conceptualized as thinking dispositions, which are separate from cognitive abilities and include open-mindedness (Facione, 2015), beliefs, and motivations (Ritchhart and Perkins, 2005). Thinking dispositions are considered “psychological characteristics of the reflective mind that underpin rational thought and action” (Stanovich et al., 2011, p. 790). “They are telling us about the individual’s goals and epistemic values, and they are indexing broad tendencies of pragmatic and epistemic self-regulation” (Stanovich, 1999, p. 158). Dispositions are more malleable than cognitive capabilities and can be improved in the short term by instruction (Stanovich, 1999).

Analogous to thinking dispositions, other intrapersonal competencies have been considered learning dispositions, habits of the mind that constitute one’s characteristic orientation toward learning (Shum and Crick, 2012). Learning dispositions refer to the way in which learners engage in and relate to the learning process (Costa and Kallick, 2008). These habits of mind are seldom performed in isolation; rather, clusters of behaviors are drawn forth and used in various situations (Costa and Kallick, 2008). Larose and Roy (1995) proposed that learning dispositions comprise three interrelated personal systems in college students—the belief system, the behavioral system, and the emotional system—and reported that dispositions toward learning are closely associated with college students’ academic performance. For example, the college student who believes that a person must be gifted in order to succeed will tend to invest less time studying and may exhibit signs of anxiety at exam time.

Weinstock and colleagues (2017) have recently argued that “a central goal of education in all domains of learning should be to develop dispositions such as willful engagement in knowing and honest justification of knowledge beliefs” (p. 285). Student learning dispositions, conceptualized as relatively modifiable and potentially mutually influencing, also provide a window into understanding individual differences in response to specific pedagogical practices. To generate learning outcomes that are sufficiently fine-grained to guide instruction, Ewell (1989) suggests that one must determine: 1) which practices work best, 2) with whom, and 3) under which specific contexts. Research in STEM that examines the combined roles of students’ learning dispositions in relation to writing-focused instruction can achieve this goal.

In this work, we consider motivation, self-efficacy beliefs, and epistemic beliefs to be students’ learning dispositions with respect to writing. Our research focused on six capstone courses in STEM disciplines (biology, chemistry, and neuroscience) across four institutions (Duke University, Morgan State University, University of Minnesota–Twin Cities, and University of North Carolina–Chapel Hill), representing a diversity of student characteristics, course contexts, and institutional settings. Each of these courses was designed to scaffold the undergraduate honors thesis-writing process while the students, who were beginning to develop skills and beliefs about scientific inquiry, conducted independent research outside class; we study the courses collectively because of the shared contexts and learning objectives. We focus on the genre of the undergraduate honors thesis as the rhetorical context in which to study students’ scientific reasoning and writing, because we view the process of writing an undergraduate honors thesis as a form of professional development in the sciences (i.e., a way of engaging students in the practices of a community of discourse). We previously found that structured capstone courses—designed both to scaffold the undergraduate thesis-writing process and to promote both metacognitive knowledge and regulation—can improve writing and reasoning skills in biology, chemistry, and economics (Dowd et al., 2015a,b; Reynolds and Thompson, 2011) and may promote critical-thinking skills (Dowd et al., 2018).

Here, we expand these research efforts by using student dispositions to capture individual differences in the context of writing-focused pedagogical practices. The dispositions of motivation, self-efficacy beliefs, and epistemic beliefs, all of which have been shown to change in response to teaching, are reviewed in the following sections.

Motivation

Most generally, motivation refers to the reasons that one has for taking a particular action. Motivation ranges from intrinsic motivation—doing something because it is inherently interesting or enjoyable—to extrinsic motivation—doing something to satisfy an external demand or to achieve a reward. A person’s position on this continuum reflects the degree to which motivations emanate from the self (Ryan and Deci, 2000). Intrinsic motivation is largely a process whereby individuals come to value an activity by making it a more central part of the self (Vallerand, 1997). Evidence indicates that a sense of competence fosters intrinsic academic motivation over extrinsic (Faye and Sharpe, 2008), and intrinsic motivation is generally associated with higher levels of engagement and learning (Ryan and Deci, 2000). Thus, by fostering competence and mastery in the university classroom, educators not only foster intrinsic motivation, but they also promote the establishment of a strong and positive sense of self.

Intrinsic and extrinsic motivation can be further delineated with regard to goals. In the domain of intellectual achievement, two types of goals have been identified: performance and mastery (Archer, 1994). Those who hold performance goals “are concerned primarily with demonstrating their ability (or concealing a perceived lack of ability), which is shown to best advantage by out-performing others, particularly if success is achieved with little effort” (Archer, 1994, p. 431). In contrast, those who hold mastery goals “want to develop their competence on a task or increase their understanding of a subject” (Archer, 1994, p. 431). Orientation toward mastery goals has been associated with using more effective learning strategies, particularly metacognitive monitoring practices, whereas orientation to performance goals has been associated with use of surface learning strategies, such as rote memory (Archer, 1994). Thus, intrinsic motivation is associated with mastery-driven goal orientation, and extrinsic motivation is associated with performance-driven goal orientation.

Self-Efficacy Beliefs

Self-efficacy refers to people’s belief that they “can produce desired effects by their actions” (Bandura, 1997, p. 3). Beliefs about efficacy are malleable, derived from experience, and domain specific. Self-efficacy beliefs have consistently been shown to have both a direct and an indirect impact on academic performance (Bandura, 1997); for example, self-efficacy has been found to impact performance through influencing cognitive and motivational processes and the engagement and persistence of learning efforts (Bartimote-Aufflick et al., 2016).

Of the four main sources of efficacy beliefs that have been postulated—performance accomplishments, vicarious experiences, social persuasion, and physiological reactions (Bandura, 1977, 1997)—performance accomplishments are particularly significant among university students (Bartimote-Aufflick et al., 2016). In reviewing 64 articles involving university students published since the year 2000, Bartimote-Aufflick and colleagues (2016) found a strong association between self-efficacy and student learning outcomes (reported in 92% of the studies) and also between self-efficacy and self-regulation, metacognition, intrinsic motivation, and learning strategies. Seven studies provided evidence that self-efficacy improves “arguably as a result of completing a course or participating in a particular learning activity” (Bartimote-Aufflick et al., 2016, p. 1924).

The consistent findings that higher self-efficacy beliefs are associated with better academic performance suggests that pedagogic interventions can be targeted to increase self-efficacy to enhance academic performance (Di Giunta et al., 2013). Inquiry-based approaches (often involving student-focused, hands-on, and collaborative activities), in particular, provide opportunities for students to gain mastery experiences that are pivotal for the development of self-efficacy (Jansen et al., 2015).

Epistemic Beliefs

Epistemic beliefs refer to an individual’s assumptions about the nature of knowledge and its justification. Development of epistemic beliefs progresses from an absolutist view of knowledge, in which knowledge is assumed to be certain and known by authorities, through a relativistic view, in which knowledge is seen as constructed and uncertain with multiple valid perspectives, to the evaluative view, in which knowledge is seen as continuously evolving and justified on the basis of reasoning and evidence (King and Kitchener, 2004). Those at the evaluativist level view knowledge not as facts or opinions, but rather as judgments that, in turn, “require support in a framework of alternatives, evidence, and argument” (Kuhn, 2005, p. 32). In the sciences, the evaluativist viewpoint is critical for the development of new knowledge; although some knowledge may be sufficiently stable that it is inefficient to think of it as provisional, one must be comfortable operating at the evaluativist level to recognize when this is the case.

Epistemic beliefs have been found to influence various dimensions of academic motivation, learning, and achievement (Schommer-Aikins, 2004; Muis et al., 2006), including the interpretation of knowledge (Kardash and Scholes, 1996); evaluation of information and the process of making meaning (Baxter Magolda, 2004); critical thinking and use of strategies (Schraw et al., 2006); and academic performance, conceptual change, and text comprehension (Bendixen and Rule, 2004). As with self-efficacy beliefs, evidence suggests that epistemic beliefs affect students’ learning through the generation of internal standards and expectations, allowing them to monitor efforts and strategies to fit different learning tasks (Bromme et al., 2010). For example, college students with more sophisticated epistemological beliefs about learning and knowledge were more likely to use educationally productive self-regulated cognitive, behavioral, and motivational strategies (Paulsen and Feldman, 2005, 2007).

The evidence that epistemology can affect learning suggests that “epistemological interventions could lead to better conceptual learning” (Lising and Elby, 2005, p. 381). Evidence such as this has led to a call for more systematic survey of epistemologies used in a range of contexts to inform science education (Russ, 2014).

Study Design and Research Questions

As described earlier, the intrapersonal competencies of motivation, sense of self-efficacy, and epistemic beliefs have all been shown to affect various learning outcomes. To conceptualize these dimensions together, we consider students’ motivation to write an undergraduate honors thesis, sense of self-efficacy (both in science and in writing), and epistemic beliefs (primarily in relation to the certainty and simplicity of knowledge and authority as a source of knowledge) as composite elements that contribute to their identity as scientists. We consider these competencies to be learning dispositions and, in particular, learning dispositions with regard to writing.

The goals of the current observational study are both conceptual and methodological. In addition to influencing other measures of student learning, the intrapersonal competencies of motivation, self-efficacy beliefs, and epistemic beliefs have been shown to change in response to teaching. Conceptually, we propose that these competencies constitute learning dispositions and serve as learning outcomes in their own right. Furthermore, we propose that patterns of interrelationships among these specific learning dispositions are likely and that differing constellations (or learning disposition profiles) may have meaningful implications for instructional practices. Methodologically, we explore the utility of cluster analysis for generating meaningful learning disposition profiles. The specific instructional practices that serve as at the context focus of for this observational study are the six capstone courses described in the Methods that are designed to scaffold the undergraduate thesis-writing process and promote metacognition.

This observational study addresses four research questions: 1) Do the learning dispositions of students’ motivations to write an undergraduate thesis, self-efficacy beliefs regarding science and writing, and epistemic beliefs change as students participate in a capstone writing course structured to scaffold the undergraduate thesis-writing process and promote metacognition? 2) What is the relationship among these learning dispositions? 3) Can clusters of students, representing meaningful profiles, be formed from the interrelationship of students’ relative positions along these learning dispositions? 4) What is the relationship of student learning dispositions to scientific reasoning in thesis writing?

METHODS

Participants

The study sample was drawn from students who participated in six capstone thesis-writing courses across four universities between 2012 and 2015. Of the 490 students who participated in these courses, our study sample was composed of 472 students (96.3%) from Duke University (n = 176), Morgan State University (n = 26), the University of Minnesota (n = 178), and the University of North Carolina at Chapel Hill (n = 92). Academic and demographic background data were collected for participating students, including students’ grade point average (GPA), major/concentration, reported ethnicity, and reported gender. The research was approved by the respective IRBs of Duke University, University of Minnesota, University of North Carolina, and Morgan State University.

Not all students participated in all parts of the study; 384 students (81.4%) completed the precourse survey, 350 students (74.2%) completed the postcourse survey, 314 students (66.5%) completed both the pre- and postcourse surveys, 434 students (91.9%) shared a thesis for assessment, and 395 students (83.7%) shared academic and demographic background data through the registrar. We have complete data for 271 students (57.4%). Among students for whom we have academic and demographic background data, our sample is 42.3% male, 14.0% identify themselves as an underrepresented minority, and the average GPA is 3.70.

At Duke University, students participated in one of the following: an optional one-semester course for thesis writers in the Biology Department, a mandatory one-semester course for thesis writers in the Chemistry Department, or an optional one-semester course for thesis writers studying neuroscience. At Morgan State University, students participated in a two-semester course for thesis writers across multiple disciplines (including biology, chemistry, engineering, and psychology); this course was mandatory for participants in specific externally funded programs, but optional for other thesis writers. At the University of Minnesota, students participated in a two-semester course that is mandatory for thesis writers in the honors program of the College of Biological Sciences. At the University of North Carolina, students participated in a one-semester course for thesis writers in the Biology Department.

Data Collection

A survey was developed to assess student learning dispositions. It was administered to students online within the first 2 weeks of each course (precourse survey), and then again within the last 2 weeks of the course (postcourse survey).

The survey was a compilation of items, many of which were drawn from previously validated measures of specific dimensions. Although this study focuses only on motivation, self-efficacy, and epistemic beliefs, the survey includes additional dimensions that will be analyzed in future research (see Supplemental Materials). Pilot data indicated response rates suffered as survey length increased, so the survey was designed to be completed within ∼15 minutes. Students completed the survey online, either in class or on their own.

Students’ motivation for undertaking a thesis was assessed by asking students to distribute 100 points among five suggested reasons based on how strongly each item connected to their own reasons. This “constant sum type” question allows us to assess motivation on a relative scale (learn which reason or reasons are most important to individual students) while permitting more nuance and flexibility than a simple ranking. These items were framed to capture the degree to which students were mastery oriented or performance oriented, using terms that would resonate with students completing an undergraduate honors thesis. For example, “to demonstrate my abilities and competencies to […] prospective employers” indicates performance orientation, while “to learn how to […] generate and apply knowledge” indicates mastery orientation. By focusing on relative motivation, we can assess students on a shared scale, without quantifying or comparing students’ overall motivation; this is an important feature of our approach, but also a limitation (we cannot say whether one student is “more motivated” than another student). For analysis, we quantified motivation as the points allocated to the mastery-oriented items, as designated in the survey. We also asked students to describe their reasons for undertaking a thesis in their own words; these free-response data are not included in the following analysis.

Students’ self-efficacy in writing was assessed using 15 items drawn from the “self-regulatory efficacy for writing” factor of the measure developed by Zimmerman and Bandura (1994). In this and other measures, the scale on which students respond was slightly modified so that it would be consistent with other similar survey items. Students were asked the extent to which they feel they can [do some task], and they responded on a five-point scale from “not at all” to “superior.” Cronbach’s α, which quantifies the reliability of the measure, was 0.86 on the precourse survey and 0.85 on the postcourse survey. Students’ science/discipline-related self-efficacy was assessed with 20 items, 11 of which were drawn from Kardash’s (2000) assessment of self-efficacy in science practice measure (the other items were additionally adapted or introduced because they resonated with the skills developed during the semester). Cronbach’s α was 0.93 on the precourse and postcourse surveys. Exploratory factor analyses indicated there were single underlying factors for self-efficacy in writing and science, respectively. Therefore, we interpret the measures to be unidimensional, and we quantify writing and science self-efficacy as the averages of the respective items.

Students’ disciplinary epistemic beliefs were assessed using 12 items from a survey implemented by Hofer (2000). The items represent two specific factors: eight items relate to students’ beliefs about the certainty and simplicity of knowledge (α = 0.74 at precourse and 0.82 at postcourse) and four items relate to students’ beliefs about external authority as a source of knowledge (α = 0.58 at precourse and 0.64 at postcourse). Exploratory factor analysis of the 12-item set revealed two factors that corresponded to the factors identified by Hofer (2000). Accordingly, we quantify beliefs about the certainty and simplicity of knowledge and beliefs about external authority as a source of knowledge as the averages of the respective items (items of opposite valence were inverted before averaging).

Finally, the extent to which students exhibited science reasoning in their thesis writing was assessed using a thesis assessment protocol originally developed for biology (Biology Thesis Assessment Protocol [BioTAP]), and subsequently applied to chemistry and economics, as described in prior studies (Dowd et al., 2015a,b; Reynolds and Thompson, 2011). The thesis assessment involves evaluation of nine specific criteria. Each thesis is independently evaluated by two raters, who have been trained to assess each criterion, and they subsequently discuss the thesis to form a consensus evaluation. Prediscussion agreement ranges from high 60% to high 70%. Postdiscussion agreement is 100%. In this work, we focus on the partial sum of the five BioTAP criteria that most strongly correspond to the single factor of science reasoning in writing (questions 1 through 5; α = 0.69). This decision was made based on prior studies in which the partial sum was shown to most strongly capture the single dominant underlying construct (Dowd et al., 2018). We explored the nine individual criteria as a matter of thoroughness, with the possibility that a particularly strong relationship could emerge and shed further light on any observed relationships, but we have not included those analyses here.

Statistical Analyses

To address question 1, regarding whether students’ learning dispositions changed across the various thesis-writing courses, we used Student’s t test to compare students’ pre- and postcourse scores on each of the measured dispositions. Effect sizes of comparisons among continuous variables are measured using Cohen’s d. Additionally, beyond analyzing mean differences, we examined the distribution of individual changes; specifically, we report the fraction of individuals who changed by more than 1 SD. To address question 2, regarding the relationship among the learning dispositions, we calculated the Pearson correlations among them for both pre- and postcourse responses. To address question 3, regarding the formation of meaningful profiles of student learning dispositions, we conducted hierarchical cluster analysis using Ward’s method for clustering and Euclidean squared-distance measures. Cluster analysis was carried out separately on the precourse survey responses and postcourse survey responses. To address question 4, regarding the relationship of student learning dispositions to scientific reasoning in thesis writing, we conducted multilevel multiple linear regression analysis to determine the amount of variance in students’ science reasoning in writing explained by student learning dispositions, both alone and above and beyond that accounted for by demographic and academic background characteristics.

In hierarchical cluster analysis, extra steps are required to establish whether the clusters are stable or spurious (i.e., to validate them). In keeping with established methods (Curry and Thompson, 1982, 1985), we conducted one-way absolute intraclass correlations (ICCs) among clusters that emerged from all (pre- or postcourse) observations and “subclusters” formed from randomly selected samples that each contained half of the observations. ICC allowed us to quantify agreement of mean values among the clustering variables (i.e., the personal dimensions of interest in this work). A cluster was considered “valid” (i.e., nonspurious) if all three ICCs (pooled and random 1, pooled and random 2, random 1 and random 2) were above 0.7. Recognizing that even this threshold can produce errors for any specific random subclusters, we performed 1000 iterations of this validation procedure for both pre- and postcourse clusters. This does not provide absolute cutoffs, but it provides quantitative insights into whether a particular cluster is likely to be spurious, stable but weakly differentiated, or quite stable and well differentiated. We report these validation percentages with each cluster to indicate our confidence in their stability.

RESULTS

Sample Analysis

As noted in the Methods, some students did not submit or share all components of the data for this study. We chose not to limit analyses to only the most complete sample because some of our work—particularly the cluster analyses—depends critically on having as much statistical power (i.e., as many observations) as possible. With each subsample, we compared the distribution with the larger sample from which it was drawn across all criteria using Student’s t test, so that we would be aware of potential bias in our results. Across all cases, we found no statistically significant differences between subsamples and the larger samples except between the smallest, complete subsample (n = 271) and the largest sample (n = 434) on the criteria of thesis assessment (i.e., the partial sum of BioTAP); in this case, the mean of the partial sum of BioTAP is larger in the subsample (μ₄₃₄ = 21.1, μ₂₇₁ = 21.8, t = 3.14, p = 0.002). We recognize this as a potential source of bias, which we address in the Discussion section.

Do Average Learning Dispositions Change as Students Participate in Capstone Writing Courses?

When looking at individual learning dispositions, we found statistically significant, positive change in mastery-driven motivation (p = 0.003), but its effect size is relatively small (0.15; Table 1 and Figure 1). Changes in self-efficacy beliefs regarding both writing and science were positive and statistically significant (p < 0.001 for both), and the effect sizes are very large (1.13 and 1.26 for writing and science, respectively; Table 1 and Figure 2). Changes in measures of epistemic beliefs were not statistically significant (Table 1 and Figure 3). Figures 1–3 not only help visualize the scale of the changes we observed, they also highlight individual students’ variability, even when the average does not change from pre- to postcourse. For example, 16% of students reported that their sense of self-efficacy in science increased by more than 1 SD from the mean.

What Are the Relationships among Learning Dispositions?

We examined correlations among learning dispositions and found that the two self-efficacy dimensions (related to beliefs about writing and science) and the two epistemological dimensions (related to beliefs about the certainty/simplicity of disciplinary knowledge and the source of disciplinary knowledge) were strongly related to one another in both the precourse and postcourse surveys. The correlations are reported in Supplemental Tables 1–4. Specifically, the correlation between students’ writing self-efficacy and science self-efficacy beliefs was 0.714 (p < 0.001) in the precourse survey (Supplemental Table S1) and 0.743 (p < 0.001) in the postcourse survey (Supplemental Table S2). Similarly, the correlation between students’ beliefs about disciplinary knowledge (“authority as source” and “certainty and simplicity”) was 0.453 (p < 0.001) in the precourse survey (Supplemental Table S1) and 0.553 (p < 0.001) in the postcourse survey (Supplemental Table S2). Other relationships were either weaker or inconsistent among pre- and postcourse measures.

Although some precourse measures are correlated with different postcourse measures (e.g., precourse mastery-oriented motivation is positively associated with all postcourse self-efficacy and epistemic beliefs; Supplemental Table S3), none of the correlations are as strong as the relationships within individual measures from pre- to postcourse (e.g., precourse mastery orientation is most strongly associated with postcourse mastery orientation: r = 0.596, p < 0.0001). Moreover, changes from pre- to postcourse are not strongly correlated between personal dimensions, except between respective self-efficacy and epistemic beliefs (Supplemental Table S4). The relatively weak correlations among the learning dispositions suggests that individual students differ in their relative positions across these five dimensions.

Do Profiles Emerge from the ­Interrelationship of Students’ Learning Dispositions?

Using cluster analysis, we identified three specific cluster profiles of students that emerge from the otherwise undifferentiated pre- and postcourse distributions (Figure 4 and Supplemental Figure S1). The only cluster that emerged from the precourse survey responses is the nonefficacious evaluativist cluster. This cluster comprised 32.3% of students, and these students could be characterized as relatively evaluativist in their epistemic beliefs (certainty: t = 5.17, p < 0.001; authority: t = 2.79, p = 0.006), lacking in self-efficacy beliefs (writing: t = −9.75, p < 0.001; science: t = −11.14, p < 0.001), and being slightly more mastery oriented (t = 2.09, p = 0.037) when compared with the precourse undifferentiated group. Two clusters emerged from the postcourse responses. The efficacious absolutist cluster includes 8.3% of respondents and is characterized by strongly absolutist epistemic beliefs (certainty: t = −15.86, p < 0.001; authority: t = −10.96, p < 0.001), motivation that is not mastery oriented (t = −2.14, p = 0.033), and sense of self-efficacy that, albeit lower than the postcourse undifferentiated group (writing: t = −4.00, p < 0.001; science: t = −4.99, p < 0.001), is still high. The 12.9% of the students described as nonefficacious uncommitted are characterized by relatively low self-efficacy beliefs (writing: t = −15.72, p < 0.001; science: t = −15.19, p < 0.001), neutral beliefs about knowledge (certainty: t = −1.76, p = 0.080; authority: t = −1.36, p = 0.174), and a tendency not to be mastery oriented in their motivation (t = −4.91, p < 0.001) when compared with the postcourse undifferentiated group.

There were groups of students, one group within the precourse distribution and one group within the postcourse distribution, with values across the five dimensions that did not meet the criteria for a cluster. We have characterized these two groups as undifferentiated pre and undifferentiated post.

Although 67.7% of precourse respondents and 78.9% of postcourse respondents fall into these undifferentiated groups, when we focus on transitions among students who completed both pre- and postcourse surveys, only 55.4% fall exclusively within undifferentiated groups (Table 2). In other words, nearly half of the students are transitioning into or out of the clusters we described. Students’ transitions from pre- to postcourse differ marginally by cluster (χ² = 5.97, p = 0.05; Table 2); notably, there are slightly more students transitioning from nonefficacious evaluativist to efficacious absolutist than one would expect if all transitions were equally likely. Clusters do not exhibit statistically significant differences in academic or demographic attributes.

Do Student Learning Dispositions Explain Variation in Scientific Reasoning in Thesis Writing?

To examine the relationship of student learning dispositions to scientific reasoning in thesis writing, we conducted multilevel multiple linear regression analysis to determine the amount of variance in students’ science reasoning in writing explained by student learning dispositions. When we controlled for multiple demographic and academic background characteristics, including campus and department/discipline treated as random effects, the relationship of students’ learning dispositions to science reasoning was not significant (Table 3).

DISCUSSION

The goals of the current study were both conceptual and methodological. We conceptualized specific personal dimensions—motivation for undertaking a thesis, self-efficacy beliefs about science and writing, and epistemic beliefs—as constituting learning dispositions and examined how they change during a thesis-writing course, relate to one another, and relate to students’ science reasoning in writing. Methodologically, we sought to delineate the interrelationship of separate learning dispositions that constitute meaningful constellations or profiles.

In this study, the educational practice was a one- or two-semester structured course designed to scaffold the thesis-writing process and promote metacognition, with a diverse sample of students participating in six courses housed primarily in three STEM disciplines across four universities. Our within-group study design does not tell us whether observed changes are explicitly a result of the courses, but our results shed light on potential causal relationships that can subsequently be tested with controlled experimental studies.

Average changes in learning dispositions from pre- to postcourse were statically significant and positive for motivation (slightly) and self-efficacy beliefs (strongly); some, but not all, students improved along these dimensions. The observation that both self-efficacy beliefs and mastery motivation increased is consistent with the notion that a sense of competence fosters intrinsic over extrinsic academic motivation (Faye and Sharpe, 2008). By highlighting the percentages of students who increased their scores by more than 1 SD from the mean, we go beyond the analysis of mean differences to examine change at the level of individual students; we view this as a primary contribution of this study. This analytic approach not only helps to visualize the scale of the changes we observe, it also highlights individual students’ variability, even when the average does not change from pre- to postcourse.

While mastery motivation has been associated with higher academic achievement, a recent integrative review indicated that performance and mastery goals can be equally adaptive when they are pursued together out of a desire to succeed instead of a desire to avoid failure (Wormington and Linnenbrink-Garcia, 2017). Therefore, the slight change in mastery-orientated motivation that we observe may reflect the fact that, although being motivated by mastery is positive and productive, it is not the strongest and most obvious marker of successful learning.

The large pre- to postcourse changes in self-efficacy are noteworthy because of the increasing evidence that self-efficacy beliefs are associated with achievement. Self-beliefs regarding one’s writing skills have been found to play an important role in the skills of formulating and expressing ideas; this occurs in part through changes to self-regulation skills and, more specifically, setting higher standards for self-evaluation (Zimmerman and Kitsantas, 2007). From those studies reviewed by Bartimote-Aufflick and colleagues (2016) that employed causal modeling (e.g., structural equation modeling or path analysis), self-efficacy was found to have a positive indirect effect on achievement by increasing deep-processing strategies, persistence, and learning strategies; and by decreasing surface approach, challenge-threat evaluations, and state anxiety. Furthermore, a recent study provided support for the hypothesized reciprocal relationship between self-efficacy beliefs and performance (Villafañe et al., 2016).

The correlational analyses (the second research question) and the cluster analyses (the third research question) offer two different ways of exploring relationships among students’ learning dispositions. The former is focused on the dimensions, while the latter is focused on the students. Supplemental Tables S1 through S4 show that the different learning dispositions are related to one another to varying degrees at the beginnings and ends of the courses; however, except for the respective pairs of self-efficacy and epistemic beliefs, the dimensions are not so strongly related that they could be considered a single construct. In other words, the absence of strongly significant relationships in our findings is in keeping with the assertion that these learning dispositions are distinct. Interestingly, in spite of changes from pre- to postcourse and relatively small correlations, all of the correlation coefficients are numerically (if not significantly) positive. In other words, mastery-oriented motivation, strong self-efficacy beliefs, and more evaluativist epistemic beliefs are variously related or unrelated to one another, but never negatively related. We also note that correlations show that individual learning dispositions at precourse relate most strongly to those same dispositions at postcourse; this provides some support for instrument reliability, though we are certainly not measuring test–retest reliability directly, as the pre and post surveys were administered months apart from one another.

The relatively weak correlations among the learning dispositions suggest that individual students differ in their relative positions across these five dimensions; the absence of strong correlations lends credence to the exploration of student-focused cluster analyses. We employed cluster analyses to delineate meaningful groups, or types, of students based on their relative positions in the constellation of the five learning disposition dimensions. The results of the cluster analyses, conducted separately on the precourse and postcourse survey responses, revealed five meaningful constellations of student profiles of learning dispositions. We have conceptually characterized two of these profiles (one precourse and one postcourse) as undifferentiated, meaning that the profiles include students who 1) do not constitute a meaningful cluster and 2) do not consistently fit into one of the other identified cluster groups; therefore, they were “undifferentiated” within the sample. However, as shown in Table 2, only 55% of students fit exclusively within these profiles.

In contrast to the groups of undifferentiated profiles, the three validated clusters reflect specific patterns of learning dispositions. Conceptually, from a science education perspective, the nonefficacious evaluativist is arguably a very desirable incoming profile; students seem primed to benefit from the “snowball effect” between self-efficacy and performance (i.e., improvement in each also benefits the other) demonstrated by Villafañe and colleagues (2016), and the seemingly intransigent epistemic beliefs (based on average changes) are already quite strong. Conceptually, the final two clusters, which emerged postcourse, are arguably less desirable endpoints. In particular, the nonefficacious uncommitted cluster reflects an underdeveloped set of beliefs about science. The extent to which these empirically derived learning disposition profiles have meaning in terms of differential response to specific educational practices or associations with other learning outcomes will need to be determined through subsequent experimental studies. While we do not have direct evidence of it, we speculate that this group may have “checked out” at this final, capstone stage and may be at risk of leaving their STEM disciplines. Perhaps students exhibiting such a prolife could be identifiable earlier in the undergraduate career, which might offer instructors and departments the opportunity to consider interventions and educational practices that foster evaluative beliefs and mastery orientation. Students in the cluster termed efficacious absolutist might also benefit from such interventions, though perhaps they are less at risk of leaving STEM. These students may be finding success as suggested by Wormington and Linnenbrink-Garcia (2017), wherein performance and mastery motivation are adaptive when pursued together.

We are encouraged by this initial effort to employed student-focused cluster analyses to delineate meaningful groups, or types, of students based on their relative positions in a constellation of five learning disposition dimensions. The three derived clusters, as well as the undifferentiated profiles, provide a means for more holistic consideration of the role of personal competencies in student learning in response to different pedagogical approaches. Going forward, we envision continued efforts to derive additional meaningful learning disposition clusters.

The role that epistemic beliefs (e.g., absolutism vs. evaluativism) play in differentiating these clusters warrants further investigation. In particular, because evaluativism is foundational in scientific reasoning—and, therefore, either implicitly or explicitly, viewed as a desirable learning outcome—strongly absolutist views seem problematic. As shown in Table 1, changes in measures of epistemological beliefs from pre- to postcourse were not statistically significant. This indicates a negligible net flow toward evaluativist thinking, but epistemic beliefs are not necessarily less malleable than self-efficacy beliefs and motivation among individual students. Academic disciplines differ in subject matter, ways of knowing, cognitive processes and methods involved in the construction of knowledge, and epistemological foundations (Phillips and Burbules, 2000; Muis et al., 2006). With more intentional efforts to teach the epistemology of a discipline, instructors may be able to introduce net positive epistemic changes. One of the ways that students develop sophisticated intellectual capacities is through being enculturated into the discourse of an academic discipline or field. Learning to “think like” and “write like” a member of an academic discipline (Meizlish et al., 2013) requires that students learn the content knowledge of that discipline and learn how practitioners use the conceptual tools that characterize the work in that discipline. It also requires that faculty teach the epistemology of the discipline manifested in the inquiry, discovery, and communication processes and engage students in the production of knowledge in that discipline. Studies are currently in progress to examine changes in epistemic beliefs in introductory STEM courses.

In considering how these learning dispositions relate to other, explicit learning outcomes, the capstone courses enabled us to examine the relationship of learning dispositions with students’ science reasoning in thesis writing. However, with background variables included in the model, the relationship of students’ learning dispositions with students’ science reasoning was not significant (Table 3), and science reasoning in writing did not vary across clusters.

In light of the studies cited earlier that report strong and positive relationships between learning dispositions and performance measures, the absence of such a relationship here may seem surprising. Several explanations are possible. It may well be that this learning outcome, scientific reasoning in writing, is not well aligned with these personal dimensions. Student learning dispositions, as we have characterized them, may be more likely to influence other learning outcomes (such as concept attainment, retention in STEM disciplines, etc.). It is also possible that other student populations—for example, introductory and intermediate-level students—would exhibit stronger, different, or more distinct learning dispositions that differ in relation (or in response) to writing-focused educational practices. For example, two types of WTL assignments, in particular, have been shown to improve learning in STEM disciplines: assignments that focus critical reflection on epistemic beliefs (Bangert-Drowns et al., 2004) and assignments that engage students in formulating a reasoned argument (Kelly et al., 2000; Kelly and Takao, 2002; Bradley, 2001; Lerner, 2007; Armstrong et al., 2008). It would be of interest to explicitly examine how student learning dispositions mediate and moderate the impact of these types of WTL practices on learning outcomes, as described by Klein (2015). The optimal way to tests hypotheses about mediation and moderation is through between-group designs that enable comparisons between different WTL instructional practices and learning outcomes; we have such studies in progress.

LIMITATIONS

The major limitations of this study are those associated with observational studies. That is, there was no effort to differentiate among the educational practices implemented in the six STEM capstone courses across the four institutions or among the groups of students taking the respective courses. Rather, this was a natural, noncomparative study of changes in students’ learning dispositions and scientific reasoning in writing while engaged in these capstone thesis-writing courses at their respective institutions. Without random assignment and a comparative experimental design, causal attributions for observed changes and outcomes cannot be made.

Another possible limitation of this work is that we did not explicitly test the construct validity of our survey. We assumed that the items drawn from pre-existing, previously validated constructs are also valid for our population, and we relied on the face validity of the items we developed. Measurement of reliability (Cronbach’s α) and exploratory factor analysis suggest that items from pre-existing instruments are internally consistent, and we have no reason to believe that they should have measured something different here. With regard to the measurement of motivation, future work will be required to go beyond face validity. We did not administer our survey and corresponding source surveys to any sample of students in our targeted population, because we prioritized including the maximum number of students in our study. We believe this approach is sufficiently robust for the analysis presented here, though these assumptions could certainly be tested in future work.

This study aimed to develop and implement an approach for delineating student learning profiles that reflect different constellations of student learning dispositions. The results were promising, in that cluster analysis revealed three conceptually distinct profiles of learning dispositions. However, subsequent experimental studies will be needed to determine the extent to which these empirically derived learning disposition profiles are related to differential response to specific educational practices or associations with other learning outcomes.

Another limitation in the current study relates to the context. Capstone thesis-writing courses are an appropriate context in which to measure scientific reasoning in writing as a primary learning outcome, but the efforts to generate and evaluate student learning dispositions and profiles will need to be extended to a range of specific educational practices with multiple learning outcomes in courses for introductory and advanced students more broadly.

CONCLUSION

There is increasing evidence that pedagogical practices that incorporate writing foster the development of a wide range of skills and types of knowledge. Writing affords one of the most effective means for making thinking visible and learning how to “think like” and “write like” disciplinary experts. In this observational, within-group study, we considered the students’ personal dimensions of mastery-oriented motivation, self-efficacy beliefs, and epistemic beliefs to be representative of their learning dispositions with respect to writing, and we went beyond analyzing simple changes in averages along single dimensions to build a more sophisticated understanding of students as complex, multidimensional learners. Cluster analysis revealed three conceptually distinct student profiles of learning dispositions. These profiles reflect different constellations of student dispositions and offer the potential to enable identification of students who may benefit from instructional efforts intentionally directed at promoting epistemic and self-efficacy beliefs and mastery motivation. The extent to which these empirically derived learning disposition profiles, and other profiles that may be derived in the future, have meaning in terms of differential response to specific educational practices or associations with other learning outcomes will need to be determined through subsequent experimental studies.

The methods and principles driving this work are not limited to writing-focused interventions and assessments. Learning dispositions are likely to be components of a student’s overall sense of scientific identity, which, in turn, is likely to both be responsive to various pedagogical initiatives. This work should contribute to that broader discussion.

ACKNOWLEDGMENTS

REFERENCES