Initial Development and Validation of the Plant Awareness Disparity Index
Abstract
Plant awareness disparity (PAD, formerly plant blindness) is the idea that students tend not to notice or appreciate the plants in their environment. This phenomenon often leads to naïve points of view, such as plants are not important or do not do anything for humans. There are four components of PAD: attitude (not liking plants), attention (not noticing plants), knowledge (not understanding the importance of plants), and relative interest (finding animals more interesting than plants). Many interventions have been suggested to prevent PAD, but without an instrument shown to demonstrate valid inferences to measure PAD, it is difficult to tell whether these interventions are successful or not. We have developed and validated the Plant Awareness Disparity Index (PAD-I) to measure PAD and its four components in undergraduate biology students. The study population was 74.32% female and 69.08% white, indicating that the need for further analysis is necessary if this instrument is to be used in a more diverse student population. We collected validity evidence based upon text content, response processes, and internal structure. Our findings demonstrate that our instrument generates reliable inferences regarding PAD with a Cronbach’s alpha of 0.884 and a six-factor structure that aligns conceptually with the four components of PAD.
INTRODUCTION
Plant awareness disparity (PAD, formerly known as plant blindness) is the tendency not to notice plants within one’s environment, leading to naïve and anthropocentric points of view, such as plants are not important to humans, are boring, or do not do anything (Wandersee and Schussler, 1999; Parsley, 2020). Some of the problems associated with PAD include lack of support for conservation of plants (Balding and Williams, 2016), prejudice among biology teachers against plants and teaching about them (Hershey, 1993), zoochauvinism, lack of representation of plants in the media, and even plant neglect in biology textbooks (Hershey, 2002; Brownlee et al., 2021). PAD does not mean that people are incapable of seeing plants, but rather that humans group plants together into a green mass that is often visualized as a backdrop for animals. For example, Schussler and Olzak (2008) noted that university students recall more animal names than plant ones, even if they are equally nameable. This phenomenon is a result of a visual cognition bias: human visual systems evolved to notice things that move and/or look like us and therefore do not perceive plants as distinctly as animals (Balas and Momsen, 2014). Thus, PAD leads to a negative impact on students’ reasoning about the importance of plant life to the biosphere and human affairs (Hershey, 2002; Uno, 2009).
PAD is composed of four components: attention, attitude, knowledge, and relative interest (Parsley, 2020). Attention is the most notable component in the literature and refers to how much attention students pay to plants in general. Attitude is how students feel about plants, particularly in educational settings. Knowledge refers to understanding the importance of plants. Relative interest indicates how interesting students find plants compared with other organisms, namely animals.
Previously, interventions surrounding PAD have focused largely on the knowledge component of PAD, attempting to help students understand more about plants in an effort to improve their levels of PAD (e.g., Frisch et al., 2010; Ward et al., 2014; Krosnick et al., 2018). This approach can be categorized as a knowledge-deficit model, which has been used extensively in the field of science communication. The knowledge deficit model refers to the idea that, if scientists merely engage with the public more to teach them about science, the public will better understand and support it. However, this model has largely been unsupported in the science communication field (Besley and Tanner, 2011). Because of this, we hypothesize that the knowledge deficit model is also insufficient in the field of PAD. Additionally, there is a distinction between this knowledge deficit model and the knowledge component of PAD. The PAD knowledge component refers specifically to the understanding of how plants are important to the biosphere and to human affairs, rather than more general content knowledge regarding plants. This is an important distinction, because instructing students on general knowledge regarding plants (using the knowledge-deficit model) may not be enough to improve their knowledge of why plants are important.
This specific type of knowledge (or lack thereof) regarding why plants matter to humans and the biosphere plays an important role in student understanding of plant-related socioscientific issues such as climate change, genetically modified organisms, food security, biofuels, and plant conservation. For example, PAD contributes to a lack of knowledge about how illegal wildlife trade affects plant conservation (in addition to animals), which often leads to a lack of protections for plants (Margulies et al., 2019). Krishnan et al. (2019) called for more food- and agriculture-related efforts to improve PAD, which has gotten worse over time due to ever-increasing urbanization. Amprazis and Papadopoulou (2018) have also called for better coverage of plants in primary school curricula to highlight their importance to human welfare and biodiversity. Using an instrument to measure PAD in pre-service teachers may help satisfy the requests to include more plants in primary school put forth by Amprazis and Papadopoulou (2018). Pre-service teachers exposed to a PAD survey will not only be familiar with the term, but they will also have a better understanding of their own levels of PAD, which will allow them to adjust for their PAD when creating and teaching school curricula (Hershey, 1993, 2002).
Many suggestions have been proposed to address PAD in multiple types of learning environments: implementing a Pet Plant Project, in which university students were asked to grow an unknown plant from seed; using a research-centered botanical curriculum; probing college students’ botanical sense of place; using a hands-on outdoor education program; and using local street trees to bring student attention to plants (Wandersee et al., 2006; Frisch et al., 2010; Ward et al., 2014; Krosnick et al., 2018). Other approaches include: highlighting teachers’ enthusiasm to increase student interest in plants, capitalizing on students’ interest in herbal drugs and medicinal plants, and seeking out a knowledgeable and friendly plant mentor (Wandersee, 1986; Wandersee and Schussler, 2001; Strgar, 2007; Fančovičová and Prokop, 2011; Pany et al., 2019). While these studies all provide valuable insight into how PAD works and what interventions have been tried thus far, it is difficult to determine how effective they are when a tool to measure PAD does not exist.
Previously, the Plant Attitudes Questionnaire (PAQ) developed by Fančovičová and Prokop (2010) was used to measure attitudes toward plants specifically, but no instrument exists to measure the entirety of PAD: attention, attitude, knowledge, and relative interest, as described by Dr. Elisabeth Schussler (personal communication). Additionally, this questionnaire was only validated in Slovakian students of 10 to 15 years of age and was specifically intended to help determine if having a garden improved PAD (then called plant blindness).
To address the lack of a more well-rounded instrument, we have developed the Plant Awareness Disparity Index (PAD-I). The PAD-I is designed to evaluate undergraduate students’ level of PAD based on the four components of PAD. The development of this instrument is also a way to determine whether these four theorized components can operate as subscales within the PAD-I, and whether these components are supported by the data collected. Here, we describe the development and validation of the instrument using three sources of validation evidence: evidence based on test content, evidence based on response processes, and evidence based on internal structure (AERA, 2014; Reeves and Marbach-Ad, 2016). Validity evidence based on test content is often established by evaluating a new instrument for content representativeness. This is typically accomplished by asking subject matter experts to systematically review the instrument. Validity evidence based on response processes is about the relationship between the underlying contruct(s) of an instrument and the actual response behavior engaged in by the test taker. An example would be to qualitatively interview students about the construct that an instrument is intended to measure, and to compare the interview participants’ responses to their instrument scores. Validity evidence based on internal structure is the empirical (and statistical) process of determining whether or not the construct(s) within an instrument reflect its statistical structure. This is typically determined via factor analysis, item response theory, or Rasch analysis. We have opted to use factor analysis to test for this type of validity evidence. However, our sample was mostly white and female, which means further study in more diverse student populations will be necessary for use in other samples. An overview of our survey design and validation process is demonstrated in Figure 1.
FIGURE 1. Overview of the development and validation process of the PAD-I. After identifying relevant constructs of PAD in the literature (attention, attitude, knowledge, and relative interest), we proceeded with initial development of survey items. We then brought these items to Dr. Elisabeth Schussler, one of the original creators of the idea of PAD (formerly referred to as plant blindness). Once these items were vetted by Dr. Schussler, we piloted them with a small sample of students. While we present these methods in a linear manner, it is important to note that this entire process was iterative and that multiple methods of validity evidence collection were used throughout the development process at various times (e.g., collecting test content validity with Dr. Schussler before and after factor analysis).
THEORETICAL COMPONENTS OF THE PAD-I
While no formal theory of PAD currently exists, our conversations with Dr. Elisabeth Schussler helped us to expand and characterize the previously theorized four components of PAD (Parsley, 2020). We reached out to Dr. Schussler because of her previous experience and publications in the field of PAD (formerly referred to as plant blindness) and her considerable expertise in the area (Wandersee and Schussler, 1999, 2001; Schussler and Olzak, 2008). We begin by presenting visual attention theory that explains the relationship between the human visual system and our inattention to plants within our environment. We follow this by presenting the link between inattention to plants and disinterest in them, followed by how this disinterest also affects attitudes. Finally, we connect this lack of interest (and the presence of negative attitudes) with a lack of motivation to learn about plants, which lays the groundwork for these four components and how they are connected both theoretically and practically within our instrument.
Attention
The attention component of PAD stems directly from the research on visual attention. Attention is a selective process, because there are strict limits on our capacity to process visual information (Lamme, 2003; Carrasco, 2011). Stimuli compete for limited attentional resources, and as such, the higher-level stimuli (such as movement) often win out (Carrasco, 2011). Plants often suffer in this scenario, because they do not visibly move of their own volition on the same timescale as animals do. Our attention is engaged by animals in a way that disadvantages plants. This phenomenon may be similar to “inattentional blindness,” a failure to notice unexpected objects when attention is otherwise engaged (Most et al., 2005).
However, this does not mean that people do not see plants at all. There can be perception without awareness, which is exactly what happens in the case of PAD (Merikle et al., 2001). Plants are perceived by the brain, but they are placed in the background of the visual field in service of other organisms, namely animals (Parsley, 2020). For example, Balas and Momsen (2014) demonstrated that, while students could still recall plant images (which demonstrates some level of perception) they could not do so to the same extent as they could with animal images (indicating more awareness for animals).
Whatever the reason behind this phenomenon may be, there is strong evidence that, while people may perceive plants, they are not aware of plants. This can have consequences for the other three components of PAD as well: relative interest, attitudes, and knowledge.
Relative Interest
The relative interest component of PAD is the idea that people are not as interested in plants as they are in animals (Parsley, 2020). This particular component has been documented in the literature somewhat, though not as commonly as the attention component. Originally, Nichols (1919) described this phenomenon as being a part of the development of animal-centric general biology courses (Brownlee et al., 2021). These courses were cited as the reason why students often considered the study of biology to be synonymous with zoology (Nichols, 1919). Strgar (2007) noted that, while students often are less interested in plants than animals, they also tend to be more interested in medicinal and recreational plants. Lindemann-Matthies (2005) found that, while Swiss children originally favored mammals over plants, with time and increased attention, children also grew to appreciate plants as well (even those that were not exotic or conspicuous). Wandersee (1986) noted that interest and motivation are two factors that may play an important role in determining whether students learn about plants or not.
Attitude
While attitude is not as commonly explored in PAD literature as is attention, attitude is widely known to be a component of PAD and why students tend to prefer to learn about animals instead of plants. Wandersee (1986) was one of the first people to denote and describe students’ lack of positive attitudes toward plants at the middle school level. One approach demonstrated to improve attitudes toward plants in students at the K–12 level is hands-on outdoor opportunities that allow students to interact with plants (Fančovičová and Prokop, 2011). Additionally, intentional visual attention can cause an increase in intensity of emotions, suggesting that attention and attitude are likely related in PAD (Mrkva et al., 2019).
Knowledge
The final component of PAD that has also been explored in the literature is knowledge. While general plant knowledge is more often referred to as botanical literacy, knowledge that is specifically related to why plants are important to humans and the biosphere is what is referred to as the knowledge component of PAD. Wandersee et al. (2006) studied this phenomenon in community college students as they introduced an activity designed to help students reflect on why plants are important to them. The botanical sense of place worksheet allowed students to revisit their past emotions and experiences surrounding plants, which also prompted them to recall why they view plants as being so important to their lives and to the humans in general as well (Wandersee et al., 2006). This approach was also used by Frisch et al. (2010) to help science educators at the K–12 level better understand why plants are important as well.
Most literature surrounding PAD and plant knowledge is about other types of botanical knowledge (aside from knowledge regarding why plants are important) and how educational interventions can improve it in students at various levels. However, research in science education as a whole also supports the relationship between any type of scientific knowledge and students’ attitudes and interests. Krapp and Prenzel (2011) noted how important it is to leverage students’ pre-existing interests and how they overlap with science curricula to encourage students to remain in science, technology, engineering, and mathematics; if this is an important idea in science education, it is likely also very important for PAD, as plants and botany are already at a disadvantage in biology education efforts.
INITIAL DEVELOPMENT OF THE PAD-I
To develop the PAD-I we considered each of the four components of PAD individually and created items that would address each component. We used the PAQ as a reference for how plant-related attitude items could be written but decided to create our own items that would address attitudes toward plants (Fančovičová, and Prokop, 2010). While the PAQ is valuable in that it measures how students feel about plants and what their attitudes toward plants are, PAD is about more than attitude. Therefore, we opted to create an instrument that would measure all facets of PAD (see Table 1).
| Component of PAD | Example itemsb |
|---|---|
| Attention | When I take a walk outside, I notice the plants that are around me.Plants blend into the background when I’m outdoors.* |
| Attitude | I have a lot of good memories about plants.I would enjoy visiting a botanical garden. |
| Knowledge | Life on Earth could not exist without plants.I think plants are unimportant to humans.* |
| Relative interest | I would rather have plants in my home than pets.I am more interested in efforts to protect and conserve animals than I am in similar efforts with plants.* |
We used a Likert-style scale consisting of “completely disagree,” “somewhat disagree,” “somewhat agree,” and “completely agree” as answer options. Positive and negative items were used in the instrument, and the negative items were reverse scored. We scored “completely disagree” as 1, “somewhat disagree” as 2, “somewhat agree” as 3, and “completely agree” as 4 (except where items were negative and reverse coded). The minimum score was 28 if students answered all items with a negative (plant-unaware) answer, and the maximum score was 112 if they answered all the items with a positive (plant-aware) answer. We included a quality-control item that instructed the respondent to select the answer, “somewhat agree.” If the respondent answered this item incorrectly, we removed the data for that participant, as this indicated the participant did not pay attention while answering the survey.
EVIDENCE BASED ON TEST CONTENT
We decided to collect evidence based on test content, because PAD (while supported by theoretical components) is not based upon a formal educational theory. As such, it was vital to make sure we represented the ideas within PAD literature that were most important to PAD and its components. This helped us lay the groundwork for a conceptually consistent instrument that included the major theoretical components first laid out by Wandersee and Schussler (2001) without neglecting any supporting ideas or information. It is for this reason we approached Dr. Schussler specifically for her expertise in this area.
Expert Review with Dr. Elisabeth Schussler
In the initial item development phase, we created items that aligned with all four components (attention, attitude, knowledge, and relative interest) based upon conversations with Dr. Schussler and previous findings within the literature (Parsley, 2020). We went through multiple rounds of revisions between the first (K.M.P.) and third author (J.L.S.) before settling on a semifinal version that was sent to Dr. Schussler as our expert reviewer for clarity and soundness of ideas.
Dr. Schussler indicated which items she perceived to belong in each component of PAD, allowing us to compare answers and ultimately reach agreement. Dr. Schussler also made suggestions designed to improve the clarity of items and made note of any problematic items that could have fit into more than one component. We clarified the items as needed and split compound items apart to ensure each item only addresses one specific idea or phenomenon. In the final step, Dr. Schussler left a comment on any terms that she thought may not be known by our target audience. After incorporating her edits and discussing any differences of opinion or questions we had about items, we arrived at the first version of the PAD-I, which was then distributed for factor analysis. This version included eight items about attitude, eight items about knowledge, six items about relative interest, and six items about attention, for a total of 28 items.
After we received results from the first round of factor analysis (which will be explained in greater detail in a later section), we made any necessary adjustments and removed any items that were not performing well. We then sent this new version to Dr. Schussler for another round of expert review. In this second round, she performed the same activities as described for the first round of expert review. In addition, she added comments and questions where necessary to point out if an item seemed to belong to more than one component (and made suggestions for how to clarify it). Once we addressed her edits, we finalized the second version of the instrument and sent it out for factor analysis again. Once we received the results of this factor analysis, a few items needed to be removed, but the instrument demonstrated reliable PAD score inferences and a consistent factor structure that remained the same over multiple iterations of factor analysis, so further revisions were not needed.
Expert Review with Dr. Kristine Callis-Duehl
Dr. Kristine Callis-Duehl is the director of Education Research and Outreach at the Donald Danforth Plant Science Center (DDPSC). Since the final version of the PAD-I was developed, she and her team have used it in various research studies through DDPSC. We approached her for input regarding our development and validation process, as well as the validity evidence based on test content. She expressed confidence that the instrument was soundly developed and that the evidence based upon test content was sufficient and well rounded, such that it adequately represents the theoretical basis of PAD. She and her team continue to use the PAD-I for various projects.
EVIDENCE BASED ON RESPONSE PROCESSES
We chose to collect evidence based on response processes for two reasons: to help confirm and support the four components of PAD (and the information Dr. Schussler introduced us to during her expert review) and to ensure that we were not inadvertently measuring only one component of PAD, such as interest in plants or attitudes toward them. We reasoned that by asking specifically worded qualitative interview questions to better get at students’ experiences with plants and how they related to PAD, we would be better able to determine whether all four component of PAD were present. As such, our interviews represented all four components of PAD equally, and we were able to compare the answers with the questions regarding each component to that of the instrument items and overall scores.
SMALL SAMPLE PILOT STUDY: INITIAL ITEM ANALYSIS AND EVIDENCE FOR RESPONSE PROCESS VALIDITY
Context and Participants
All methods were approved by the University of Memphis Institutional Review Board under proposal number FY2018-323. Our study took place over two trimesters in 2018 and included all students in an undergraduate botany course at a small midwestern university. Thirty-eight students (100%) consented to participate in the first trimester and 40 students (100%) consented to participate in the second. The demographic sample was largely white and female (see Table 2 for demographic information). The course consisted of primarily junior-level (third-year) undergraduate students, was required for all biology majors, and lasted 10 weeks. While the course was introductory in skill level and largely lecture based, the professor also used a mixture of class discussion, the Socratic method, PowerPoint slides for students to add information to, worksheets, exposure to primary literature that also involved group activities, and debates that required preparation outside of the classroom. The topics covered included plant anatomy, morphology, physiology, and diversity. Basic ecology was a programmatic (departmental) mandate that was woven throughout the course. Course work included two-unit exams (consisting of a mix of multiple-choice, fill-in-the-blank, short-answer, and short essay questions and drawing/labeling drawings), class participation and assignments, a class discussion with worksheets and reflections on the book WaldenWarming by Primack (2014), and a final exam. The final served as a third unit exam with an added section covering material from the entire course. Like the two-unit exams, the format was a mix of question types. The course also required concurrent enrollment in a weekly, 2-hour-long botany lab, which constituted 20% of the overall grade in the course and included three lab quizzes and an inquiry-based research project.
| Semester 1 | Gender | Ethnicity | ||
|---|---|---|---|---|
| Female | 26 | Asian/Asian American | 3 | |
| Male | 10 | Native Hawaiian, or Other Pacific Islander | 1 | |
| Hispanic, Latino, or Spanish origin | 3 | |||
| White | 29 | |||
| Semester 2 | Gender | Ethnicity | ||
| Agender | 1 | Another race/not listed | 1 | |
| Female | 30 | Asian/Asian American | 3 | |
| Male | 8 | Black/African American | 2 | |
| Hispanic, Latino, or Spanish origin | 6 | |||
| White | 27 |
| Test | Pretest mean | Posttest mean | df | F | pa | Partial eta-squared (effect size) |
|---|---|---|---|---|---|---|
| PAD-I | 82.3 | 85.2 | 28 | 7.261 | 0.012* | 0.206 |
| Attitude | 3.12 | 3.22 | 28 | 2.764 | 0.108 | |
| Attention | 2.85 | 3.00 | 28 | 6.235 | 0.019* | 0.182 |
| Relative Interest | 2.14 | 2.13 | 28 | 0.029 | 0.866 | |
| Knowledge | 3.42 | 3.57 | 28 | 16.715 | <0.001* | 0.374 |
Data Collection
In the pilot study, we used a mixed-methods research design by administering the survey as a pre/posttest (n = 60 across two trimesters) and collecting interview data (n = 10 across two trimesters) to establish validity evidence based on response processes and proof of concept. The survey was administered at the beginning and end of the two trimesters. This version included eight items about attitude, eight items about knowledge, six items about relative interest, and six items about attention, for a total of 28 items. Interview participants were selected based on having a range of pretest scores on the PAD-I so as to get at student ideas about plants from differing levels of PAD. In the interviews, we asked students about different concepts related to PAD (e.g., plant mentors, positive and negative experiences with plants, and memories or experiences surrounding plants), as previous studies have indicated these are important factors that contribute to whether or not a student demonstrates PAD (Wandersee and Schussler, 2001). In the second trimester, we added questions regarding the extent to which students had trouble understanding the survey or answering any of the questions and soliciting suggestions for how to make the survey more accessible and clearer.
Data Analysis
All collected data were de-identified before analysis with a random ID number. Students received the same ID number for both surveys and interviews. All names used in the Results section are pseudonyms. To evaluate the PAD-I, we calculated averages for all four subscales within the PAD-I for all of the students. Each subscale average had a range of 1 to 4, as each item within the subscales was scored from 1 to 4. We calculated subscale averages by adding all the item scores within each respective subscale together and dividing by the number of items within that respective subscale. The highest possible overall score for the instrument was a 112, the lowest score possible was 28. We also completed a one-way repeated-measures analysis of variance (ANOVA) for the PAD-I and all four subscales within it. Statistical analyses were conducted identically across both trimesters.
To determine whether the instrument demonstrated valid inferences related to PAD using evidence based upon response processes, we used descriptive coding and specifically looked for answers to our interview questions indicating that there were problems with the instrument and any suggestions from participant for how to improve the instrument, in case changes needed to be made to the survey to make it more understandable (Miles et al., 2014). We also collected answers that indicated that the instrument demonstrated valid inferences related to PAD using evidence based on response processes and that it made sense to the respondents.
We then created mini case studies of all the participants to compare their pre–post PAD-I scores with their interview data. This allowed us to demonstrate that students with a range of PAD-I scores also demonstrated a range of PAD in their interviews, providing validity evidence based on response processes. We again used descriptive coding to look for specific examples of PAD (or lack thereof) within each interview (Miles et al., 2014).
PILOT STUDY RESULTS
Survey Results
Please note that a higher score on the PAD-I indicates a decreased level of PAD (or an increased level of appreciation for plants), while a lower score indicates a higher level of PAD (or a decreased level of appreciation for plants). In both trimesters, we found that students’ scores were significantly higher on the PAD-I on the posttest as compared with the pretest. The Attention and Knowledge subscales also increased significantly on the posttest PAD-I in both trimesters (see Tables 3 and 4). In addition, the Relative Interest subscale increased significantly on the posttest PAD-I in trimester 2 (see Table 4).
| Test | Pretest mean | Posttest mean | df | F | p | Partial eta-squared (effect size) |
|---|---|---|---|---|---|---|
| PAD-I | 80.0 | 84.2 | 35 | 21.039 | <0.001* | 0.375 |
| Attitude | 2.99 | 3.05 | 35 | 1.260 | 0.269 | |
| Attention | 2.75 | 2.92 | 35 | 6.30 | 0.017* | 0.153 |
| Knowledge | 3.39 | 3.60 | 35 | 45.546 | <0.001* | 0.565 |
| Relative Interest | 2.06 | 2.18 | 35 | 5.086 | 0.030* | 0.127 |
The category with the largest effect size in both trimesters was knowledge, indicating that many students felt significantly more confident in their knowledge of plants across both trimesters. Attention had a low effect score in both trimesters, which demonstrates that fewer students increased their score for this concept (as compared with the other concepts and the survey overall). In the second trimester, relative interest had the lowest effect score, which was a change from the first trimester, when there was no significant increase in the relative interest scores. It appears that relative interest is what changes the least (if at all) when considering changes from pre- to posttest. This indicates a relative stability in student interest, regardless of how knowledge or attention may change over time. Attitude also did not change significantly, indicating that it tends to be stable along with relative interest.
However, it is important to note that these results (along with our qualitative findings) may shift depending upon the demographic characteristics of the sample being studied. If a study population is significantly different from the one recruited for this study, measures will need to be taken to ensure that the inferences demonstrated by the instrument remain valid and reliable.
Interview Results
To determine whether the instrument demonstrated valid and reliable PAD score inferences via evidence based on response processes, we asked interview participants in trimester 2 about any issues they had when taking the PAD-I. Four out of five interview participants responded that the survey was clear and made sense to them and that they would not make any changes. However, Brenda offered crucial feedback when she answered, “Well, I mean for number three it says, ’I have taken plant courses for my degree.’ [College] only offers one. We have [inaudible] and then I think we did have a more in-depth botany class, but we don’t offer anything else besides environmental courses.” This indicated that the item regarding plant courses would not work as well for programs with few plant science offerings and perhaps was not accessible for some students because of this. We opted to remove this item from the second iteration of the instrument (between EFA round 1 and EFA round 2).
We demonstrate here that the PAD-I responds to differing levels (both high and low) of PAD in undergraduate biology students. We use choice quotes to demonstrate these findings, while further details can be found in Appendix A in the Supplemental Material. To determine whether response processes reflected PAD-I scores and concepts, we created mini case studies and used open coding to determine whether the answers to the interview questions aligned with students’ levels of PAD as measured by the PAD-I.
Nick, Trimester 1
Nick had the highest PAD-I score of the class at 100 out of 112. When asked why he thought he attained this score, Nick answered:
Well, I think it has to do with the things I was saying earlier, just because of my interests within nature. So, I think it’s become more, I don’t know, of a passion as I’ve grown. I was never really on the complete major environmental track. I was pre-med, but then there was a switch and that felt right for me, so I kind of went with it. I have been doing it since then and enjoying that, so I think that’s why I probably got that 100, because I really agree with a lot of these things.
Nick’s choice to switch majors is an example of his lack of PAD as it was driven by a desire to have a career that he truly loved because it involved nature, rather than one that would make him money. Nick’s love of nature led not only to a lowered level of PAD but also to a change in his career.
Throughout his interview, Nick recounted previous experiences when he was younger that stoked his interest in plants. He also spoke of how his mother was his plant mentor and how his interest in plants and his desire to learn more about them has only increased over the years. All of these ideas together indicate that Nick does not demonstrate a significant amount of PAD, and this is reflected in his PAD-I score. When asked if he thought anything about plants was boring, Nick responded that he could not think of anything, further demonstrating his low PAD levels. At the end of the trimester, Nick’s score stayed at 100, indicating that his PAD levels did not change (despite his new knowledge of plants from his botany course), likely because of his already-existing appreciation for plants.
Ashley, Trimester 1
Ashley had the lowest score of the class on the PAD-I with 68 out of 112. Due to technical difficulties, Ashley’s response to the question of how she thought she attained her score was not recorded. However, when asked how her opinions of plants had changed since the survey, she answered:
Since I’ve taken botany, I’ve learned a lot more information about plants. Physiology, anatomy, I have a lot more respect for the different processes that I didn’t know existed. It’s a lot more complex than I thought it was. I thought it was very simple compared with animal physiology. They’re two different categories but it’s more complex than I thought it was.
Ashley spoke of how she entered her botany course with misconceptions that plants were simpler than animals or performed fewer physiological processes. These ideas changed in the time between taking the PAD-I and completing the interview, but they may have contributed to her low score on the PAD-I at the beginning of the trimester.
When asked about her previous experiences with plants, Ashley noted that she did have some pleasant memories of being around plants while her father acted as her plant mentor. However, she also mentioned not having a very good relationship with plants anymore (unlike Nick). Furthermore, she cited that several things bored her about plants, especially the detailed terminology used to describe them. The combination of previous misconceptions of plants, a poor current relationship with plants, and a distaste for the jargon associated with learning about them seems to have discouraged Ashley. This explains her low score of 68 on the PAD-I, though it could be much lower (the lowest possible score is 28). The fact that her score was not lower may be explained by her previous positive attitudes toward and experiences with plants. It appears that previous experiences are not enough to maintain low levels of PAD and that these need to be supplemented with a continuation of positive experiences and relationships with plants. At the end of the trimester, Ashley scored a 67 on the PAD-I, which was 1 point lower than her original score. The relative consistency in her score likely demonstrates that knowledge of plants gleaned from her botany course was not enough to improve her level of PAD. Additionally, while she states she learned a lot from the course, her knowledge score also decreased by 1 point, indicating that perception of gained general plant knowledge does not necessarily translate to the understanding of why plants are important (which is the specific type of knowledge our instrument aims to measure).
Tiffany, Trimester 2
Tiffany scored a 97 out of 112 on the PAD-I, which was the highest score in the class for that trimester. When asked why she thought she obtained this score, Tiffany answered:
Well, I’ve grown up in … My backyard is basically a forest, so we do a lot of outdoor activities, and my parents always … made us play outside, and so I’ve always been around plants. And my mom’s a big plant lady, so she would bring me to the garden store when I was younger all the time. So, I’ve kinda had that exposure and background.
Tiffany cited being outdoors a lot and learning from her mother as a reason for her high score, a similar story to the one Nick told in trimester 1. This makes sense, as both students received the highest PAD-I scores in their respective trimesters.
Throughout the rest of the interview, Tiffany’s answers were similar to Nick’s answers. She cited several pleasant memories of being around plants when she was younger and reported having two plant mentors: her mother and father. She also reported a continued positive relationship with plants (similarly to Nick in trimester 1) and added that she thinks her relationship with plants has only improved since childhood. Tiffany did express disappointment regarding plants, because they are not as interactive as animals; however, she still seemed to maintain interest in them. The combination of positive experiences with plant mentors, a continued interest in plants, and learning new information about them likely contributed to her PAD-I score. Tiffany’s score also stayed the same at the end of the trimester (97) just like Nick’s did in trimester 1. This indicates that her newfound knowledge of plants did not impact her level of PAD.
Brendon, Trimester 2
Brendon scored a 64 out of 112, which was the lowest score of the class for that trimester. When asked why he thought he obtained this score, Brendon answered:
While I do enjoy nature, I’m more of a microbiologist. [Botany is] just [inaudible] required for my major. I don’t per se care about plants. I don’t have a background in plants outside of this course, speaking like academically. I would say that’s probably why, I just don’t have much of an affinity towards plants outside of, like, soil microbiology.
Brendon describes that he has more of an affinity for microbiology than plants and that he does not care much about plants. This is likely a reason for his low PAD-I score.
Brendon did describe some previous experiences with plants and plant mentors throughout his interview. However, these experiences were far less prevalent and numerous than those of students with higher PAD-I scores (such as Tiffany and Nick). Additionally, Brendon did not report having a significant relationship with plants anymore beyond eating fruits that he enjoyed. Brendon also expressed disappointment that plants do not move or interact with humans in the same way that animals do, and noted that he finds this boring, as he is interested in biomechanics. Brendon was the only participant in both trimesters to improve his PAD-I score, as his posttest score increased to a 76. This indicates that, for Brendon, something about his experiences in the botany course did improve his level of PAD-I. This seems to be because his attention subscore increased dramatically from 9 to 17 points. His knowledge score also increased from 22 to 24, while his relative interest actually decreased from 12 to 11, and his attitude increased from 21 to 24. Of all the subscores, his attention changed the most, indicating that his experiences in the botany course mostly affected his attention to plants.
It is clear in both trimesters that positive experiences with plants, both past and present, play a large role in whether or not a student exhibits more PAD. However, in both students with the lowest scores, previous positive experiences did take place, but they seem to have been overpowered by lack of current experiences and finding plants boring. It is worth noting that Nick indicated no negative experiences with or opinions of plants, while Tiffany did feel that plants were sometimes boring due to their lack of movement. This likely explains why Nick received a higher score than Tiffany did, indicating that the PAD-I is potentially capable of delineating among differing levels of PAD even at high or low ends of the scoring spectrum.
Based upon the results of our interviews, we concluded that our instrument demonstrated valid PAD inferences via evidence based on response processes. Students reporting less PAD in interviews had high PAD-I scores (indicating higher appreciation for plants), and those reporting more PAD in interviews received low PAD-I scores (indicating decreased appreciation for plants). We were also able to use students’ language about their experiences to refine the language of our items to make them clearer and easier to understand (such as ensuring complex items were simplified and clarifying items that could fall under two components of PAD so that they only addressed one). These minor adjustments and the exclusion of the item regarding course content added to the overall quality and clarity of the items in the instrument.
EVIDENCE BASED ON INTERNAL STRUCTURE
Validity evidence based on internal structure was an iterative process that we intertwined with evidence based on response process and evidence based on test content. To create a structurally sound instrument that reflected the four components of PAD, we needed to ensure that the items we created for each component of PAD consistently loaded onto the appropriate factor that represented that component. This was the best way to ensure that each component was reflected in the overall structure of the instrument.
Exploratory Factor Analysis: Round 1
To determine what validity evidence the PAD-I demonstrates based on internal structure, we conducted two rounds of exploratory factor analysis (EFA), which allowed us to determine the factor structure of the instrument and whether it was stable. All methods outlined here were approved by the University of Memphis Institutional Review Board under proposal number FY2019-392.
In the first round of EFA, we used a quantitative factor analysis design and sent out emails through two existing science education Listservs, the Society for Advancement of Biology Education Research (SABER) and National Association for Research in Science Teaching (NARST), to recruit instructors who were willing to have their students participate. The PAD-I survey was administered via Qualtrics with a consent form at the beginning. Students spent approximately 15–20 minutes total on the survey, and our target population was undergraduate students taking a biology class. We received a total of 1231 respondents for the PAD-I, which came to 1062 after data cleaning to remove any incomplete responses or any participants who did not respond correctly to the quality-control item (see Table 5 for demographic information).
| Classification | N | % | Gender | N | % | Ethnicity | N | % |
|---|---|---|---|---|---|---|---|---|
| Freshman | 557 | 52.45% | Another gender not listed here | 3 | 0.28% | American Indian or Alaska Native | 7 | 0.66% |
| Sophomore | 211 | 19.87% | Female | 679 | 63.94% | Another race not listed here | 65 | 6.12% |
| Junior | 198 | 18.64% | Male | 363 | 34.18% | Asian or Asian American | 100 | 9.42% |
| Senior | 70 | 6.59% | Prefer not to answer | 5 | 0.47% | Black/African American | 163 | 15.35% |
| Other | 24 | 2.26% | Did not respond | 12 | 1.13% | Multiple races selected | 77 | 7.25% |
| Did not respond | 2 | 0.19% | Native Hawaiian or Other Pacific Islander | 4 | 0.38% | |||
| White | 646 | 60.83% | ||||||
| Total | 1062 | Total | 1062 | Total | 1062 |
A preliminary reliability analysis on the PAD-I gave an acceptable Cronbach’s alpha score at 0.85. We analyzed the results of the first round of EFA using a maximum-likelihood factor extraction with direct oblimin rotation within the psych package in R (Revelle, 2019). We used the fa.parallel function within the psych package to generate a scree plot and the accompanying recommendation of how many factors should be extracted for the analysis (see Appendix B in the Supplemental Material). Maximum-likelihood extraction and direct oblimin rotation are often used for confirmatory factor analysis (CFA), which is used to confirm the hypothesized factors of an instrument. However, this methodology has also been used to create factor-loading scores that can then be transformed into item discrimination parameters for use in item response theory and Rasch analyses that can offer us insight into how individual items are operating within the instrument (Revelle, 2019).
Our first EFA results for the PAD-I revealed a six-factor model, differing from the original hypothesized four-factor model (attitude, attention, knowledge, and relative interest). The six factors were: Caring for or Investment in Plants (three items), Necessity of/Importance of Plants (four items), Plants Better than Animals (five items), Animals Better than Plants (three items), Attention to Food Plants (three items), and Positive Affect (five items). Names for the factors were determined by examining what items loaded onto each factor and observing what concepts or ideas these items had in common. All items loaded onto their respective factors with a score of 0.3 or higher as required for EFA (χ2 = 666.92, df = 225, p < 0.01, Tucker Lewis index [TLI] = 0.917, root-mean-square error of approximation [RMSEA] = 0.043).
The six factors of the PAD-I still aligned well with the original attitude, attention, knowledge, and relative interest components of PAD (see Figure 2), so we proceeded with edits to remove any items that did not load onto a factor, as well as clarify and reword items that loaded poorly onto a factor. We also added a newly hypothesized factor called “General Attention,” which included three items, two of which were recycled from the original PAD-I instrument and one of which was newly created. We did this because the only attentional factor that was gleaned from factor analysis was attention to food plants, which may point to a tendency for students to only notice plants in the context of what they do for humans.
FIGURE 2. Alignment of preliminary hypothesized factors from factor analysis with previously hypothesized factors based on the four components of PAD.
We added the general attention factor to compare the two to determine whether this was the case in the next round of analysis. After adding the new factor, we went through more rounds of revisions with Dr. Schussler before settling on the second version of the instrument. The second version was 30 items long, with each factor containing three to six items per factor. There were three items in Caring for or Investment in Plants, six in Necessity of Plants/Importance of Plants, six in Plants Better than Animals, three in Animals Better than Plants, four in Attention to food plants, five in Positive Affect, and three in the newly added General Attention factor. This change in length meant that the new minimum score that could be obtained with the instrument was 30 if student chose all negative (plant-unaware) answers and 120 if the student chose all positive (plant-aware) answers.
Exploratory Factor Analysis: Round 2
In the second round of EFA, we again sent out emails through the two existing science education Listservs that we used for the first round of EFA (SABER and NARST) to recruit instructors who were willing to have their students participate. We cleaned the data to remove any incomplete responses or any responses that did not respond correctly to the quality-control item, as described earlier in the first round of EFA. Before cleaning, we had 700 responses, and after cleaning, we had 553 due to the large number of incomplete responses and some participants who did not answer the quality-control item correctly (see Table 6 for demographic information).
| Classification | N | % | Gender | N | % | Ethnicity | N | % |
|---|---|---|---|---|---|---|---|---|
| Freshman | 250 | 45.21% | Another gender not listed here | 11 | 1.99% | American Indian or Alaska Native | 5 | 0.90% |
| Sophomore | 138 | 24.95% | Female | 411 | 74.32% | Another race not listed here | 27 | 4.88% |
| Junior | 76 | 13.74% | Male | 129 | 23.33% | Asian or Asian American | 43 | 7.77% |
| Senior | 63 | 11.39% | Prefer not to answer | 2 | 0.36% | Black/African American | 43 | 7.77% |
| Other | 17 | 3.07% | Multiple races selected | 52 | 9.40% | |||
| Did not respond | 9 | 1.63% | Native Hawaiian or Other Pacific Islander | 1 | 0.18% | |||
| White | 382 | 69.08% | ||||||
| Total | 553 | Total | 553 | Total | 553 |
We used another maximum-likelihood factor extraction with direct oblimin rotation within the psych package to determine whether the six-factor model was still appropriate (Revelle, 2019). However, this time, we tested a few different models based on feedback we received from the second EFA indicating that a few of the items were not loading as we had hypothesized after the first round of EFA. Of the four models we tested, two included seven factors and two included six. We reviewed goodness-of-fit indices to make our decision about the model that would best fit our data. The variations in the models were in the number of factors (six or seven) and which items we removed (items that loaded on the wrong factor and items that did not have a loading score of 0.3 or higher).
The scree plot originally generated using the fa.parallel function in psych indicated that our instrument had seven factors (see Appendix C in the Supplemental Material). These factors were almost identical to the factors we found at the end of EFA round 1, with the exception of a few items that loaded onto different factors than they had originally. We decided to test another seven-factor model without these items, a six-factor model without these items, and a six-factor model that excluded a few extra items that did not load (see Table 7). After removing items 13, 14, and 20 (two items from the Plants Better than Animals factor and one from the Attention to Food Plants factor), the scree plot indicated we should only use a six-factor model (see Appendix D in the Supplemental Material). However, we decided to test a seven-factor version as well to see how it would affect loading scores and cross-loadings.
| Model | Descriptionb | χ2 | df | TLI | RMSEA | p |
|---|---|---|---|---|---|---|
| One | Seven factors; no items removed | 474.98 | 246 | 0.936 | 0.042 | <0.001 |
| Two | Seven factors; items 13*, 14*, and 20** removed | 331.48 | 183 | 0.951 | 0.039 | <0.001 |
| Three | Six factors; items 13*, 14*, and 20** removed | 426.46 | 204 | 0.934 | 0.046 | <0.001 |
| Four | Six factors; items 13*, 14*, and 20–22** removed | 301.73 | 165 | 0.955 | 0.04 | <0.001 |
The scree plot that was generated after items 13, 14, and 20 were removed indicated that a six-factor model would be a better fit for our data, so we moved forward with the third and fourth models. A scree plot was generated for the fourth model that removed items 13, 14, and 20–22 (two more items regarding attention to food plants), and it indicated that a six-factor model was still the best choice (see Appendix E in the Supplemental Material). We eventually decided the fourth model would be best, as it was the one that had the best goodness-of-fit scores. Every item in this model loaded with a score of 0.3 or above (see Table 8). Model four removed the Attention to Food Plants factor entirely, and instead focuses on one factor named Attention toward Plants (see Figure 3). This new factor combines items from the previous General Attention and Attention to Food Plants factors to create a well-rounded representation of the fact that attention to all types of plants is an important component in the PAD-I. The rest of the factors remained the same across all four models, which indicates that our factor structure is very stable. In the final version of the PAD-I there are 25 items, which also makes it easier to score, as the scale is 25 to 100 and can easily be transformed into a percentage by subtracting 25 from the final score and dividing this by 75 (see Appendix F in the Supplemental Material). The Cronbach’s alpha of this final version is 0.884 and each factor has a reliability of 0.7 or higher, indicating a reliable instrument. The six factors of the PAD-I still align very well with the original four components of PAD (see Figure 3). Two of the original four components relate to two empirical factors: attitude relates to “Positive Affect toward plants” and “Caring for or Investment in Plants,” while relative interest relates to “Plants Better than Animals” and “Animals Better than Plants.” We still refer to the original four components conceptually, because the results of our factor analyses align with these components, but in the case of the instrument itself, we use the six-factor terminology, as that is what we found in our analyses.
| Item | Caring for or Investment in Plants (α = 0.77) | Necessity of Plants/ Importance of Plants (α = 0.83) | Plants Better than Animals (α = 0.71) | Animals Better than Plants (α = 0.82) | Positive Affect (α = 0.77) | General Attention (α = 0.83) |
|---|---|---|---|---|---|---|
| 1. I enjoy caring for houseplants. | 0.725 | |||||
| 2. I enjoy caring for plants in an outdoor environment. | 0.897 | |||||
| 3. I care about the plants that are in my neighborhood. | 0.454 | |||||
| 4. Plants are important because they help reduce the effects of climate change. | 0.562 | |||||
| 5. Plants are an important source of food for the world. | 0.708 | |||||
| 6. Plants are important to ecosystems. | 0.800 | |||||
| 7. Plants are important because they are a source of oxygen. | 0.711 | |||||
| 8. Plants are important because they are a source of new medicines. | 0.602 | |||||
| 9. Animals need plants in order to survive. | 0.719 | |||||
| 10. I think plants are more useful to learn about than animals. | 0.691 | |||||
| 11. I think plants are more interesting to learn about than animals. | 0.674 | |||||
| 12. If I had to choose, I would rather keep houseplants than animal house pets. | 0.431 | |||||
| 13. When I go outdoors, I am more likely to notice the individual plants around me than any animals in the environment. | 0.335 | |||||
| 14. Learning about animals interests me more than learning about plants. | 0.762 | |||||
| 15. Animal conservation is more interesting to me than plant conservation. | 0.711 | |||||
| 16. I think animals are more interesting than plants, in general. | 0.838 | |||||
| 17. I enjoy going outdoors because of all the plants in the environment. | 0.408 | |||||
| 18. I would enjoy visiting a botanical garden. | 0.564 | |||||
| 19. I have a lot of good memories about plants. | 0.605 | |||||
| 20. Being around plants makes me feel happy. | 0.876 | |||||
| 21. In general, I think plants are very interesting organisms. | 0.409 | |||||
| 22. I notice the crops that are grown near where I live. | 0.402 | |||||
| 23. When I take a walk outside, I notice the plants around me. | 0.508 | |||||
| 24. When I am in a wooded area I notice individual plants, not just the forest as a whole. | 0.825 | |||||
| 25. I notice all the plants in my environment, not just those that I eat. | 0.758 |
FIGURE 3. Alignment of EFA-reinforced six-factor model with original four components of PAD.
Although we have shown that the PAD-I demonstrates valid and reliable PAD score inferences, this analysis is limited by the demographic characteristics of our sample. Specifically, if the instrument is to be used with a population that is more diverse than the population used for this study, it will be necessary to repeat validation efforts to ensure the instrument is still usable in these new contexts. Factor analysis and reliability testing will need to be repeated, and qualitative data will also be necessary to ensure the instrument is being consistently interpreted by students across demographic lines.
DISCUSSION
Our instrument measures PAD as first described by Wandersee and Schussler (1999), and it specifically incorporates the four components: attitude, attention, knowledge, and relative interest (Parsley, 2020). Our results indicate that attention toward plants is a very important component of PAD, as evidenced by the differences in attention to food plants and other plants in our study. This finding aligns with that of Schussler and Olzak (2008) and Balas and Momsen (2014). Our model of the PAD-I includes six factors: Caring for or Investment in Plants, Necessity of Plants/Importance of Plants, Attention toward Plants, Positive Affect toward Plants, Plants Better than Animals, and Animals Better than Plants. The evidence would indicate that these factors continue to align well with and support the original four theorized components of PAD as described by Dr. Elisabeth Schussler, as two of the original four components can be broken down into further subcategories when considering the factor structure of our instrument. In other words, PAD has four components that align conceptually with the six factors of the PAD-I.
This survey builds upon some of the work done in developing the PAQ by incorporating attitudes toward plants, and it does so by tying in the rest of PAD’s components into a more holistic view of PAD (Fančovičová and Prokop, 2010). Our results further support the idea that people who are more invested in plants or care for them in some way have decreased PAD based on the qualitative evidence of this in our interviews (Balding and Williams, 2016). This may also help students overcome their prejudice against plants. In future studies, pre-service teachers exposed to this survey may potentially also have a better understanding of their levels of PAD, which has the ability to help improve their botany teaching (Hershey, 1993; 2002). For example, if pre-service teachers are not only exposed to the idea of PAD, but actually know how much PAD they demonstrate (via our instrument) they could potentially adjust for this when designing curricula for their future classes and intentionally teach with more plants in these curricula.
The development of this tool will allow instructors to conduct studies regarding how well their own interventions work in reducing student levels of PAD. More specifically, in further studies, this survey could be used to reinforce or negate the findings of Schussler and Olzak (2008) that university students recall more animal names than plant ones, even if they are equally nameable. If researchers were to investigate PAD using both a picture-based assessment such as that used by Schussler and Olzak (2008) and combine it with this self-reported PAD-I, they could get a more robust understanding of PAD. This understanding would not only include the attentive state of PAD (as evidenced by the picture assessment) but also the affective states of PAD (as evidenced by the PAD-I). Now that there is a survey to measure PAD that demonstrates valid and reliable score inferences, we can begin to design studies that quantitatively test whether previously described learning interventions work with university students (Wandersee et al., 2006; Frisch et al., 2010; Ward et al., 2014; Krosnick et al., 2018). The PAD-I will also allow for future comparative studies to determine how PAD changes over time.
The results of our study not only have the potential to change how we measure PAD, but also how we approach it conceptually. We provide structural data that support the original four-component model of PAD described by Parsley (2020), as the PAD-I consists of six factors that align conceptually and qualitatively with these four components of PAD. It can be said that our instrument breaks down two of the four components (relative interest and attitude) into two, more granular subcategories. Regardless, this is the first time that data have been used to support the hypothesized four components of PAD.
Additionally, we provide evidence that a knowledge deficit model of PAD is not sufficient. In the pilot study, the significant change in score with the largest effect size in both trimesters was knowledge, indicating that more students felt significantly more confident in their knowledge of plants across both trimesters. However, the lack of a similar pattern in attention, relative interest, and attitude indicates that, while botany courses do affect students’ knowledge, they may not necessarily have an impact on the other three components of PAD. This indicates that relying on a knowledge deficit model of PAD is not sufficient and will not impact the rest of the problems that comprise PAD.
The knowledge deficit model, which originated in science communication research, refers to the idea that, if scientists simply teach the public more about science, the public will come to appreciate science more. However, this model is outdated and has largely been unsupported in the science communication community (Besley and Tanner, 2011). Unfortunately, this is still one of the driving models in the PAD community, as several interventions surrounding PAD rely on getting students to understand more about plants (e.g., Frisch et al., 2010; Ward et al., 2014; Krosnick et al., 2018). While knowledge is a component of PAD, it is the specific understanding of why plants are important to the environment and to people that is the most important type of knowledge in this scenario. Therefore, we suggest more interventions that better integrate this type of knowledge with something that will also engage student interest in, and attitude and attention toward, plants. It is important to consider all four conceptual components of PAD (attitude, attention, knowledge, and relative interest) when designing these interventions to better get at the entirety of a student’s PAD.
Considerations for Use
This instrument will be useful for those who are interested in the problem of PAD and how we can find concrete ways to address it both in and outside the formal classroom setting. While we maintain the utility of our instrument, there are still important caveats to consider. First, while we did receive many respondents during our factor analyses, the majority of our respondents were white and female. This indicates that, if the instrument is used to measure PAD in a different demographic (especially a more diverse one), more analysis will be needed to determine whether the factor structure remains the same. The characteristics of our demographic samples across both the pilot and factor analysis studies represent a limiting factor of this instrument. Therefore, if instructors wish to use this instrument in a population with different demographic characteristics, it is important to retest the factor structure and determine whether the validity and reliability characteristics hold true in new audiences.
In any situation in which an instrument is being used in a context other than the one for which it was originally developed, best practice dictates that a CFA should be completed for the new sample to determine whether the factor structure holds (Knetka et al., 2019). This is true for multiple differences in sample characteristics, whether they are demographic, socioeconomic, or simply a difference in the education level of the respondents. To make the instrument usable in other settings (such as new geographic environments, K–12 learning environments, informal learning environments, and outreach programs), the instrument would need to be validated in these contexts as well.
Future Directions
PAD has been shown to begin and continue throughout the K–12 education experience, and it is for this reason that we intend to validate the instrument for a younger population next. We also plan to seek out more diverse populations for further validation of the instrument in both old and new settings. Partnerships with informal education venues such as science centers, botanical gardens, and environmental education programs will be able to determine whether a particular informal education approach differs in effectiveness compared with more formal education approaches, and as such, we will be validating the instrument in these settings too. Doing so will allow researchers to measure whether their interventions or outreach programs are improving PAD (Wandersee, 1986; Wandersee and Schussler, 1999, 2001; Hoekstra, 2000; Strgar, 2007; Fančovičová and Prokop, 2011; Balding and Williams, 2016; Pany et al., 2019).
LIMITATIONS
The limitations of our study include potential overlap in subjects, as we used the same Listservs to collect data during factor analysis. This survey is a self-report measure and therefore is limited by the participants’ opinions of their own behavior. This research was only conducted with U.S.-based undergraduates in biology-related courses, and as such, the instrument will need to be revalidated if it is used outside the United States, in a different language, or in another type of class (such as psychology courses). While we did have large sample sizes for the factor analysis, the demographic that made up the majority of our sample was majority white and majority female. As such, if the instrument is to be tested in samples with different demographic makeups, CFA is necessary to ensure the factor structure remains the same.
ACKNOWLEDGMENTS
We thank Dr. Elisabeth Schussler for her help in thinking about these issues, for serving as our expert reviewer, and for her thoughtful comments on earlier versions of this paper. We also thank Dr. Kristine Callis-Duehl for her input on this paper and our validation process. This work was supported in part by a grant from the University of Memphis College of Arts and Sciences Research Grant Fund. This support does not necessarily imply endorsement by the university of research conclusions. We would like to acknowledge Dr. Jason Koontz for his assistance in collecting the data described in the pilot study of this paper.
