A Tale of Two Institutions: Analyzing the Impact of Gamified Student Response Systems on Student Anxiety in Two Different Introductory Biology Courses

INTRODUCTION

A national survey of more than 30,000 college students reported that more than 50% of students experience feeling overwhelmed with anxiety (American College Health Association, 2010). Anxiety is a psychosocial emotional state related broadly to feelings triggered by the sympathetic nervous system, including mental arousal, nervousness, and tension (Spielberger, 2010), which could also overlap with persistent and longer-term mental disorders (Hofmann, 2007). As an emotional state, anxiety can be experienced by students when they are worried about failure and are unable to assert control over the outcome (Pekrun et al., 2007). High anxiety negatively impacts student attitudes about their undergraduate science classroom experiences and often becomes the primary contributing factor toward reduced persistence in a major (Brownlow et al., 2000; England et al., 2019). Although some specific and optimal anxieties have been considered motivating (i.e., the Yerkes-­Dodson law; Yerkes and Dodson, 1908; Keeley et al., 2008), reducing overall anxiety in the science classroom can indeed maximize student success (Cooper et al., 2018). A common form of anxiety often seen in large-enrollment undergraduate classrooms is called achievement anxiety, whereby students develop anxiety around assessments or evaluative situations (Covington, 1992; Cooper et al., 2018). Students can experience achievement anxiety as test anxiety (Culler and Holahan, 1980; Chapell et al., 2005; Gerwing et al., 2015), but can also experience other anxieties, like those related to classroom communication apprehension and various types of social anxieties (Zeidner and Matthews, 2005). Communication anxiety in a classroom occurs when students fear that they will perform inadequately in front of the instructor or their peers (Rocca, 2010). This phenomenon is fairly prevalent among undergraduates, with one study documenting that 70% of students experienced communication anxiety at least once (Bowers and Gesten, 1986). Social anxiety refers to the “marked and persistent fear of social or performance situations in which embarrassment may occur,” and often occurs during group problem solving in class (Jefferson, 2001, p. 4). Despite the effectiveness of active-learning practices in increasing engagement and improving retention rates (Freeman et al., 2007; American Association for the Advancement of Science [AAAS], 2010), some practices, such as students being called on to answer a question, are known to contribute to these different types of anxieties (England et al., 2017, 2019). National calls to improve the retention of undergraduates in the science, technology, engineering, and math (STEM) fields have led to the introduction of active-learning practices into many entry-level STEM courses (Freeman et al., 2007; AAAS, 2010). Thus, it is imperative to understand the impact of active-learning practices on student experiences in a science classroom.

One popular active-learning testing tool is the clicker, which is a handheld student response system (SRS) that enables an instructor to gauge student learning and understanding in real time (Sun, 2014). Clickers enable what is known as formative assessment, a lower-stakes form of assessment well documented to enhance student learning (Roediger and Karpicke, 2006). Clickers have been positively correlated with active student engagement (Robinson, 2007; Anthis, 2011; Elicker and McConnell, 2011), learning gains (Sun, 2014), and long-term content retention (Crossgrove and Curran, 2008) and, above all, are viewed positively by students (Preszler et al., 2007). However, clicker use in biology classrooms was shown to specifically increase achievement anxiety, especially when points were awarded for accuracy, despite the fact that it decreased anxiety related to other aspects of the classroom experience (Cooper et al., 2018). Moreover, students may feel motivated to participate in clicker-based activities to earn points as an extrinsic reward rather than as an intrinsic reward to gain mastery of the material. Additionally, Zhu (2007) has pointed out other drawbacks such as high cost to the students and technological difficulties.

Other popular active-learning SRS modalities are those that use gamelike elements, specifically referred to as “gamified student response systems” (GSRSs). GSRSs include competitive elements like leaderboards and earning points within stipulated time periods and commonly incorporate attributes of video game design, such as an audio accompanying game play (Cheong et al., 2014). GSRSs are an increasingly popular approach for improving user engagement and motivation in work-related or educational tasks (Hamari et al., 2014). A popular GSRS on an online platform is Kahoot!, which uses the same multiple-choice format as clickers but is free for students and easily accessible to instructors. Kahoot! can be used by students on their laptops, tablets, or smartphones, racing against the clock to a backdrop of upbeat music to earn points and compete against their peers for first place. Research on Kahoot! has shown that the gamified elements not present in SRSs like clickers (e.g., audio and competition through points) increase student motivation and engagement (Wang and Lieberoth, 2016) as well as attention and learning performance (Barrio et al., 2016). Indeed, studies have also reported that student performance on Kahoot! questions were positively correlated to exam performance (Yabuno et al., 2019) and that using Kahoot! made learning course material more enjoyable (Cheong et al., 2014; Pettit et al., 2015). Despite these positive attributes, gamelike elements like those found in Kahoot! can be perceived by some students as anxiety inducing (Turan et al., 2016). Moreover, class climates that include competitive elements are associated with negative student experiences in STEM (Seymour and Hunter, 2019). Given the continued use of Kahoot! in undergraduate classrooms, it is important to further explore whether the game induces emotions related to anxiety in ways related to its gamelike elements.

Despite widespread acceptance of active-learning practices, no two introductory-level courses are identical, and institutional context is imperative for understanding the effectiveness of educational pedagogical practices (American Association for the Advancement of Science, 2010). We decided to explore how Kahoot! interventions impact student perceived anxiety by conducting parallel studies in introductory biology classrooms at two public research-intensive universities in the United States. In this study, we fill gaps in the anxiety literature, especially as it relates to Kahoot!, by investigating the following research questions:

1. What are the baseline similarities and differences in anxiety between students at each university?

2. How does Kahoot!, when compared with other common pedagogies, impact anxiety?

3. What are student’s perspectives on anxiety and gamelike elements in Kahoot! play?

METHODS

Course Descriptions

U1.

The introductory biology course at the University of Alabama at Birmingham (UAB), a southeastern research-intensive public university henceforth referred to as “University 1” (U1) was one of our test courses. The U1 instructor had no prior experience with SRSs and had not previously participated in education studies assessing their effectiveness. Two concurrent sections were taught in Fall 2017 (16-week semester) and author S.R. taught one of these sections (n = 119). The course included a detailed survey of the plant, fungus, and animal kingdoms and major human organ systems. This course is second in the introductory biology sequence for biology and other STEM majors. Students in this course met for two 75-minute lecture sessions per week. Additionally, the instructor encouraged her students to participate in optional once-a-week discussion sessions outside class taught by teaching assistants (TAs) and supplemental instruction peer leaders (see Table 1 for all U1 pedagogical practices). This active-learning course is considered low-structure, as pre- and postclass assignments were optional for the students (Eddy and Hogan, 2014). Each each of the four lecture exams were worth 25% of student’s overall grade. There were weekly bonus quizzes that contributed an extra 10% toward the student’s total lecture grade.

U2.

An introductory biology course at University of California at Irvine (UCI), henceforth referred to as “University 2” (U2), was chosen as a model of a large-enrollment introductory biology course. The instructor, author J.F.S., had previous familiarity with GSRSs and participation in educational assessment studies related to GSRSs. Two concurrent sections of the introductory course (n = 842 total) at U2 were examined in this study. The course included the typical first half of an introductory biology sequence designed for first-year science students, covering topics such as cell biology, energy transformations, molecular biology, and genetics. Students in this course met for three 50-minute lecture periods per week and one 50-minute discussion-based session led by a trained graduate TA (Lieu et al., 2017). This course was considered a high-structure course, as students had graded preclass assignments, graded in-class work with extensive active-learning components, and graded weekly review quizzes (Eddy and Hogan, 2014; see Table 1 for all U2 pedagogical practices). Student volunteers were called on to answer questions during class daily, and volunteers were asked to come to the front of the class for demonstrations on a few occasions. Student grades in this course were based on 40% final exam, 40% total for two midterms, 6.6% for online preclass assignments, 4.8% for discussion sections, 4.5% for online weekly quizzes, and 4.1% for participation in class.

Recruitment and Procedure

U1.

Pre surveys were administered to students within the first 2 weeks of the Fall 2017 semester at U1. To minimize the likelihood of wearout effects of the GSRS, wherein engagement may fade after students become oversaturated with a new technology (Wang, 2015), Kahoot! was employed three times for approximately 10 to 15 minutes throughout the semester (Yabuno et al., 2019) in the form of a class quiz. Due to the limitation posed by the character limit of the question stem as included in Kahoot! play, only multiple-choice questions that targeted the lower-­order cognitive skills were used. These questions required a minimum level of understanding and comprehension and did not focus on deeper conceptual understanding (Zoller, 1993). We note that SRSs using recall questions, when compared with higher-level reasoning questions, have not been found to have an effect on how students approach SRS questions (Knight et al., 2013). Students received bonus participation points for completion but not for accuracy, as other classroom assessments were also not a part of the lecture grade. Participation points were also not assigned for in-class participation, as assigning point values could possibly contribute to student anxiety (Covington, 1992; Cooper et al., 2018).

At three intervals throughout the 16-week semester, consenting students were asked to complete shorter surveys (see Survey) after playing Kahoot!. These surveys were deployed at least two class periods before an examination. Students were not explicitly provided the investigator’s hypothesis, and instructors did not openly discuss their attitudes about Kahoot! toward anxiety throughout the semester. We did this to mitigate demand characteristics (Nichols and Maner, 2008) where students may give responses to confirm to the investigator’s hypothesis. In the last 2 weeks of the semester, post surveys, which were identical to the pre surveys (see Supplemental Material), but with the addition of course-specific components and a demographic questionnaire, were administered. For all surveys, as well as during Kahoot! play, students used their own nonidentifying code names. On the post survey, students included their official names and code names so that student responses could be associated with final course grades. Given the fact that students used code names, student participation was not known to an instructor until final course grades were submitted. After all post surveys were completed, students self-reported demographics, including gender, race/ethnicity, level of college completed by parent/guardian(s), year in school, major, highest biology course taken in high school, number of biology courses taken in college, grade point average (GPA), Scholastic Aptitude Test (SAT)/ACT, honors status, pre-professional track, and career aspirations (see Supplemental Material for demographic questions). U1 students self-reported information, which is a routine customary practice for education studies at U1. The U1 instructor had access to students’ final grades in the course. This study was approved as exempt by the UAB Institutional Review Board (IRB) protocol no. 300000404.

U2.

In the Fall 2017 10-week quarter, pre surveys were administered to two course sections in the U2 course. Similar to U1 instructor, the U2 instructor did not openly discuss their views of Kahoot! on student anxiety and also did not explicitly state the research hypothesis. The U2 format of the Kahoot! questions was similar to U1 anonymous Kahoot! sessions and were deployed four times at U2, with each session lasting approximately 10 minutes and having about six questions each. Online surveys were used to accommodate the larger number of students in the course (n = 842). As opposed to U1, student demographics from U2 were collected from the registrar, including major, race/ethnicity, gender, class level, SAT scores, first-generation status, low-income status, section, and final grade. Just as at U1, U2 students did not earn a participation or achievement grade for playing Kahoot! This study was approved as exempt by the UCI IRB (protocol 2013-9833).

Surveys

U1.

Students’ perception of their general class anxiety was captured through a seven-item, seven-point Likert scale instrument adapted from Papanastasiou and Zembylas (2008) to measure anxiety and perceived difficulty levels surrounding research (which are correlated and contributive to anxiety). The factor structure of this scale was delineated by Papanastasiou (2005). The scale was 7 points, where 1 was no anxiety, and 7 was high anxiety. For this study, the word “research” in each item was replaced with the words “biology lecture”; this was the only change made to the instrument. The seven items began with “biology lecture” and ended with each of the following: “makes me nervous,” “is stressful,” “makes me anxious,” “scares me,” “is complex,” “is complicated,” and “is difficult.” Also included were three scales intended to measure student test anxiety, communication anxiety, and social anxiety. The test anxiety scale was from the Motivated Strategies for Learning Questionnaire (MSLQ; Pintrich 1991), a five-item scale measured via a seven-point Likert scale. Both the communication and social anxiety scales were taken from the Personal Report of Communication Apprehension-24 (PRCA-24; McCroskey et al., 1982). These scales are both composed of six items measured via five-point Likert scales. While McCroskey et al. (1985) created a score mean mean for each student taking the survey, using McCroskey’s method limited capturing statistical variation in student data, so we decided to use each student answer as an independent data point (see Data Analysis for more detail). Pre and post surveys assessed students’ perceived anxiety via 24 total items (Table 1) based on the aforementioned instruments. Beyond those 24 questions, U1 assessed self-reported pre and post anxiety on a scale of one to seven for the common classroom pedagogical practices, as shown in Table 1.

After each round of Kahoot! play, students at U1 were asked to answer seven Likert questions related to their attitudes about the game, including their motivation to study in class, perceived stress related to Kahoot!, and perceived engagement related to Kahoot! play. As there had been no work in this class context at U1 on any type of SRS, at each interval, U1 students were asked an additional free-response question: “Please provide any comments you have about Kahoot! (positive, neutral, or negative) in the space below.” To exclude any kind of bias influencing the student answers, we chose not to mention either “game” or “anxiety” in our free-response questions for U1 students. Comments were coded as 115 complete ideas, rather than splitting comments after conjunctions (“and,” “but,” “or”), so that we could find interaction effects of themes. This work provided exploratory data to demonstrate U1 student attitudes about the GSRS. U1 students completed a total of three interval surveys. Demographic questionnaires at U1 were administered after all other questionnaires to avoid the possible influence of stereotype threat (Spencer et al., 1999; Eddy and Brownell, 2016). The complete survey layout is shown in Table 2, and all surveys can be found in the online Supplemental Material.

U2.

The same pre and post surveys used to assess students’ perceived anxiety (as reported in Table 1) were used for U2, with slight modifications in terminology. For example, “Supplemental Instruction” leaders were referred to as “Teaching Assistant” leaders at U2, and there were differing software modalities for student online quizzes in some cases as well (e.g., Canvas quizzes vs. Mastering Biology quizzes). There were no interval surveys or other additional surveys given to student participants at U2 due to time constraints.

Data Analysis

All data sets were analyzed using linear mixed effects (LME) models with the lme4 package in R (Theobald and Freeman, 2014; Theobald, 2018). These models are a form of linear regression analysis and attempt to explain a response variable based on a number of fixed effects and random effects. Fixed effects include both categorical (e.g., gender, ethnicity) and quantitative (e.g., age, GPA) variables that are hypothesized to influence the response variable, whereas random effects are categorical variables that may impact the response but in an unknown manner (e.g., student ID). Model predictions for categorical fixed effects represent the estimated differences between groupings, whereas model predictions for quantitative fixed effects are slopes of the regression of the response variable on the value of the fixed effect. All code and data spreadsheets are available in the online Supplemental Material.

Due to the large difference in sample sizes, small differences in questions administered, and other uncontrolled differences between the universities, we analyzed the U1 and U2 data separately to represent different parallel studies of Kahoot! deployment. To analyze students’ baseline perceptions of anxiety for each university, we binned the 24 Likert items from the presemester surveys into four anxiety classes (general biology anxiety, test anxiety, social anxiety, and communication anxiety) and performed LME analyses. Some of the questions on the social and communication anxiety instruments were phrased in such a way that agreement indicated lower anxiety; we inverted student responses to these (using the formula response − (m+ 1), where m is the maximum possible Likert response for the question), such that higher Likert responses in the models always indicated higher anxiety perceptions. We also used the postsemester surveys to assess students’ perceptions of anxiety related to a wide variety of pedagogical techniques used in the courses, and how these anxieties compared with the anxiety perceived during Kahoot! play.

Our full models included the gender and ethnicity of the respondent as fixed variables, reasoning that these demographic attributes might correlate with different base levels of anxiety. Analyses were performed either using raw ethnicity identifications or by binning ethnicity data into underrepresented minorities (URMs; specifically, Black/African American or any Latin American ethnicity) and groups overrepresented in STEM (specifically, White/Caucasian or any Asian or Asian-American ethnicity); the choice of method did not influence the conclusions from the models, so we employed the URM binning method, as it provided more degrees of freedom and thus more statistical power to the analysis. Additionally, we included the number of years the student had been in college and whether or not they were first-generation or returning students as fixed effects. Previous studies suggest that students with these characteristics tend to have higher anxiety. Course grades were used instead of GPAs, because most students were in the early stages of their university careers, making GPA less informative compared with the more-experienced students. Finally, student ID (for U1 and U2) and course section (for U2 only) were incorporated as random effects in the model, improving our ability to detect pre versus post changes in anxiety level against a background of unmeasured individual variation. The significance of each fixed effect was quantified by comparing the full model versus a series of drop-out models missing one term or interaction in the full model using the anova function in R. The final effect sizes reported in the Results section were obtained from refined models that removed nonsignificant predictors from the statistical models. Pairwise post hoc contrasts were computed from the fit model data using the emmeans package in R. For the postsemester pedagogical methods data, we used emmeans to perform Dunnett’s test (Sokal and Rohlf, 2014), contrasting each pedagogical method to Kahoot! play.

In addition to these analyses, we also performed a separate LME analysis on the midsemester data collected from the U1 students after each Kahoot! session. These models were analyzed and refined as described in the previous paragraph.

Free-response data were also obtained at U1; to analyze this qualitative data, two authors (S.J.A. and J.J.M.) used directed-approach qualitative content analysis using themes applicable to the research question (Hsieh and Shannon, 2005), which specifically were “positive,” “neutral/mixed,” or “negative” given the question stem, otherwise known as sentiment analysis (Routray et al., 2013). The researchers independently coded a total of 115 free-response answers from all U1 students responding to the question: “Please provide any comments you have about Kahoot! (positive, neutral, or negative) in the space below.” Researchers used these three themes, which were identified before coding (positive, neutral/mixed, or negative) with 84% initial agreement across all responses, and reached a 100% consensus by settling disagreements.

RESULTS

Response Rate

U1.

Of the 119 students enrolled in the course, 99 (83%) consented and completed the pre survey and 64 of 96 (67%) completed the post survey; 82 completed interval 1; 70 completed interval 2; and 66 completed interval 3. In sum, 60 (50%) students had completed the pre and post surveys and were included in the primary analyses; 55 (46%) students completed all three midsemester surveys and were included for the U1-specific interval analyses.

U2.

Of the 842 students initially enrolled in the two sections of the course, 810 students (96%) consented and completed the pre survey. In the last 2 weeks of the quarter, 728 of 827 students still enrolled in the course (88%) completed the post survey. Of these students, 694 students (84%) had completed both surveys and thus were included in the analysis.

Baseline Anxiety

U2.

In the presemester surveys, students’ anxiety levels varied significantly by anxiety type and also by gender. Across all students at U1, test and general anxieties were statistically higher than communication and social anxieties, with female students reporting significantly higher general, test, and communication anxiety than males (Supplemental Figure 1A; see Supplemental Table 1). For female students, there was a clear and statistically significant hierarchy between the categories, with test > general > communication > social anxiety; for male students, the hierarchy was generally similar (test = general > communication = social), but the differences were less extreme.

U2.

At U2, all students exhibited the same hierarchy of anxiety perceptions, with general > test > communication > social, notably reversing the top two categories in comparison with U1. However, as at U1, U2 students also ranked general and test anxieties significantly higher than communication and social anxieties. U2 presemester responses to questions from all four categories of anxiety were significantly higher for females than males (Supplemental Figure 1B; see also Supplemental Table 1). At U2, the grade a student would ultimately earn in the class was also significantly related to presemester perceptions of general and test anxiety for both male and female students, with higher scores correlating with lower anxiety reports (Supplemental Figure 2; see also Supplemental Table 1). This trend was more pronounced for male students; for instance, the regression slope of grade versus Likert response on general anxiety survey instruments was twice as steep for male students. Thus, the difference between U2 males and females in terms of general anxiety is even more pronounced among high-scoring students than average students. Interestingly, an opposite effect of grade was observed for female students’ perceptions of social anxiety, with higher-scoring students reporting a small but significant increase in presemester social anxiety.

Influence of Kahoot! Play on Anxiety

U1.

We asked U1 students on the postsemester surveys to rate various pedagogical and study techniques with an intent to explore their impact on their anxiety levels relative to Kahoot! play. Kahoot! was consistently rated less stressful than more than 20 other techniques, including common practices such as studying for exams or answering questions in class (Figure 1). We also administered several midsemester surveys at U1, immediately following each Kahoot! session. These surveys probed students’ motivation levels and how they perceived Kahoot! as influencing their experience of the class. Student responses to these questions did not significantly change over the course of the semester and were not detectably influenced by gender, race/ethnicity, or college experience. However, some responses were significantly influenced by the grade a student would ultimately earn for an introductory biology course, with lower-performing students being more likely to report positive Kahoot! experiences than higher-scoring students. These reports included feeling less stressed on an exam due to Kahoot!, Kahoot! influencing studying overall, preference for Kahoot! over other SRSs like clickers, and Kahoot! improving engagement with lecture material (Figure 2).

U2.

At U2, we used postsemester questions related to pedagogical and study techniques. As at U1, U2 students reported that Kahoot! was less stressful than nearly all other classroom practices (Figure 1). Only two practices, “Answering clicker questions in class” and “Listening to your TA lecture,” were perceived as less anxiety inducing than playing Kahoot! High-performing students (i.e., those who ultimately earned higher scores for the course) at U2 consistently reported lower anxiety surrounding most pedagogical techniques (Figure 3). Only Kahoot! and weekly Mastering Biology quizzes were statistically unrelated to student aptitude, suggesting Kahoot! may be especially effective, compared with other pedagogies, for engaging lower-performing students. Similar to the observations with baseline anxiety (Supplemental Figure 2), the relationship of final grade with students’ perceptions of anxiety related to pedagogical methods was significantly more pronounced for male students than female students (Supplemental Figure 3A). Interestingly, first-generation female students reported significantly higher levels of anxiety related to pedagogical methods than either male first-generation students or returning students of either gender.

Student Experiences with Playing Kahoot!

U1.

On the first interval survey, 20 of 81 (25%) U1 students completed the free-response question. On the second interval survey, 58 of 70 (83%) U1 students completed the free-response question, and for the third interval survey, 37 of 66 (56%) U1 students completed the free-response question, for a total of 115 responses. Themes that emerged were related to positive, negative, or mixed (neither clearly negative or positive) experiences of game play and/or anxiety.

Of the 115 total comments (Table 3), 13 referred to Kahoot! gamelike elements (timing, music, competitiveness) negatively, the majority at 63 were neutral or did not comment on gamelike elements, and 39 referred to the gamelike elements, such as competition and upbeat music, positively. For example, one student reported: “I think Kahoot! before class positively engages students. There is competition, laughter, and self-critique. These actions would otherwise not take place without Kahoot!” Overall, 45% of the total 115 comments were made about gamelike elements, and the majority of those gamelike element comments were positive; 38% of the total comments were about positive gamelike elements (in other words, 75% of the 45% comments about gamelike elements were positive).

Sixteen comments were made related to Kahoot! alleviating anxiety, the majority or 93 comments were neutral or did not comment on anxiety, and six comments indicated Kahoot! was anxiety inducing based on specific feedback about Kahoot!, such as the student who reported: “I like Kahoot! but I feel like the music gives me anxiety and the time limit contributes to my anxiety as well.” In all, 19% of the total 115 comments were about anxiety. Of the total 115 comments, 13% of comments were about negative associations with anxiety (in other words, 63% of those 19% specific comments were negative).

We were additionally interested in student comments that mentioned both Kahoot! gamelike elements and anxiety, as these suggested that students recognized that the game play either alleviated or induced anxiety. Of the 115 total comments, 14 incorporated both of these themes, with positive, mixed (not clearly positive nor negative), and negative associations between the two themes. These student comments made specific connections between the gamelike elements and anxiety such as the comment “[Kahoot!] makes [class] too competitive, and therefore stressful.” In sum, 12% of the total student comments noted an interaction between gamelike elements and anxiety, and the majority (43%) of these specific comments were not strictly positive or negative, but instead were a mix of positive and negative sentiments. In other words, a total of 6% of the 115 comments expressed mixed opinions about Kahoot!, its gamelike elements, and related anxieties.

Of note, one student at U1 reported to a researcher feeling a sensory overload during game play due to a diagnosed neurological disorder and stepped out of the room during the Kahoot! sessions. We did not collect any additional data on how this game affected students with clinically diagnosed anxiety or other types of neurological disorders.

U2.

As referenced in Methods, we did not collect qualitative data specific to students from U2. We note there has been previous work by the U2 course instructor on GSRS (Yabuno et al., 2019). Yabuno and colleagues demonstrated that both Kahoot! and clickers were associated with higher exam grades and high levels of engagement in an introductory anatomy course with the U2 instructor (Yabuno et al., 2019).

DISCUSSION

Across large-enrollment biology courses using Kahoot! play at two different universities, we investigated trends in student self-reported anxiety both at the beginning of the semester and how Kahoot! anxiety compared with other pedagogical techniques used during the semester. We specifically wanted to investigate which variables had significant effects on reported anxiety and how students viewed Kahoot! in relation to anxiety. We detected similar trends at two universities despite differing but overlapping survey modalities and course design, suggesting that our results may reflect more generalizable effects of Kahoot! and other GSRSs on student anxiety.

Baseline Anxiety

There were clear contrasts between student cohorts (see Supplemental Table 1), course experiences, and baseline student perceptions of anxiety at U1 and U2, but also distinct similarities. Students at both universities agreed that general and test anxiety were more problematic than social or communication anxiety (Supplemental Figure 1). Considering that students report test anxiety from pop quizzes worth as little as 1% of the overall course grade (Khanna, 2015), the ubiquity of test anxiety in our study could be explained by the weight of examinations (England et al., 2017, 2019): despite differences in course structure, exams counted toward at least 70% of the overall course grade at both universities. U1 and U2 students differed, however, in which anxiety class they ranked as the worst, with U1 students exhibiting relatively greater anxiety toward test-taking than U2 students. Given U1 students were more academically experienced than U2 students, with the median student being in their second year of university instruction, whereas most U2 students were in their first semester, academic experience could have impacted perceptions of certain types of stresses (Misra and McKean, 2000). Students at U2 who ultimately earned lower grades for the course also reported significantly higher levels of general and test anxiety (Supplemental Figure 2), consistent with previous work showing lower grades linked with performance avoidance, heightened anxiety, and negative self-efficacy (Koul et al., 2012).

Female students at both universities also reported significantly greater anxiety than male students on the presemester questionnaires (Supplemental Figures 1–3). This result is not surprising, given known discrepancies between male and female student experiences in biology (Eddy et al., 2014; Eddy and Brownell 2016), as well as female students having increased anxiety in the classroom (Chapell et al., 2005; Koul et al., 2012). Interestingly, female students at U2 who would ultimately earn higher grades, and also U2 female first-generation college students, reported even higher relative anxiety compared with male students with the same characteristics. These correlative effects with gender at U2 may also be explained by the relative lack of college experience of these first-semester students, but it may also reflect other, unmeasured differences between the student bodies at U1 and U2.

In addition to these differences between U1 and U2 in perceptions of anxiety held by students at the beginning of the semester, there were also important differences in the courses themselves. For instance, class sizes at U2 were more than five times as large as at U1. The U2 instructor also employed a high-structure course design with daily low-stakes graded assignments, whereas the U1 instructor primarily used high-stakes exams to assess students (Eddy and Hogan, 2014; Lieu et al., 2017). Because of these structural and baseline differences, we felt it was inappropriate to draw direct statistical comparisons between student attitudes at U1 and U2. Instead, we present below parallel observations that we believe support our conclusion that students at the two universities responded to Kahoot! in similar manners.

Kahoot! Impact on Anxiety Compared with Other Classroom Practices

Former work has demonstrated popular active-learning and classroom practices are associated with anxiety (England et al., 2017; Cooper et al. 2018). While there are students’ perceptions of Kahoot! as fun and effective (Yabuno et al., 2019), little is known about how this GSRS impacts students’ perceived anxiety. One of our central goals was to determine how Kahoot! compared with other classroom pedagogies in terms of course-related anxiety.

Based on average Likert-scale responses, students at both the universities agreed that Kahoot! play caused less anxiety compared with other classroom pedagogical practices (Figure 1). Indeed, there were relatively few practices that were not perceived as significantly more anxiety inducing than Kahoot! For instance, six of eight lecture practices we inquired about at U1, and seven of eight at U2, were rated significantly higher by students (Figure 1). No practices at U1 were significantly less anxiety inducing than Kahoot!, and only two were noted by students at U2—answering clicker questions in class and listening to a TA’s lecture during a supplemental discussion section. As discussed in the Introduction, the differences between clickers (and SRS) and Kahoot! (a GSRS) are related to gamification. It is possible that the average U2 student found the GSRS more anxiety inducing than an SRS due to the gamelike elements like competition or timing. However, there was evidence that Kahoot! was viewed more positively relative to other instructional methods by students who ultimately earned lower scores for the course. At U1, lower-performing students were more likely to report that Kahoot! had a positive impact on their class performance related to student anxiety (Figure 2), and at U2, the anxiety associated with Kahoot! play relative to other techniques decreased as student final score decreased (Figure 3). Both of these observations support the conclusion that Kahoot!, and perhaps other GSRSs, could impact engagement (Yabuno et al., 2019) but without substantially increasing anxiety. Thus, Kahoot! could be a part of an effective curriculum that increases the engagement of students in STEM gateway courses (AAAS, 2010; Yabuno et al., 2019). One possible explanation for our result is that students who were more worried about their grades may have appreciated an activity that was independent of points earned in a course, whereas high-achieving students may have undervalued Kahoot! play for the same reason. This possibility remains unexplored in this work but warrants future study. Considering the fact that lower-performing students may be differentially impacted by higher levels of anxiety, further studies in this direction could certainly explore the relationship between performance and anxiety levels (Cooper et al., 2018).

Previous studies demonstrated that active-learning practices can contribute to classroom anxieties (England et al., 2017). Therefore, it is striking that a GSRS like Kahoot! had such a different effect on students, given its competitive, gamelike elements. Considering that students at our universities and others (Khanna, 2015) reported highest anxiety on assignments with a point value, it is possible that Kahoot! did not contribute as much to student-reported anxiety because Kahoot! assignments were not a point-valued assessment for either U1 or U2 students. Students may be more likely to experience anxiety during activities that carry a point value, so it is worth considering when adding SRS or GSRS into classroom pedagogy that students may find them less stressful if there is no point value attached (Cooper et al., 2018). However, in the light of expectancy value theory, it is also possible that the absence of a point value could reduce student effort and associated learning gains with Kahoot! play (Covington, 1992; Wigfield and Eccles, 2000; Cooper et al., 2018). We acknowledge that our studies did not compare graded Kahoot! play to nongraded Kahoot! play, nor was our correlational design capable of connecting long-term academic outcomes to Kahoot!, both of which could be fruitful areas for future research.

Student Perspectives on Kahoot!

Up to 50% of students who switch from STEM degrees do so in part because of competitive course climates, and up to 42% of those who stay in their degree programs also perceive competition as an issue (Seymour and Hunter, 2019; Weston et al., 2020). Given these data, we were especially interested in asking students how gamelike elements like competition, present in Kahoot! but not in other SRSs, contributed to their anxiety. We wanted to specifically ask these questions to students at U1, because the instructor did not have prior experience with GSRSs and because the U1 instructor was able to allocate 10 minutes at three intervals (30 minutes total) to administer course surveys. U1 represented an instructor who was adopting Kahoot! for the first time, similar to others who may choose to incorporate a GSRS after reading this article. The majority of U1 student comments about Kahoot! game elements were positive (Table 3), recapitulating student opinions of other SRSs (Preszler et al., 2007; Anthis, 2011; Elicker and McConnell, 2011) and GSRSs (Cheong et al., 2014; Pettit et al., 2015; Wang and Lieberoth, 2016). These data affirm that, for the majority of the U1 class, students did not perceive aspects of Kahoot! that are unique relative to other SRSs—specifically the game-play elements like competition—as major contributors to their anxiety. However, most of the few comments that mentioned Kahoot!-induced anxiety were negative (Table 3). This suggests that most students left the course with positive attitudes about our GSRS, but for a minority of students, Kahoot!-induced anxiety is what they remembered the most. Gamified elements such as upbeat music, competition, and time pressure, despite being noted positive aspects of Kahoot! play by many, were in fact indicated in four of 112 student comments as negative or anxiety-inducing aspects of Kahoot! play. Indeed, the gamelike elements of Kahoot! have been reported as negative experiences for a small subset of student users (Licorish et al., 2018). Educators have a continued responsibility to understand all student perspectives, not just the class majority (Tanner, 2013). Further research is needed to assess whether GSRSs are a detriment to the overall classroom experience for students who report increased anxiety, especially for students at lower achievement levels. Achievement levels were not tied to free responses, so we cannot say for certain whether or not these negative aspects of Kahoot! ultimately had an impact on student grades, although our Likert surveys supported the opposite conclusion, wherein lower-performing students were more likely to enjoy Kahoot! Future work should directly assess whether pedagogical gains associated with GSRSs justify the increased anxiety experienced by some students.

We recognize that student perspectives beyond those assessed here may have impacted student attitudes about Kahoot! Being made aware that a student felt sensory overload from bright lights during Kahoot! play as well as its associated music, we realized that play may not be an effective learning engagement tool for every student in a college classroom. We recommend announcing a disclaimer before using Kahoot! play in the classroom to be inclusive of the students with sensory stimuli sensitivity to make appropriate accommodations where necessary.

Study Limitations and Outlook

Analogous to much of educational research, this study is limited by its reliance on self-reporting. Future studies of classroom anxiety should continue to supplement self-reporting metrics with physiological data (e.g., biosensors; McNeal et al., 2020). As there were no “control” courses that did not use Kahoot!, our study was not capable of assessing whether using Kahoot! contributed to changes in anxiety over the course of the semester, or whether Kahoot! use was associated with differential student performance or assessment success, leaving a number of interesting unanswered questions. While a strength of this study was the use of multiple sections across two different institutions, the U1 and U2 groups differed slightly in their study methodology, which may have affected data comparisons (Bowling, 2005). However, we note that none of our analyses rely on direct comparisons between the student bodies at the two universities and present the two different approaches as parallel studies.

Another concern with the activity itself may be that Kahoot! could increase competition between students in the classroom, with potentially negative or distracting effects. We aimed to mitigate the negative burden of competition by not assigning grades associated with game play, which as explained earlier, can reduce anxiety. Second, students used aliases when logging in; there is no requirement that real names be used. Third, there was infrequent use of Kahoot! during the semester, which attempted to avoid the appearance of consistent competition, thereby reducing the chances of rivalries being formed. Based on their free responses, the possibility exists that the competition of points-in-game and associated scoreboards was still a detriment to some students at U1. While we did not collect data on student’s sense of belonging in STEM, the possibility exists that these confounds of response systems, overall academic achievement, and perceived anxiety could ultimately impact sense of belonging in their STEM degrees and careers (Trujillo and Tanner, 2014). Educators have a responsibility to continue to study the impacts of all the elements of GSRSs if we choose to use them in our classrooms. While this study investigated Kahoot! deployment in an in-person class, GSRS, which are virtual tools, may be especially appropriate for online classes. Furthermore, we recommend that Kahoot! should be implemented infrequently throughout a semester; perhaps limiting Kahoot! use to formative assessment review three to four times per semester may serve as a good launching point for instructors who would like to implement Kahoot! in their classrooms.

CONCLUSIONS

This work demonstrates that introductory biology students at two different research-intensive universities with similar baseline trends for anxieties (Supplemental Figure 1) all found Kahoot! play to be less anxiety inducing than more than 20 other common classroom activities (Figure 1). Our data indicate that nongraded, infrequent Kahoot! play was particularly effective for lower-performing students (Figures 2 and 3). Active-learning strategies are known to yield promising results for lower-performing students (Haak et al., 2011) but also often stimulate anxiety in ways that impact attrition from STEM programs (England et al., 2017). Our results indicate that Kahoot! may be a promising active-engagement tool that retains the benefits of active learning without contributing toward increasing anxiety compared with other pedagogical tools. Considering the fact that a major objective of STEM education reform is helping raise science interest and capabilities in lower-performing student groups (AAAS, 2010; Woodin et al., 2010), we propose that Kahoot! may be an especially attractive classroom practice for biology educators.

ACKNOWLEDGMENTS

REFERENCES