ASCB logo LSE Logo

Recommendations for Effective Integration of Ethics and Responsible Conduct of Research (E/RCR) Education into Course-Based Undergraduate Research Experiences: A Meeting Report

    Published Online:https://doi.org/10.1187/cbe.18-10-0203

    Abstract

    Advancement of the scientific enterprise relies on individuals conducting research in an ethical and responsible manner. Educating emergent scholars in the principles of ethics/responsible conduct of research (E/RCR) is therefore critical to ensuring such advancement. The recent impetus to include authentic research opportunities as part of the undergraduate curriculum, via course-based undergraduate research experiences (CUREs), has been shown to increase cognitive and noncognitive student outcomes. Because of these important benefits, CUREs are becoming more common and often constitute the first research experience for many students. However, despite the importance of E/RCR in the research process, we know of few efforts to incorporate E/RCR education into CUREs. The Ethics Network for Course-based Opportunities in Undergraduate Research (ENCOUR) was created to address this concern and promote the integration of E/RCR within CUREs in the biological sciences and related disciplines. During the inaugural ENCOUR meeting, a four-pronged approach was used to develop guidelines for the effective integration of E/RCR in CUREs. This approach included: 1) defining appropriate student learning objectives; 2) identifying relevant curriculum; 3) identifying relevant assessments; and 4) defining key aspects of professional development for CURE facilitators. Meeting outcomes, including the aforementioned E/RCR guidelines, are described herein.

    INTRODUCTION

    Scientific advancement is dependent on the ethical integrity and responsible conduct of research (RCR) of all members of the science, technology, engineering, and mathematics (STEM) community. The importance of achieving this goal is underscored by the existence of codes of ethics or RCR standards within STEM and other disciplines, which broadly define integrity as “good citizenship applied to professional life” (Steneck, 2007). Accordingly, such standards are based on several ethical tenets: honesty, fairness, objectivity, openness, trustworthiness, and respect for others (National Academy of Sciences, 2009). While these tenets are universal to all scientific disciplines, the extent to which they are enacted within each discipline and the nature of how they are enacted varies. Regardless, adherence to these principles is key for continued scientific progress, because it promotes responsible research practices, effective collaboration among researchers, and researcher accountability. Furthermore, these principles ensure that research is conducted and disseminated with high fidelity, thereby continuing to serve the public good and increase the public’s trust and support for STEM.

    Consequently, the importance of ethics/RCR (E/RCR) education in preparing new generations of scientists cannot be overstated. This importance is reflected in the considerable emphasis to increase E/RCR education of trainees as advocated for by U.S. federal agencies, such as the National Science Foundation (NSF; America COMPETES Act, 2007) and the National Institutes of Health (NIH; 1989, 2009), as well as similar organizations in other countries (e.g., Office for Human Research Protections, 2019). Despite the importance of E/RCR instruction of this nature, however, the primary responsibility for educating emergent scientists in E/RCR has historically been delegated to their research mentor(s) and, thus, occurs informally as the new scholar joins a laboratory to conduct research in an apprenticeship-based model (Steneck and Bulger, 2007). As a result, training in this context is often highly variable and less reliable. Attempts to formalize E/RCR education in response to federal mandates (America COMPETES Act, 2007; NIH, 2009) have resulted in diverse approaches, with a majority (82%) of research-intensive institutions adopting a framework that requires trainees to complete one or more online instructional modules (Phillips et al., 2018), such as the Collaborative Institutional Training Initiative (Braunschweiger and Goodman, 2007). Other institutions (17%) have adopted the more involved training requirements set forth by the NIH and require in-person E/RCR education for all of their research trainees, regardless of whether the trainees are funded by the NIH or not (Phillips et al., 2018).

    Despite these concerted efforts, several caveats remain. First, the training mechanisms described above generally occur at the postbaccalaureate level. Phillips and colleagues (2018) note, for instance, that institutions often minimize E/RCR education for undergraduate researchers, in some cases resorting to providing printed handouts to undergraduate researchers as their mode of E/RCR instruction. This may be due to the mentor’s lack of ethics pedagogical expertise, a lack of being required to implement E/RCR training, a lack of a repository of shared E/RCR pedagogical resources, a lack of time or financial means to develop either the expertise or the resources needed (Phillips et al., 2018), or a lack of time to interact with students more directly around E/RCR issues. Second, E/RCR instructional approaches for both graduate and undergraduate students (e.g., online modules, in-person workshops, printed materials) frequently adopt a stand-alone format that places E/RCR education outside the context of the research sphere, despite studies on the efficacy of stand-alone E/RCR courses demonstrating only modest gains in cognitive and noncognitive student outcomes, including transfer of knowledge and skills (Shachter, 2003; Antes et al., 2009; Gasparich and Wimmers, 2014; Batz et al., 2015; Mabrouk, 2016; Frantz et al., 2017; Keiler et al., 2017; Wahila et al., 2017; Watts et al., 2017). Finally, such training is often intentionally designed to broadly address E/RCR issues across multiple contexts (Dubois and Dueker, 2009); therefore, context-specific discussion of E/RCR standards and practices is either deficient or largely nonexistent.

    Owing to these shortcomings and the lack of attention such issues have been afforded, we propose that E/RCR education should be reconsidered and restructured to improve efficacy and to better align with postsecondary STEM reform efforts. This is increasingly important for undergraduate STEM populations, because many national calls and initiatives have resulted in efforts to involve students in the practice and culture of research. Most notable among these efforts is the advent of course-based undergraduate research experiences (CUREs), which are increasingly becoming the first exposure undergraduates have to the research process (Auchincloss et al., 2014; Bangera and Brownell, 2014; Brownell and Kloser, 2015). CUREs are defined as research experiences that are embedded into formal laboratory courses (Auchincloss et al., 2014). Current evidence within the biology education literature indicates that CUREs are effective at promoting students’ development of scientific process skills, persistence, and attitudes about learning in STEM (Jordan et al., 2014; Shaffer et al., 2014; Ward et al., 2014; Brownell et al., 2015; Olimpo et al., 2016; Rodenbusch et al., 2016; Corwin et al., 2018). However, despite the salient role that CUREs play in developing students’ research skills, there is a generalized absence of E/RCR education within most CUREs and only a small number of documented efforts to integrate E/RCR education into such learning environments (Olimpo et al., 2017). This could be due to a general absence of mandates for incorporation of E/RCR education within CURE contexts, which itself could be confounded by the variable structure of individual CUREs (e.g., full-semester vs. modular CUREs, national vs. faculty-developed CURE models).

    Regardless, CUREs provide an unparalleled opportunity to incorporate E/RCR education in a manner that is relevant to the students and that allows them to encounter repeated opportunities to apply the learned E/RCR standards during the research process. Because such instruction likewise occurs within the context of an established course, E/RCR education within CUREs could ostensibly be implemented in a more standardized manner than is typically the case for individual faculty laboratories. Given that CUREs are designed to afford students greater access to research-intensive experiences than faculty-mentored apprenticeships (Wei and Woodin, 2011; Bangera and Brownell, 2014), E/RCR education within CURE contexts will not only exponentially increase the number of individuals receiving instruction in this area, but will also serve to acculturate students to accepted norms of scientific research early in their careers.

    In addition, it is pertinent to note that the instructional efforts referenced above are not intended to exclusively benefit those individuals within the STEM disciplines. Increasingly, CUREs have involved students from a diverse array of academic backgrounds (e.g., psychology, sociology) and with diverse professional interests (Batzli, 2005; Olimpo et al., 2016; Ballen et al., 2017). Integration of E/RCR education within CUREs therefore has the potential to substantially impact all students’ awareness of and attention to ethical considerations and responsible conduct of research as they relate to scientific endeavors. Ideally, such exposure will also better equip students to think critically about the applicability of E/RCR issues within their own fields of study and in their own future careers.

    In this article, we summarize the dialogue and recommendations derived from a meeting of the Ethics Network for Course-based Opportunities in Undergraduate Research (ENCOUR), which focused on the integration of E/RCR education in the context of CUREs. Suggested guidelines are provided that articulate why integration of formal E/RCR education within CUREs is important and how CURE designers and educators can integrate this instruction to more effectively facilitate students’ learning and skills in ethical and responsible research.

    The Importance of E/RCR Education in CUREs

    CUREs provide a promising mechanism to immerse students in the rigorous process of research, thereby increasing access to real-world scientific experiences and expanding the benefits associated with undergraduate research to a larger number of students than the traditional apprenticeship model can allow (Harrison et al., 2011; Bangera and Brownell, 2014; Brownell et al., 2015; Shaffer et al., 2014; Rodenbusch et al., 2016). CUREs have been proposed to involve five key aspects of research: 1) the use of scientific practices, including asking questions; developing hypotheses and approaches to test them; gathering, analyzing, and interpreting data; and communicating the results; 2) discovery, pursuing questions with outcomes that are currently unknown to the scientific community; 3) broadly relevant work, either by addressing questions that are germane to the local community or to the discipline/society as a whole; 4) collaboration, working in a group to solve a problem; and 5) iteration, revising and repeating experiments and troubleshooting problems to address challenges and enhance confidence (Auchincloss et al., 2014). Although the emphasis placed on these five aspects varies among instructors, with some educators placing greater emphasis on experimental design and other educators emphasizing the importance of novel scientific products that can result in publication (Spell et al., 2014), the overarching goal of all CUREs is to immerse students in the processes and practices inherent to scientific investigations.

    The benefits of integrating CUREs early within undergraduate curricula (e.g., Rodenbusch et al., 2016) and the increasing number of CUREs being implemented in the freshman year (Jordan et al., 2014; Brownell et al., 2015; Rodenbusch et al., 2016; Fisher et al., 2018) indicate that CUREs are likely to be the first research experience for many students. Thus, engagement in CUREs may establish students’ expectations of how professional conduct and practice function in research settings. Based on the centrality of E/RCR for research and the fact that E/RCR decisions occur in the process of conducting research within CUREs, we postulate that E/RCR education should be integrated within CUREs to ensure that students learn to conduct research with integrity from day one. Furthermore, we believe that integrating E/RCR education within CUREs is ideal for improving E/RCR instruction in a broader sense, because: 1) it allows for meaningful E/RCR education to be directly infused into a relevant research environment, rather than as a detached course, workshop, or seminar; 2) it reinforces E/RCR principles by providing immediate application of concepts with respect to the research being conducted in the CURE; 3) it provides opportunities for reflection on the fundamental role of E/RCR within the day-to-day practice of research; and 4) it has the potential to increase the number of students who receive formal and purposeful E/RCR education as part of the academic experience, especially as CUREs become more common components of undergraduate curricula. Finally, and importantly, E/RCR decisions occur in the process of conducting research within CUREs, and it is important that we, as scientists and educators, ensure that our students approach these decisions with the same ethical standards demanded of non-course-based research contexts.

    Current State of E/RCR Education Integration within CUREs

    The scarcity of publications on this topic suggests a current lack of intentional and systematic efforts to integrate E/RCR into CUREs. To initiate a purposeful conversation on this topic, three authors of this report (J.T.O., L.D.M., G.R.F.) convened a meeting that gathered representatives from a diverse range of institutions (e.g., community colleges, 4-year institutions, minority-serving institutions) and with diverse roles (e.g., CURE instructors, CURE designers, program directors, bioethicists, discipline-based education researchers). This consortium was tasked with providing a set of guidelines and identifying existing resources to facilitate effective E/RCR integration in the context of CUREs. We present these guidelines and an online repository of curated resources, available to all interested stakeholders, in the sections that follow.

    CONSIDERATIONS FOR EFFECTIVE E/RCR TEACHING AND LEARNING

    Given extensive variation in the topical nature and structure of each CURE, we argue that purposeful and effective E/RCR education in these contexts requires coordination among all parties involved in the organization, design, and day-to-day implementation of the course (i.e., curriculum designers, instructors, teaching assistants, and researchers). We propose that instructional teams seeking to effectively implement E/RCR education within their CUREs consider adopting a scaffolded approach that includes: 1) identification of appropriate E/RCR student learning objectives (SLOs) and specific topics that are relevant to the CURE; 2) design and/or identification of curricular minilessons that are aligned with respective assessment(s) and SLO(s); 3) development and/or identification of appropriate assessments that are aligned with respective curriculum and SLO(s); and 4) facilitation of professional development for those individuals implementing E/RCR education within CUREs (e.g., instructors of record, teaching assistants, peer leaders; Figure 1).

    FIGURE 1.

    FIGURE 1. Model for effective integration of E/RCR education within CUREs.

    Identification of E/RCR SLOs for CUREs

    During a series of brainstorming and discussion sessions, we identified six SLOs (Table 1) that we propose are broadly relevant to all CUREs in the biological sciences. Collectively, these SLOs reflect the general goals of E/RCR education in the context of CUREs as well as specific E/RCR topics, standards, and skills that are applicable to CUREs within the discipline. It is important to note, however, that these SLOs are not meant to be used “as is” for all CUREs (i.e., we do not advocate for a “one-size-fits-all” mentality). Instead, we propose that these SLOs be used as a starting point to develop more specific objectives tailored to the type of research being conducted as part of each unique CURE. Indeed, we argue that such integration will serve to increase the relevance of E/RCR education for students enrolled in the CURE, thereby augmenting the efficacy of E/RCR instruction in that context (American Association for the Advancement of Science, 2009).

    TABLE 1. Proposed SLOs and examples of aligned curricular interventions and assessments

    SLOsSpecial considerationsCurriculum example(s)Assessment example(s)
    1. Students will be able to describe the importance of E/RCR as part of the research process.Emphasis should be placed on the ethical values that drive the scientific pursuit (i.e., honesty, fairness, trustworthiness, objectivity, openness, and respect) rather than on examples of misconduct.Instructors can assign students the following article and make use of scaffolded discussion prompts to begin to connect general topics addressed in the article with the research focus of the CURE (Resnik, 2011).Students’ views regarding the importance of E/RCR as part of the research process can be formatively evaluated using one or more free-response prompts. In turn, responses can be used as the basis for further discussion.
    2. Students will be able to define research misconduct, questionable research practices, proper data acquisition and management, collaboration, and authorship in the context of the CURE.The specific topics to be addressed depend on the type of data being obtained in the CURE. If the goal is to publish the findings generated in the CURE, the topics of authorship and authors’ responsibilities should be addressed.Active-learning approaches should be used; e.g., the following three-part exercise could be implemented: 1) an overview of applicable ethical guidelines; 2) analysis of a relevant case study; and 3) interactive role-play of the case study (Teixeira-Poit et al., 2011).Knowledge can be assessed using Hirsch’s survey (Hirsch et al., 2005), which consists of 30 content questions covering all nine RCR areas defined by the Office of Research Integrity.
    Case responses can be assessed using a case-study rubric (e.g., Seiler et al., 2011; Loui and Revelo, 2015).
    3. Students will be able to identify potential ethical concerns associated with the development and/or implementation of their own research.Students should be able to apply E/RCR standards to identify areas of their own projects where potential questionable research practices could arise. This will allow students to be fully aware of the E/RCR standards that apply specifically to their projects.Use the Decision Procedure Checklist (DPC; Keefer et al., 2014) to analyze potential ethical concerns encountered in the CURE. This checklist walks a student through the process of identifying stakeholders, resources to address the problem, and the short- and long-term consequences of the proposed solutions.The Decision Procedure Scoring Guide (Keefer et al., 2014) allows instructors to score responses to the DPC as a summative assessment.
    4. Students will be able to articulate and/or implement mechanisms to address potential ethical concerns that might arise in the conduct and reporting of their own research.Emphasis should be placed on how to effectively facilitate student discussion of E/RCR concerns, with the CURE instructor, as such concerns arise.
    5. Students will be able to collaborate respectfully and professionally.Emphasis should be placed on defining roles and responsibilities, identifying mechanisms for effective decision-making and team accountability, and defining when and how to end unproductive collaborations.While not unique to CUREs, engaging students in creating a group contract (Crowe and Hill, 2006) can serve to address one or more special considerations associated with this SLO.Student collaboration can be assessed using the Association of American Colleges and Universities Teamwork Valid Assessment of Learning in Undergraduate Education (VALUE) Rubric (Rhodes, 2009). Specific guidelines and instructions for using the rubric are included with the rubric itself.
    6. Students will be able to articulate potential scientific and social implications of their research.In CURES with a community-engagement dimension, this SLO should go beyond awareness of implications and address also the responsibilities, benefits, and challenges of engaging the community in the research process.Activities can include discussions and the creation of a cognitive map depicting the ethical implications of students’ research projects.Cognitive map analysis can be used for assessment purposes by analyzing the complexity, relationships, and message of the cognitive map (Jones et al., 2014).

    Please note that the curriculum and assessment examples provided for SLO #3 are also applicable to SLO #4.

    Choosing Appropriate SLOs

    Vertical integration of CURE-based E/RCR education into the larger context of ethics instruction mandated by the institution, as well as the context of ethics education incorporated into the series of courses that each student will take as part of their experience at the undergraduate level, was identified by all ENCOUR members as being of critical importance. In other words, E/RCR education within the CURE should be considered as one component of the comprehensive E/RCR instruction that students receive within their degree programs (e.g., ethics courses, institutionally mandated certification requirements for individuals engaging in faculty-mentored research). Therefore, if the CURE is implemented at the beginning of a student’s career, the SLOs associated with ethical concepts should arguably be tailored to some of the more basic E/RCR principles. However, if the CURE is intended for upper-division students, then any previous ethics education should be considered, and the E/RCR objectives could be focused on reinforcement of previous E/RCR instruction and could have a more nuanced approach. This requires purposeful crosstalk between educators at various course levels so as to avoid redundancy while simultaneously maximizing impact. Importantly, such vertical integration is supported by prior work suggesting that situating E/RCR education within the context of the student’s experience ensures relevance (Watts et al., 2017; Phillips et al., 2018).

    Additionally, it is critical to note that E/RCR education is relevant not only to the STEM disciplines but to all fields of inquiry. Evidence suggests, for instance, that undergraduates are increasingly being exposed to authentic scholarship in the humanities, and, as such, there is a need for E/RCR instruction in these contexts (Grobman and Kinkead, 2010; Shanahan, 2011). Accordingly, we furthermore contend that E/RCR instructional practices across the curriculum need to be taken into consideration when determining how to appropriately introduce E/RCR topics within CUREs. In these instances, interdisciplinary collaboration among faculty members and/or with institutional centers (e.g., centers for teaching and learning) could serve as an effective mechanism to more meaningfully and holistically incorporate E/RCR education as part of a student’s undergraduate experience.

    Once vertical and cocurricular integration are considered, each instructor should then develop learning goals and objectives that are specific to the experiences the students will encounter within the CURE (e.g., Table 1). For instance, if a CURE involves human subjects research or use of vertebrate organisms, then objectives should focus on the role of the Institutional Review Board (IRB) or the Institutional Animal Care and Use Committee, respectively, in the research process. Similarly, if the CURE requires the use of large-scale data sets, then the instructor might place greater consideration on objectives that relate to data management and falsification of data. Thus, while this report outlines six broad SLOs, these should be considered as a place to begin when implementing E/RCR education within a specific CURE rather than a prescribed list of mandatory objectives. When considering the content associated with the SLOs, it is also critical to ensure that the focus of the instruction is not to teach students how to avoid being “bad” scientists, but rather to provide them with the education and skills necessary to be ethical and accurate scientists, thus creating a culture of responsibility (Yarborough and Hunter, 2013).

    Effective Integration of E/RCR Lessons into CUREs

    After thinking broadly about E/RCR education in one’s CURE, the next step is to consider the logistics of its effective integration within the course curriculum. First, to ensure that students understand the importance of ethical research practices, we believe it is necessary to explicitly list ethical competency as part of the overall SLOs for the CURE. Establishing this as a cornerstone of the intended goals of the CURE will subsequently allow instructors to use one or more design-based approaches (e.g., backward design; Cooper et al., 2017) to plan a curriculum and its assessment(s) to meet this objective. In addition, students will see this as a normal and integral part of the research process.

    We discussed a number of components that we felt would be beneficial to successful E/RCR education in the context of a CURE. First, E/RCR education should be initiated at the very beginning of the CURE to highlight not only its importance, but also how it fits within the research process. Accuracy and diligence in maintaining one’s research notebook could, for instance, be addressed immediately as students begin to formulate research questions of interest to explore. Furthermore, research must be designed from the start to follow ethical guidelines, including those established by the institution and relevant funding agencies, thereby preventing issues from arising as the project develops. Following this initial introduction to E/RCR, instruction on additional topics should occur in conjunction with pertinent activities within the CURE. For example, a unit on the ethical guidelines for data collection could be provided once students have designed their projects but before they have started to collect data (e.g., Smith et al., 2007). Embedding E/RCR education within the research process will expose students to the information they need at the exact moment it is needed, thereby providing them with the skills necessary for conducting research in a responsible manner throughout the project.

    While it may initially seem overwhelming to incorporate E/RCR education into a CURE that already requires a significant amount of instructor and student time, we feel that integrating it into the research process in small units is a feasible and effective approach. This has been shown to be successful as a general approach for RCR training (Peiffer et al., 2011) but has not yet been explicitly implemented in CUREs. In addition, Watts et al. (2017) found that providing opportunities for practicing instructional content should be frequent, student-centered, and appropriately distributed throughout the course. For the impact of E/RCR education to be maximized, it is critical that E/RCR units be activity-based (Nebeker, 2014), incorporating such practices as the use of group reflections, role-playing exercises, computer simulations, or class debates. For example, a scenario situation that requires students to address unethical behaviors from the perspective of different participants (student researchers, faculty mentors, community members, etc.) can provide them with valuable insight into the continuum of ethical behavior and the importance of considering multiple points of view.

    More broadly, to facilitate the curriculum development process and to provide resources for CURE designers and instructors interested in integrating E/RCR education into their courses, we searched the literature and other media to identify examples of existing activities and materials that could be repurposed for effective use in the context of CUREs and that aligned with the SLOs referenced earlier. A curated list of these resources is available on the ENCOUR website at http://encour.utep.edu.

    Guidelines for Assessment

    Thorough assessment of E/RCR educational interventions is fundamental to ensure that the approaches used are effective. Decisions regarding how E/RCR education will be assessed should occur during the CURE development phase and should align with course-specific SLOs. Importantly, an initial formative assessment should be included to determine incoming students’ knowledge in E/RCR. This allows the instructor to identify areas of need and thus strengthen education efforts in those areas. In addition, instructors whose CUREs have implications beyond the classroom (e.g., the research is being published, the CURE has sponsors/donors, the CURE involves community engagement) should consider the additional ethical implications and obligations derived from these relationships and find appropriate assessments that include those dimensions. In such instances, ownership of data and authorship of any publications become more difficult to determine. Consequently, education on and assessment of these issues is critical.

    The fundamental goal of providing E/RCR education is to improve students’ understanding of E/RCR standards and, ultimately, to promote research integrity. Therefore, assessment should be aligned with these goals and measure dimensions of knowledge, skills, and attitudes rather than using self-reported measures of perceived usefulness of the instruction (Steele et al., 2016). In an effort to identify available instruments that could be used for this purpose, we conducted a search on several academic databases (e.g., ERIC, SCOPUS, Philosopher’s Index, Google Scholar) using different combinations of the terms “research ethics,” “responsible conduct of research,” or “research integrity” and “assessment,” “survey,” or “evaluation.” These searches returned 76 publications that were relevant to the topic of E/RCR assessment either within the context of biology research specifically or scientific research in general. These publications included previously validated instruments measuring different aspects of E/RCR education, curricular interventions with paired in-house assessments, examples of assessment for courses, and models for paired curriculum-assessment design, among other topics. Although no publications were identified that specifically addressed assessment of E/RCR education within CUREs, 27 of the 76 publications provided instruments or approaches that could potentially be adapted for assessment of E/RCR in CUREs. A compendium describing these references can be found in the resource database (http://encour.utep.edu).

    Importantly, while we advocate for the purposeful use of assessments to measure achievement of SLOs, we recognize that this process might seem daunting for individuals with limited time and/or experience in this area. We therefore intentionally designed the ENCOUR resource database to be a user-friendly platform wherein instructors could identify assessment tools that align with the documented E/RCR SLOs for their CUREs, thus enabling them to more easily tailor assessments to meet their own teaching and learning needs. Likewise, we strongly encourage instructors to establish collaborations with individuals (e.g., statisticians, education researchers) and/or entities on their campuses who, in light of their expertise, can contribute meaningfully to the assessment process. In a broader sense, we wish to emphasize that CURE facilitators should be mindful that all forms of assessment have utility in advancing our understanding of the impact of E/RCR education within CUREs, and, therefore, instructors should be encouraged to adopt assessment practices that are most appropriate for their own course contexts and instructional goals.

    Assessment of Interventions.

    Among those previously validated instruments most commonly used to assess ethics education are the Defining Issues Test-2 (DIT-2; Rest et al., 1999), the Professional Decisions and Values test (PVD; Rezler et al., 1992), the Moral Competence Test (MCT; Lind, 1977), the Sphere-Specific Moral Reasoning and Theory Survey (SMARTS; Curzer et al., 2014), and the Engineering Science Issues Test (ESIT; Borenstein et al., 2010). DIT-2, MCT, and SMARTS use moral social dilemmas, PVD uses case scenarios in medicine and law, and ESIT uses case studies focused on professional practices in engineering and science. However, none of these tests use research-specific scenarios, and they measure general ethics skills rather than E/RCR. A series of more appropriate assessments, which focus on research-specific ethics, has recently been developed and validated (see online resource for a full list and description of the assessments). For example, the Professional Decision Making in Research measure (DuBois et al., 2016) uses research-specific case scenarios and has parallel forms that can be used in a pre–post manner to assess the effect of a research-specific ethics intervention. The Research Ethics Knowledge and Analytic Skills Assessment includes pre- and postintervention tests to assess ethical research with human subjects (Taylor et al., 2012). Yet another example is the comprehensive assessment package developed by DuBois and colleagues (2008), which includes three instruments that evaluate three different levels of E/RCR education: knowledge, skills, and sense of preparedness.

    Although the use of previously validated instruments is encouraged (Antes and DuBois, 2014), in some cases, developing in-house metrics that specifically measure the SLOs relevant to one’s CURE might be both necessary and preferred over the use of an instrument that measures generalized ethical conduct (Watts et al., 2017). Additionally, the use of more open-ended assessments, such as concept maps (Hirsch et al., 2005; Jones et al., 2014), interviews, and think-aloud cases (Seiler et al., 2011), can allow the instructor to gain rich insight into the E/RCR decision process. If used as a means of formative assessment, these methods can likewise provide the instructor with critical feedback that will subsequently allow for clarification of any student misconceptions that might exist.

    Regardless, we caution instructors to use approaches that are pragmatic in nature when selecting one or more of the aforementioned assessment techniques. Scoring open-ended responses, for instance, can be significantly more time-consuming than evaluating closed-ended responses. Conducting interviews and analyzing interview data, as a second example, require experience and expertise in that methodology. While our intent is not to dissuade educators from engaging in rigorous assessment of E/RCR educational interventions within CUREs, purposeful consideration of the feasibility and role of assessment as an inherent component of the implementation process is critical in ensuring that intended outcomes are meaningfully captured.

    Program or Institutional Assessment.

    Directors, administrators, and/or affiliated faculty and researchers involved in the CURE development process might also want to consider the need for higher-order assessment of E/RCR integration throughout their programs (e.g., assessment of E/RCR education across the curriculum, assessment of E/RCR education across all CUREs in a program). Published resources for E/RCR assessment at the programmatic or institutional level are scarce and range from information-gathering surveys to comprehensive programmatic evaluations. On the information-gathering side, Minifie and colleagues (2011) described a survey asking faculty to report the E/RCR topics addressed in their courses and the resources and approaches being used. This allowed the researchers to identify specific RCR topics that were not sufficiently covered by the existing E/RCR education efforts at their institution. Similarly, Thompson (2014) performed a comprehensive program assessment in an EdD program. The assessment surveyed both students and faculty on RCR competencies and documented E/RCR integration within the curriculum. This was achieved via a matrix used by faculty to examine their syllabi and other course materials to identify core RCR content for each course within the program. This study identified gaps in RCR education in the program and allowed faculty members and curriculum designers to address those gaps. More comprehensive evaluations include a framework and online tool published by Olson (2010) that guides the user to develop a customized evaluation model by asking questions related to the goals, resources, and efforts of a particular program in the nine core RCR areas defined by the NIH (2009). Finally, Tractenberg and FitzGerald (2012) developed a comprehensive course or program assessment tool, the Mastery Rubric, that includes assessment of E/RCR knowledge, skills, and abilities according to different levels of student expertise. Thus, while some program- or institutional-level assessments exist, there is a clear need for further development of these types of tools.

    Other Assessment Considerations.

    All E/RCR assessments that were identified rely on knowledge questions, case scenarios, or self-reports of preparedness to apply E/RCR decisions. Therefore, these instruments assess hypothetical scenarios or knowledge rather than actual student adherence to E/RCR expectations. CUREs provide a space wherein E/RCR education can happen side by side with the actual application of these standards to the research being conducted. We posit that this environment provides a unique opportunity for the development of context-dependent curricula and assessments that rely on actual application of E/RCR standards rather than on hypothetical cases.

    Considerations for Professional Development

    Regardless of their roles, we contend that all individuals involved in CURE instruction will require professional development and institutional support to achieve effective integration of E/RCR education within the CURE curriculum. Two key factors requiring thoughtful consideration were identified: 1) What type of professional development is needed for the CURE curriculum developer?; and 2) What type of professional development is needed for the instructor(s) teaching the CURE? Importantly, for CURE developers, professional development must include not only E/RCR education itself, but also training in how to integrate E/RCR instruction into the context of the CURE as it is developed. While this is not always feasible, integration of E/RCR education in this manner increases the likelihood that E/RCR instruction will become a cohesive part of the entire project. Thus, timing of the professional development for the CURE developer is critical, and this has historically not been considered at many institutions where CUREs have been created. We argue that institutional support should be provided as a necessary early step in CURE development. In addition, the institution should consider who will be responsible for ensuring that E/RCR professional development occurs and how it will be conducted. This includes considerations regarding whether certification in E/RCR instruction would be a required part of the professional development process and, if so, how that would be accomplished within the institution (e.g., Kalichman, 2014).

    Importantly, efforts to provide professional development to instructors seeking to implement E/RCR instruction within their CUREs must consider that these instructors can be quite diverse with respect to position (e.g., full-time faculty, adjuncts, graduate teaching assistants, peer instructors), educational background, teaching experience, and prior level of E/RCR education. This requires that the professional development for these individuals be comprehensive enough for the novice and yet flexible enough to provide relevant instruction for those with more experience. In addition, because we advocate for an active-learning approach to E/RCR instruction, it will also be necessary to provide professional development on how best to implement this type of learning within the classroom environment. We propose the following considerations as critical facets of professional development initiatives for CURE instructors: 1) All individuals involved in the CURE need to understand the pertinent E/RCR components and appropriate institutional guidelines as they apply to the CURE; 2) The person(s) responsible for CURE design should have access to E/RCR curricular and assessment resources that have been identified and field-tested for use in CUREs, including those that are self-generated; and 3) CURE instructors should understand that E/RCR education incorporation within CUREs cannot be approached in a “one-size-fits-all” manner, but instead must be tailored to the specific needs of each CURE.

    One key component to providing this broad array of ethical education would be to ensure that CURE developers and instructors have access to a larger CURE ethics community of practice, which is one of the primary goals of ENCOUR (http://encour.utep.edu). As a result of engagement in various professional development opportunities and communities of practice, CURE educators are ideally not only better equipped to facilitate E/RCR instruction within their courses, but are also afforded the chance to enhance their own E/RCR knowledge and skills in the domain.

    CONCLUSIONS AND FUTURE DIRECTIONS

    In this article, we advocate for the integration of E/RCR education within CUREs via use of a four-pronged approach that involves purposeful consideration of one’s course learning objectives and associated outcomes. We further contend that such preparation is essential for the development of ethically responsible scholars and leaders within the biological sciences and, in turn, the broader advancement of the scientific enterprise. While this is the case, it is important to note that pedagogical and professional development efforts of this nature are not unique to CUREs, nor to the biological sciences subdisciplines. Indeed, all course- or apprenticeship-based laboratory experiences in STEM would benefit from explicit discussion of core E/RCR principles (DuBois and Dueker, 2009; Olimpo et al., 2017), both with respect to the experience itself and the STEM domain more broadly. For example, such a conversation might focus on the falsification of data or, as applicable, the IRB process involved in conducting human subjects research.

    In addition, we posit that the benefits associated with integration of E/RCR education in CUREs are far-reaching and can influence not only students’ understanding of the role of E/RCR as it relates to scientific inquiry, but also its practical importance in everyday personal and professional decision-making tasks. While beyond the scope of the present meeting report, we contend that this latter aspect is especially critical as students advance in their studies and elect to pursue novel opportunities (e.g., study abroad, internships) that require interaction with individuals and organizations with diverse E/RCR standards and expectations (e.g., Clements et al., 2013).

    Complementary to our focus on practitioner-oriented concerns related to the incorporation of E/RCR instruction within CURE contexts, we assert that future research is needed with respect to both the development and validation of novel E/RCR assessments and evaluation of novel curricular interventions and instructional approaches in the domain. Evidence obtained from these efforts will be paramount in advancing our understanding of how to best prepare emergent scholars to be ethically responsible citizens and global leaders in STEM.

    ACCESSING MATERIALS

    All resources discussed herein can be freely accessed at http://encour.utep.edu. We acknowledge that this website is a work in progress and encourage readers to contact the corresponding author (J.T.O., ) should they wish to disseminate E/RCR materials through this venue and/or join the network. Specifically, we envision this online space to be a platform for continued collation and curation of emergent resources, including both published and unpublished work by community members. Such a repository is intended to complement those outcomes reported by the ENCOUR leadership team (found under the “Publications” tab), with the overarching goal of advancing high-quality E/RCR instruction in CUREs.

    ACKNOWLEDGMENTS

    Support for this work was provided by a grant from the NSF (DBI-1727867). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the NSF.

    REFERENCES

  • America COMPETES Act. (2007). America Creating Opportunities to Meaningfully Promote Excellence in Technology, Education, and Science Act of 2007, Pub. L. 121 Stat. 572. Retrieved October 3, 2018, from www.congress.gov/110/plaws/publ69/PLAW-110publ69.pdf Google Scholar
  • American Association for the Advancement of Science. (2009). Vision and change in undergraduate biology education: A call to action (p. 81). Washington, DC. Retrieved October 3, 2018, from http://visionandchange.org/files/2011/03/Revised-Vision-and-Change-Final-Report.pdf Google Scholar
  • Antes, A. L., & DuBois, J. M. (2014). Aligning objectives and assessment in responsible conduct of research instruction. Journal of Microbiology & Biology Education, 15(2), 108–116. https://doi.org/10.1128/jmbe.v15i2.852 MedlineGoogle Scholar
  • Antes, A. L., Murphy, S. T., Waples, E. P., Mumford, M. D., Brown, R. P., Connelly, S., & Devenport, L. D. (2009). A meta-analysis of ethics instruction effectiveness in the sciences. Ethics & Behavior, 19(5), 379–402. https://doi.org/10.1080/10508420903035380 MedlineGoogle Scholar
  • Auchincloss, L. C., Laursen, S. L., Branchaw, J. L., Eagan, K., Graham, M., Hanauer, D. I., … Dolan, E. L. (2014). Assessment of course-based undergraduate research experiences: A meeting report. CBE—Life Sciences Education, 13(1), 29–40. https://doi.org/10.1187/cbe.14-01-0004 LinkGoogle Scholar
  • Ballen, C., Blum, J., Brownell, S., Hebert, S., Hewlett, J., Klein, J., … Cotner, S. (2017). A call to develop course-based undergraduate research experiences (CUREs) for nonmajors courses. CBE—Life Sciences Education, 16(2), mr2. https://doi.org/10.1187/cbe.16-12-0352 LinkGoogle Scholar
  • Bangera, G., & Brownell, S. E. (2014). Course-based undergraduate research experiences can make scientific research more inclusive. CBE—Life Sciences Education, 13(4), 602–606. https://doi.org/10.1187/cbe.14-06-0099 LinkGoogle Scholar
  • Batz, Z., Olsen, B. J., Dumont, J., Dastoor, F., & Smith, M. K. (2015). Helping struggling students in introductory biology: A peer-tutoring approach that improves performance, perception, and retention. CBE—Life Sciences Education, 14(2), ar16. Retrieved October 3, 2018, from http://eutils.ncbi.nlm.nih.gov/entrez/eutils/elink.fcgi?dbfrom=pubmed&id
=25976652&retmode=ref&cmd=prlinks LinkGoogle Scholar
  • Batzli, J. M. (2005). Points of view: A survey of survey courses: Are they effective? A unique approach? Four semesters of biology core curriculum. Cell Biology Education, 4, 125–128. LinkGoogle Scholar
  • Borenstein, J., Drake, M. J., Kirkman, R., & Swann, J. L. (2010). The Engineering and Science Issues Test (ESIT): A discipline-specific approach to assessing moral judgment. Science and Engineering Ethics, 16(2), 387–407. https://doi.org/10.1007/s11948-009-9148-z MedlineGoogle Scholar
  • Braunschweiger, P., & Goodman, K. W. (2007). The CITI program: An international online resource for education in human subjects protection and the responsible conduct of research. Academic Medicine, 8(9), 861–864. Retrieved October 3, 2018, from https://insights.ovid.com/pubmed?pmid
=17726392 Google Scholar
  • Brownell, S. E., Ekmat-Scafe, D. S., Singla, V., Chandler Seawell, P., Conklin Imam, J. F., Eddy, S. L., … Cyert, M. S. (2015). A high-enrollment course-based undergraduate research experience improves student conceptions of scientific thinking and ability to interpret data. CBE—Life Sciences Education, 14(2), ar21. https://doi.org/10.1187/cbe.14-05-0092 LinkGoogle Scholar
  • Brownell, S. E., & Kloser, M. J. (2015). Toward a conceptual framework for measuring the effectiveness of course-based undergraduate research experiences in undergraduate biology. Studies in Higher Education, 40(3), 525–544. https://doi.org/10.1080/03075079.2015.1004234 Google Scholar
  • Clements, J. D., Connell, N. D., Dirks, C., El-Faham, M., Hay, A., Heitman, E., … Husbands, J. L. (2013). Engaging actively with issues in the responsible conduct of science: Lessons from international efforts are relevant for undergraduate education in the United States. CBE—Life Sciences Education, 12(4), 596–603. LinkGoogle Scholar
  • Cooper, K. M., Soneral, P. A., & Brownell, S. E. (2017). Define your goals before you design a CURE: A call to use backward design in planning course-based undergraduate research experiences. Journal of Microbiology & Biology Education, 18(2). https://doi.org/10.1128/jmbe.v18i2.1287 Google Scholar
  • Corwin, L. A., Runyon, C. R., Ghanem, E., Sandy, M., Clark, G., Palmer, G. C., … Dolan, E. L. (2018). Effects of discovery, iteration, and collaboration in laboratory courses on undergraduates’ research career intentions fully mediated by student ownership. CBE—Life Sciences Education, 17(2), ar20. https://doi.org/10.1187/cbe.17-07-0141 LinkGoogle Scholar
  • Crowe, M., & Hill, C. (2006). Setting the stage for good group dynamics in a semester-long projects in the sciences. Journal of College Science Teaching, 35(4), 32–35. Google Scholar
  • Curzer, H. J., Sattler, S., Dupree, D. G., & Smith-Genthôs, K. R. (2014). Do ethics classes teach ethics? Theory and Research in Education, 12(3), 366–382. https://doi.org/10.1177/1477878514545209 Google Scholar
  • DuBois, J. M., Chibnall, J. T., Tait, R. C., Vander Wal, J. S., Baldwin, K. A., Antes, A. L., & Mumford, M. D. (2016). Professional Decision-Making in Research (PDR): The validity of a new measure. Science and Engineering Ethics, 22(2), 391–416. https://doi.org/10.1007/s11948-015-9667-8 MedlineGoogle Scholar
  • Dubois, J. M., & Dueker, J. M. (2009). Teaching and assessing the responsible conduct of research: A Delphi consensus panel report. Journal of Research Administration, 40(1), 49–70. Retrieved October 3, 2018, from www.ncbi.nlm.nih.gov/pubmed/22500145 MedlineGoogle Scholar
  • DuBois, J. M., Dueker, M. J. M., Anderson, E. E., & Campbell, J. (2008). The development and assessment of an NIH-funded research ethics training program. Academic Medicine, 83(6), 596. MedlineGoogle Scholar
  • Fisher, G. R., Olimpo, J. T., McCabe, T. M., & Pevey, R. S. (2018). The Tigriopus CURE—A course-based undergraduate research experience with concomitant supplemental instruction. Journal of Microbiology & Biology Education, 19(1). https://doi.org/10.1128/jmbe.v19i1.1503 MedlineGoogle Scholar
  • Frantz, K. J., Demetrikopoulos, M. K., Britner, S. L., Carruth, L. L., Williams, B. A., Pecore, J. L., … Goode, C. T. (2017). A comparison of internal dispositions and career trajectories after collaborative versus apprenticed research experiences for undergraduates. CBE—Life Sciences Education, 16(1), ar1. https://doi.org/10.1187/cbe.16-06-0206 LinkGoogle Scholar
  • Gasparich, G. E., & Wimmers, L. (2014). Integration of ethics across the curriculum: From first year through senior seminar. Journal of Microbiology & Biology Education, 15(2), 218–223. https://doi.org/10.1128/jmbe.v15i2.850 MedlineGoogle Scholar
  • Grobman, L.Kinkead, J. (Eds.). (2010). Undergraduate research in English studies. Urbana, IL: National Council of Teachers of English. Google Scholar
  • Harrison, M., Dunbar, D., Ratmansky, L., Boyd, K., & Lopatto, D. (2011). Classroom-based science research at the introductory level: Changes in career choices and attitude. CBE—Life Sciences Education, 10(3), 279–286. Retrieved October 3, 2018, from http://eutils.ncbi.nlm.nih.gov/entrez/eutils/elink.fcgi?dbfrom=pubmed&id=21885824&retmode=ref&cmd=prlinks LinkGoogle Scholar
  • Hirsch, P. L., Linsenmeier, J. A. W., Smith, H. D., & Walker, J. M. T. (2005). Enhancing core competency learning in an integrated summer research experience for bioengineers. Journal of Engineering Education, 94(4), 391–401. https://doi.org/10.1002/j.2168-9830.2005.tb00867.x Google Scholar
  • Jones, M., van Kessel, G., Swisher, L., Beckstead, J., & Edwards, I. (2014). Cognitive maps and the structure of observed learning outcome assessment of physiotherapy students’ ethical reasoning knowledge. Assessment & Evaluation in Higher Education, 39(1), 1–20. https://doi.org/10.1080/
02602938.2013.772951 Google Scholar
  • Jordan, T. C., Burnett, S. H., Carson, S., Caruso, S. M., Clase, K., DeJong, R. J., … Hatfull, G. F. (2014). A broadly implementable research course in phage discovery and genomics for first-year undergraduate students. MBio, 5(1), e01051–13. https://doi.org/10.1128/mBio.01051-13 MedlineGoogle Scholar
  • Kalichman, M. (2014). Rescuing RCR education. Accountability in Research, 21(1), 68–83. MedlineGoogle Scholar
  • Keefer, M. W., Wilson, S. E., Dankowicz, H., & Loui, M. C. (2014). The importance of formative assessment in science and engineering ethics education: Some evidence and practical advice. Science and Engineering Ethics, 20(1), 249–260. https://doi.org/10.1007/s11948-013-9428-5 MedlineGoogle Scholar
  • Keiler, K. C., Jackson, K. L., Jaworski, L., Lopatto, D., & Ades, S. E. (2017). Teaching broader impacts of science with undergraduate research. PLOS Biology, 15(3), e2001318. https://doi.org/10.1371/journal.pbio.2001318 MedlineGoogle Scholar
  • Lind, G. (1977). Moral competence test (MCT). Retrieved April 5, 2018, from www.uni-konstanz.de/ag-moral/mut/mjt-engl.htm Google Scholar
  • Loui, M. C., & Revelo, R. A. (2015). Cooperative learning and assessment of ethics sessions in a summer undergraduate research program. CUR Quarterly, 36(1), 4–10. Google Scholar
  • Mabrouk, P. A. (2016). What knowledge of responsible conduct of research do undergraduates bring to their undergraduate research experiences? Journal of Chemical Education, 93, 46–55. https://doi.org/10.1021/acs.jchemed.5b00264 Google Scholar
  • Minifie, F. D., Robey, R. R., Horner, J., Ingham, J. C., Lansing, C., McCartney, J. H., … Moss, S. E. (2011). Responsible conduct of research in communication sciences and disorders: Faculty and student perceptions. Journal of Speech Language and Hearing Research, 54(1), S363–S391. https://doi.org/10.1044/1092-4388(2010/09-0262) MedlineGoogle Scholar
  • National Academy of Sciences. (2009). On being a scientist: A guide to responsible conduct in research (3rd ed.). Washington, DC: National Academies Press. Google Scholar
  • National Institutes of Health (NIH). (1989). Requirement for programs on the responsible conduct of research in national research service award institutional training programs. In NIH guide for grants and contracts (Vol. 18, no. 45, p. 1). Bethesda, MD. Retrieved October 3, 2018, from https://grants.nih.gov/grants/guide/historical/1989_12_22_Vol_18_No_45.pdf Google Scholar
  • NIH. (2009). NOT-OD-10-019: Update on the requirement for instruction in the responsible conduct of research. Retrieved October 3, 2018, from https://grants.nih.gov/grants/guide/notice-files/NOT-OD-10-019.html Google Scholar
  • Nebeker, C. (2014). Smart teaching matters! Applying the research on learning to teaching RCR. Journal of Microbiology & Biology Education, 15(2), 88–92. https://doi.org/10.1128/jmbe.v15i2.849 MedlineGoogle Scholar
  • Office for Human Research Protections. (2019). International compilation of human research protections. Retrieved January 10, 2019, from www.hhs
.gov/ohrp/international/compilation-human-research-standards/index
.html Google Scholar
  • Olimpo, J. T., Diaz-Martinez, L. A., Bhatt, J. M., & D’Arcy, C. E. (2017). Integration of RCR and ethics education into course-based undergraduate research experiences in the biological sciences: A needed discussion. Journal of Microbiology & Biology Education, 18(2). https://doi.org/10.1128/jmbe.v18i2.1344 MedlineGoogle Scholar
  • Olimpo, J. T., Fisher, G. R., & DeChenne-Peters, S. E. (2016). Development and evaluation of the Tigriopus course-based undergraduate research experience: Impacts on students’ content knowledge, attitudes, and motivation in a majors introductory biology course. CBE—Life Sciences Education, 15(4), ar72. https://doi.org/10.1187/cbe.15-11-0228 LinkGoogle Scholar
  • Olson, L. E. (2010). Developing a framework for assessing responsible conduct of research education programs. Science and Engineering Ethics, 16(1), 185–200. https://doi.org/10.1007/s11948-010-9196-4 MedlineGoogle Scholar
  • Peiffer, A. M., Hugenschmidt, C. E., & Laurienti, P. J. (2011). Ethics in 15 min per week. Science and Engineering Ethics, 17(2), 289–297. https://doi.org/10.1007/s11948-010-9197-3 MedlineGoogle Scholar
  • Phillips, T., Nestor, F., Beach, G., & Heitman, E. (2018). America COMPETES at 5 years: An analysis of research-intensive universities’ RCR training plans. Science and Engineering Ethics, 24(1), 227–249. https://doi.org/10.1007/s11948-017-9883-5 MedlineGoogle Scholar
  • Resnik, D. B. (2011). What is ethics in research & why is it important? National Institute of Environmental Health Sciences, 1(10), 49–70. Retrieved October 3, 2019, from www.niehs.nih.gov/research/resources/bioethics/whatis/index.cfm Google Scholar
  • Rest, J. R., Narvaez, D., Thoma, S. J., & Bebeau, M. J. (1999). DIT2: Devising and testing a revised instrument of moral judgment. Journal of Educational Psychology, 91(4), 644–659. Retrieved October 3, 2018, from www.scopus.com/record/display.uri?eid=2-s2.0-0033263069&origin
=inward&txGid=f4f684fc0f605f6d3c4feacd2a348296 Google Scholar
  • Rezler, A. G., Schwartz, R. L., Obenshain, S. S., Lambert, P., McLGibson, J., & Bennahum, D. A. (1992). Assessment of ethical decisions and values. Medical Education, 26(1), 7–16. https://doi.org/10.1111/j.1365-2923.1992
.tb00115.x MedlineGoogle Scholar
  • Rhodes, T. (2009). Assessing outcomes and improving achievement: Tips and tools for using the rubrics. Washington, DC: Association of American Colleges & Universities. Retrieved October 3, 2018, from www.aacu.org/value-rubrics Google Scholar
  • Rodenbusch, S. E., Hernandez, P. R., Simmons, S. L., & Dolan, E. L. (2016). Early engagement in course-based research increases graduation rates and completion of science, engineering, and mathematics degrees. CBE—Life Sciences Education, 15(2), ar20. https://doi.org/10.1187/cbe.16-03-0117 LinkGoogle Scholar
  • Seiler, S. N., Brummel, B. J., Anderson, K. L., Kim, K. J., Wee, S., Gunsalus, C. K., & Loui, M. C. (2011). Outcomes assessment of role-play scenarios for teaching responsible conduct of research. Accountability in Research, 18(4), 217–246. https://doi.org/10.1080/08989621.2011.584760 MedlineGoogle Scholar
  • Shachter, A. M. (2003). Integrating ethics in science into a summer undergraduate research program. Journal of Chemical Education, 80(5), 507. https://doi.org/10.1021/ed080p507 Google Scholar
  • Shaffer, C. D., Alvarez, C. J., Bednarski, A. E., Dunbar, D., Goodman, A. L., Reinke, C., … Elgin, S. C. R. (2014). A course-based research experience: How benefits change with increased investment in instructional time. CBE—Life Sciences Education, 13(1), 111–130. https://doi.org/10.1187/cbe-13-08-0152 LinkGoogle Scholar
  • Shanahan, J. O. (2011). Scaffolding of research skills in the humanities curriculum. In Yaveneh Klos, N.Olin Shanahan, J.Young, G. (Eds.), Creative inquiry in the arts & humanities (pp. 7–14). Washington, DC: Council on Undergraduate Research. Google Scholar
  • Smith, K., Wueste, D., & Frugoli, J. (2007). Using “ethics labs” to set a framework for ethical discussion in an undergraduate science course. Biochemistry and Molecular Biology Education, 35(5), 332–336. https://doi.org/10.1002/bmb.94 MedlineGoogle Scholar
  • Spell, R. M., Guinan, J. A., Miller, K. R., & Beck, C. W. (2014). Redefining authentic research experiences in introductory biology laboratories and barriers to their implementation. CBE—Life Sciences Education, 13(1), 102–110. https://doi.org/10.1187/cbe.13-08-0169 LinkGoogle Scholar
  • Steele, L. M., Mulhearn, T. J., Medeiros, K. E., Watts, L. L., Connelly, S., & Mumford, M. D. (2016). How do we know what works? A review and critique of current practices in ethics training evaluation. Accountability in Research, 23(6), 319–350. https://doi.org/10.1080/08989621.2016.1186547 MedlineGoogle Scholar
  • Steneck, N. H. (2007). Introduction to the responsible conduct of research (rev. ed.). Washington, DC: Office of Research Integrity. Retrieved October 3, 2018, from https://ori.hhs.gov/sites/default/files/rcrintro.pdf Google Scholar
  • Steneck, N. H., & Bulger, R. E. (2007). The history, purpose, and future of instruction in the responsible conduct of research. Academic Medicine, 82(9), 829–834. https://doi.org/10.1097/ACM.0b013e31812f7d4d MedlineGoogle Scholar
  • Taylor, H. A., Kass, N. E., Ali, J., Sisson, S., Bertram, A., & Bhan, A. (2012). Development of a research ethics knowledge and analytical skills assessment tool. Journal of Medical Ethics, 38(4), 236–242. https://doi.org/10.1136/medethics-2011-100025 MedlineGoogle Scholar
  • Teixeira-Poit, S. M., Cameron, A. E., & Schulman, M. D. (2011). Experiential learning and research ethics: Enhancing knowledge through action. Teaching Sociology, 39(3), 244–258. https://doi.org/10.1177/0092055X11407346 Google Scholar
  • Thompson, C. J. (2014). Responsible conduct of research assessment of doctor of education candidates, graduate faculty, and curriculum considerations. Innovative Higher Education, 39(5), 349–360. https://doi.org/10.1007/s10755-014-9289-0 Google Scholar
  • Tractenberg, R. E., & FitzGerald, K. T. (2012). A mastery rubric for the design and evaluation of an institutional curriculum in the responsible conduct of research. Assessment & Evaluation in Higher Education, 37(8), 1003–1021. https://doi.org/10.1080/02602938.2011.596923 Google Scholar
  • Wahila, M. J., Amey-Proper, J., Jones, W. E., Stamp, N., & Piper, L. F. J. (2017). Teaching advanced science concepts through freshman research immersion. European Journal of Physics, 38(2), 025704. https://doi.org/10.1088/1361-6404/aa5959 Google Scholar
  • Ward, J. R., Clarke, H. D., & Horton, J. L. (2014). Effects of a research-infused botanical curriculum on undergraduates’ content knowledge, STEM competencies, and attitudes toward plant sciences. CBE—Life Sciences Education, 13(3), 387–396. https://doi.org/10.1187/cbe.13-12-0231 LinkGoogle Scholar
  • Watts, L. L., Medeiros, K. E., Mulhearn, T. J., Steele, L. M., Connelly, S., & Mumford, M. D. (2017). Are ethics training programs improving? A meta-analytic review of past and present ethics instruction in the sciences. Ethics & Behavior, 27(5), 351–384. https://doi.org/10.1080/10508422.2016.1182025 MedlineGoogle Scholar
  • Wei, C. A., & Woodin, T. (2011). Undergraduate research experiences in biology: Alternatives to the apprenticeship model. CBE—Life Sciences Education, 10(2), 123–131. https://doi.org/10.1187/cbe.11-03-0028 LinkGoogle Scholar
  • Yarborough, M., & Hunter, L. (2013). Teaching research ethics better: Focus on excellent science, not bad scientists. Clinical and Translational Science, 6(3), 201–203. https://doi.org/10.1111/cts.12035 MedlineGoogle Scholar