ABSTRACT

This study examines validity evidence of a state policy-directed teacher evaluation system implemented in Arizona during school year 2012-2013. The purpose was to evaluate the warrant for making high stakes, consequential judgments of teacher competence based on value-added (VAM) estimates of instructional impact and observations of professional practice (PP). The research also explores educator influence (voice) in evaluation design and the role information brokers have in local decision making. Findings are situated in an evidentiary and policy context at both the LEA and state policy levels.

The study employs a single-phase, concurrent, mixed-methods research design triangulating multiple sources of qualitative and quantitative evidence onto a single (unified) validation construct: Teacher Instructional Quality. It focuses on assessing the characteristics of metrics used to construct quantitative ratings of instructional competence and the alignment of stakeholder perspectives to facets implicit in the evaluation framework. Validity examinations include assembly of criterion, content, reliability, consequential and construct articulation evidences. Perceptual perspectives were obtained from teachers, principals, district leadership, and state policy decision makers. Data for this study came from a large suburban public school district in metropolitan Phoenix, Arizona.

Study findings suggest that the evaluation framework is insufficient for supporting high stakes, consequential inferences of teacher instructional quality. This is based, in part on the following: (1) Weak associations between VAM and PP metrics; (2) Unstable VAM measures across time and between tested content areas; (3) Less than adequate scale reliabilities; (4) Lack of coherence between theorized and empirical PP factor structures; (5) Omission/underrepresentation of important instructional attributes/effects; (6) Stakeholder concerns over rater consistency, bias, and the inability of test scores to adequately represent instructional competence; (7) Negative sentiments regarding the system's ability to improve instructional competence and/or student learning; (8) Concerns regarding unintended consequences including increased stress, lower morale, harm to professional identity, and restricted learning opportunities; and (9) The general lack of empowerment and educator exclusion from the decision making process. Study findings also highlight the value of information brokers in policy decision making and the importance of having access to unbiased empirical information during the design and implementation phases of important change initiatives.

The purpose of this mixed methods research study was to assess the impact and influence of a pre-international experience course on Arizona State University (ASU) students before they study or intern abroad. Currently, the study abroad pre-departure orientation for ASU participants consists of online modules and a two-hour face-to-face orientation. In this action research study, the practitioner-researcher re-designed an ASU School of Politics and Global Studies (SPGS), one-credit course that focused exclusively on cross-cultural awareness and sensitivity. A needs assessment was distributed to a sample of 800 returning study abroad participants and was used to influence the study, along with an extensive literature review and two cycles of action research. The dissertation research and study was conducted during the ASU fall 2013 semester. Quantitative data and qualitative data were collected using eight different measures. To better understand the impact of a pre-international experience curriculum for ASU study abroad and international internship participants before they go abroad, this research study investigated the following research questions: (1) What cultural impact does a pre-international experience course have on students who complete the course before studying or interning abroad? (2) What specific cultural competencies are gained by the participants after participating in the pre-international experience course? (3) How has developing the curriculum, teaching the curriculum and implementing the innovation influenced and informed my practice as an international educator and the Assistant Director of the Arizona State Study Abroad Office? The following five assertions were identified within the quantitative and qualitative analysis of the collected data to answer the three research questions: (1) Students are more confident in their abilities to cross cultures after successfully completing taking the new course; (2) Students are more aware of other cultures and their own culture after successfully completing taking the new course; (3) Students gained important knowledge about understanding others' worldviews after successfully completing taking the new course; (4) Students gained general openness toward intercultural learning and to people from cultures different from their own after successfully completing the new course; (5) Developing and implementing a pre-international experience course changed me as a leader, instructor and researcher. Implications for future implementation and research are discussed.

Virtual Patient Simulations (VPS) are web-based exercises involving simulated patients in virtual environments. This study investigates the utility of VPS for increasing medical student clinical reasoning skills, collaboration, and engagement. Many studies indicate that VPS provide medical students with essential practice in clinical decision making before they encounter real life patients. The utility of a recursive, inductive VPS for increasing clinical decision-making skills, collaboration, or engagement is unknown. Following a design-based methodology, VPS were implemented in two phases with two different cohorts of first year medical students: spring and fall of 2013. Participants were 108 medical students and six of their clinical faculty tutors. Students collaborated in teams of three to complete a series of virtual patient cases, submitting a ballpark diagnosis at the conclusion of each session. Student participants subsequently completed an electronic, 28-item Exit Survey. Finally, students participated in a randomized controlled trial comparing traditional (tutor-led) and VPS case instruction methods. This sequence of activities rendered quantitative and qualitative data that were triangulated during data analysis to increase the validity of findings. After practicing through four VPS cases, student triad teams selected accurate ballpark diagnosis 92 percent of the time. Pre-post test results revealed that PPT was significantly more effective than VPS after 20 minutes of instruction. PPT instruction resulted in significantly higher learning gains, but both modalities supported significant learning gains in clinical reasoning. Students collaborated well and held rich clinical discussions; the central phenomenon that emerged was "synthesizing evidence inductively to make clinical decisions." Using an inductive process, student teams collaborated to analyze patient data, and in nearly all instances successfully solved the case, while remaining cognitively engaged. This is the first design-based study regarding virtual patient simulation, reporting iterative phases of implementation and design improvement, culminating in local theories (petite generalizations) about VPS design. A thick, rich description of environment, process, and findings may benefit other researchers and institutions in designing and implementing effective VPS.

ABSTRACT Education policymakers at the national level have initiated reforms in K-12 education for that past several years that have focused on teacher quality and teacher evaluation. More recently, reforms have included legislation that focuses on administrator quality as well. Included in far-reaching recent legislation in Arizona is a requirement that administrators be evaluated on a standards-based evaluation system that is linked to student outcomes. The end result is an annual summative measure of administrator effectiveness that impacts job retention. Because of this, Arizona administrators have become concerned about rapidly becoming proficient in the new evaluation systems. Administrators rarely have the explicit professional development opportunities they need to collaborate on a shared understanding of these new evaluation systems. This action research study focused on a group of eight administrators in a small urban district grappling with a new, complex, and high-stakes administrator evaluation that is a component of an all-encompassing Teacher Incentive Fund Grant. An existing professional learning time was engaged to assist administrators in lessening their concerns and increasing their understanding and use of the evaluation instrument. Activities were designed to engage the administrators in dynamic, contextualized learning. Participants interacted in a group to interpret the meaning of the evaluation instrument share practical knowledge and support each other's acquisition understanding. Data were gathered with mixed methods. Administrators were given pre-and post-surveys prior to and immediately after this six-week innovation. Formal and informal interviews were conduct throughout the innovation. Additionally, detailed records in the form of meeting records and a researcher journal were kept. Qualitative and quantitative data were triangulated to validate findings. Results identified concerns and understanding of administrators as they attempted to come to a shared understanding of the new evaluation instrument. As a result of learning together, their concerns about the use of the instrument lessened. Other concerns however, remained or increased. Administrators found the process of the Administrator Learning Community valuable and felt their understanding and use of the instrument had increased. Intense concerns about the competing priorities and initiatives led to the administrators to consider a reevaluation of the competing initiatives. Implications from this study can be used to help other administrators and professional development facilitators grappling with common concerns.

This mixed methods participatory action research study explored how an instructional coach influenced a state mandated curriculum adoption at a Title 1 urban middle school. The purpose of this study was to identify ways in which an instructional coach supported a veteran staff during the adoption of new curriculum standards. The instructional coach/action researcher employed a three pronged coaching approach that incorporated individual and team coaching sessions and increased networking to encourage and support the development of social capital. This study was informed using Vygotsky's Social Learning Theory, Wenger's Communities of Practice, Coleman's Social Capital Theory, and Hall and Horde's Concerns-Based Adoption Model. The study is heavily weighted in favor of qualitative data which includes participant reflections, coach individual session and team session reflections, field-notes, team meeting videos, and exit interviews. Several themes emerged supporting the use of a differentiated coaching approach, the promotion of social capital, and the identification of initiative overload as a barrier to curriculum adoption. The quantitative data analysis, pre and post study Stages of Concern Questionnaires, produced evidence that participants experienced minor shifts in their concerns relating to the adoption of Common Core State Standards. Results were used to inform coaching decisions based on individual participant needs as well as to augment the qualitative findings. Ideas for further research are discussed.

The pathway for entering the K-12 classroom as a teacher varies compared to what was once the traditional model of teacher preparation. In this mixed-methods action research study, I explore supporting first year alternatively certified urban and rural intern teachers through a multicomponent distance induction program. The induction model in this study was based on the theoretical framework of Bandura's social learning theory and Wenger's communities of practice. The purpose of this study was to identify the extent in which a multicomponent distance learning induction program impacts first year intern teachers' sense of self-efficacy, understand their successes, their challenges, and to identify how intern teacher evaluations change. Quantitative data included results from a self-efficacy survey and the Student Teaching Assessment Instrument (STAI). Qualitative data was collected through intern teachers' blogs, cadre leader video narratives, and cadre leader STAI narrative responses. Six themes emerged including topics such as building relationships with other education professionals, receiving feedback from the cadre leader, identifying struggles and application of college coursework into the K-12 classroom. Key findings reveal interns are least efficacious in student engagement, relationships with other educators support an intern teacher emotionally and pedagogically, intern teachers struggle with work-life balance, and cadre leaders observed intern teachers as having improved their skills in student engagement, instructional practices, and classroom management. Implications to practice include a structured approach to introducing student engagement, creating a best practices library of video examples, and a pre-orientation (Super Saturday) of topics prior to stepping into the classroom with students.

This mixed methods action research study explores the impact of a multilevel intervention on retention indicators of special education induction teachers and the leadership capacities of the special education induction coaches and coordinator. The purpose of this investigation was to understand the impact of developing and implementing an action research study on three different levels of participants the special education induction coaches, teachers and me. A theoretical framework based upon Bandura's (1977, 1982) work in Social Learning Theory, and in self and collective efficacy informs this study. The conceptual framework developed based upon the tenets of Authentic Leadership Theory and special education mentor programs inform the development of the intervention and data collection tools. Quantitative data included results collected from the Psychological Capital Questionnaire (PCQ), Authentic Leadership Questionnaire (ALQ), and the Special Education Induction Teacher Questionnaire (SEITQ). The qualitative data included results collected from the SEITQ open-ended questions, Email Reflective Response (ERR), organic and structured focus groups, fieldnotes, and the Teachers' Final Letter. Findings include: a) I changed as a leader and a researcher, b) the special education induction coaches began to think and act as leaders, c) the special education induction teachers' retention indicators increased, d) by actively participating in the co-construction of the special education induction program, both the coaches and the teacher provided valuable insights as pertains to developing a program that supports special education induction teachers. Implications and next steps are discussed.

ABSTRACT Professional Development (PD) is an important tool in the field of education. Successful PD programs are those that include adult learning methods and opportunities for experiential learning and discussion. The university where this action research was conducted does not offer formal training to adjunct instructors. The adjunct instructors are hired based primarily on their content knowledge. This research was conducted to understand, if the application of a blended training model for adjuncts influences the adjunct's perception of meeting their student's educational needs and the student's perception that their personal education needs are met. The blended learning included the delivery of a framework that incorporated Andragogy, Content Knowledge and Technology (ACKT). The purpose of the ACKT framework is to supplement adjunct's content knowledge expertise with adult learning methods and technology. The effectiveness of the framework was measured by using a quasi-experimental, pre to post intervention assessment. The treatment group and control group each contained twenty-two adjunct instructors from the university. The treatment group received training on the framework prior to commencing the class and participated in two focus groups during the semester. In addition, the treatment group was observed teaching in their classroom. The control group did not receive training, or participate in focus groups; however they were observed teaching in their classroom. The results of the action research showed significant improvement for the adjunct instructors in the treatment group. Specifically, knowledge of and application of andragogy showed a large improvement. In addition, the social influence of the adjuncts in the treatment group showed a large improvement. Less significant was the improvement in the efficacy of the students in the treatment group classes compared to those in the control group classes. However, the data suggests that the students in the treatment group better applied the content learned and they were more aware of other's educational needs than their peers in the control group. The study supports the need for adjunct instructor PD. Through a PD program adjunct instructors increase their own efficacy and this improvement translates into increased content transfer for the students in the classroom. Based on the strong evidence for adjunct instructor improvement this research will continue by expanding the blended learning model to more of the adjunct instructors at the university, and continuing to evaluate the effectiveness of the model in meeting student's educational needs.

This action research project explores masters level graduate student writing and academic identity during one semester in an interdisciplinary masters program. Informing this study is a two part theoretical framework including the Academic Literacy Model (Lea and Street) and Wenger's concept of identity. The purpose of this exploration was to understand how first semester graduate students experienced academic writing and what characteristics of their academic identity emerged. A mixed-methods approach was used to collect both quantitative and qualitative data. Quantitative data included results from the Inventory of Processes in Graduate Writing (Lavelle and Bushrow, 2007) and the Graduate Student Identity Survey. Qualitative data was collected through researcher observations, student blog entries, writing group transcripts, and individual interviews. The following themes emerge from the data: a) graduate students attribute their successes in writing to previous experiences, b) graduate students experience struggles related primarily to academic quality and faculty expectations, c) graduate students negotiate ways of being in the academy through figuring out expectations of faculty and program, d) work done in the writing group meetings shows evidence of meaning-making for the graduate students, e) the focus of the MA program was critically important to graduate students in the graduate writing project, e) participants' role as graduate students felt most strongly in contexts that include academic activity, and f) students acknowledge change and increasingly identify themselves as writers. Ideas for future cycles of research are discussed.