Years of teaching experience have long been taken as an important indicator of teacher and school quality in primary and secondary schools. However, research on the effects of teacher years of experience on student outcomes has shown mixed results. To investigate the source of variation causing these mixed results, this study examined a broad range of student and classroom compositions that might affect the relationship between teacher years of experience and student outcomes, including standardized test scores in English Language Arts and Mathematics, Social-Emotional Test scores, and the count and severity of student disciplinary incidents. Using multilevel regression, this study analyzed longitudinal data obtained from the largest school district in the state of Arizona, Mesa Unified School District, which is classified as a mid-high poverty district. Despite the large sample size, a very weak and non-significant relationship was found for teacher years of experience on any student outcome. In contrast, student and family demographic backgrounds were significantly related to student outcomes. English and Math scores were significantly associated with student giftedness, home languages, and English learning status. Social-emotional test scores were significantly related to race/ethnicity, home languages, and special education status. Both discipline counts and severity were statistically associated with race/ethnicity, and gender, while special education status was significantly related only to discipline severity. In addition, classroom composition variables based on student and family demographics, and students’ prior year scores on the outcome were significantly related to student outcomes. These results suggest that student characteristics, contextual classroom composition variables arising from student backgrounds are more important in predicting student outcomes than teacher years of experience, at least in a mid-high poverty school district. Therefore, student placements might be a key to addressing educational inequity for high-need students in mid-high poverty schools, consistent with prior studies in high-poverty schools.

Similar-identity role models, including instructors, can benefit science undergraduates by enhancing their self-efficacy and sense of belonging. However, for students to have similar-identity role models based on identities that can be hidden, instructors need to disclose their identities. For concealable stigmatized identities (CSIs) – identities that can be hidden and carry negative stereotypes – the impersonal and apolitical culture cultivated in many science disciplines likely makes instructor CSI disclosure unlikely. This dissertation comprises five studies I conducted to assess the presence of instructor role models with CSIs in undergraduate science classrooms and evaluate the impact on undergraduates of instructor CSI disclosure. I find that science instructors report CSIs at lower rates than undergraduates and typically keep these identities concealed. Additionally, I find that women instructors are more likely to disclose their CSIs to students compared to men. To assess the impact of instructor CSI disclosure on undergraduates, I report on findings from a descriptive exploratory study and a controlled field experiment in which an instructor reveals an LGBTQ+ identity. Undergraduates, especially those who also identify as LGBTQ+, benefit from instructor LGBTQ+ disclosure. Additionally, the majority of undergraduate participants agree that an instructor revealing an LGBTQ+ identity during class is appropriate. Together, the results presented in this dissertation highlight the current lack of instructor role models with CSIs and provide evidence of student benefits that may encourage instructors to reveal CSIs to undergraduates and subsequently provide much-needed role models. I hope this work can spark self-reflection among instructors to consider revealing CSIs to students and challenge the assumption that science environments should be devoid of personal identities.

This paper examines the effect of a weekly student self-assessment assignment on student performance in an undergraduate math course. Self-assessment is an increasingly popular type of formative assessment with close ties to self-regulated learning theory. In this randomized controlled trial, 88 students enrolled in MAT 142 were divided into four treatment groups, receiving the self-assessment assignment for either half the semester, the full semester, or not at all. There was no main effect of the treatment on students’ course performance (F(3,80) = 0.154, p = 0.999). However, students’ level of compliance with the assignments (F(1, 63) = 6.87, p = 0.011) and class attendance (F(1, 83) = 12.34, p < 0.001) both significantly predicted exam scores, suggesting that conscientiousness predicts performance. I conducted focus groups to understand how students felt toward the self-assessments. Participants expressed distaste toward the assignments and provided suggestions for improvements. I describe these improvements, among others, in an effort to outline future directions for this research. I also describe a new model of student self-assessment based on theories of adaptive testing and self-regulated learning.

This dissertation comprises two projects: (i) Multiple testing of local maxima for detection of peaks and change points with non-stationary noise, and (ii) Height distributions of critical points of smooth isotropic Gaussian fields: computations, simulations and asymptotics. The first project introduces a topological multiple testing method for one-dimensional domains to detect signals in the presence of non-stationary Gaussian noise. The approach involves conducting tests at local maxima based on two observation conditions: (i) the noise is smooth with unit variance and (ii) the noise is not smooth where kernel smoothing is applied to increase the signal-to-noise ratio (SNR). The smoothed signals are then standardized, which ensures that the variance of the new sequence's noise becomes one, making it possible to calculate $p$-values for all local maxima using random field theory. Assuming unimodal true signals with finite support and non-stationary Gaussian noise that can be repeatedly observed. The algorithm introduced in this work, demonstrates asymptotic strong control of the False Discovery Rate (FDR) and power consistency as the number of sequence repetitions and signal strength increase. Simulations indicate that FDR levels can also be controlled under non-asymptotic conditions with finite repetitions. The application of this algorithm to change point detection also guarantees FDR control and power consistency. The second project focuses on investigating the explicit and asymptotic height densities of critical points of smooth isotropic Gaussian random fields on both Euclidean space and spheres.The formulae are based on characterizing the distribution of the Hessian of the Gaussian field using the Gaussian orthogonally invariant (GOI) matrices and the Gaussian orthogonal ensemble (GOE) matrices, which are special cases of GOI matrices. However, as the dimension increases, calculating explicit formulae becomes computationally challenging. The project includes two simulation methods for these distributions. Additionally, asymptotic distributions are obtained by utilizing the asymptotic distribution of the eigenvalues (excluding the maximum eigenvalues) of the GOE matrix for large dimensions. However, when it comes to the maximum eigenvalue, the Tracy-Widom distribution is utilized. Simulation results demonstrate the close approximation between the asymptotic distribution and the real distribution when $N$ is sufficiently large.

This dissertation explores applications of machine learning methods in service of the design of screening tests, which are ubiquitous in applications from social work, to criminology, to healthcare. In the first part, a novel Bayesian decision theory framework is presented for designing tree-based adaptive tests. On an application to youth delinquency in Honduras, the method produces a 15-item instrument that is almost as accurate as a full-length 150+ item test. The framework includes specific considerations for the context in which the test will be administered, and provides uncertainty quantification around the trade-offs of shortening lengthy tests. In the second part, classification complexity is explored via theoretical and empirical results from statistical learning theory, information theory, and empirical data complexity measures. A simulation study that explicitly controls two key aspects of classification complexity is performed to relate the theoretical and empirical approaches. Throughout, a unified language and notation that formalizes classification complexity is developed; this same notation is used in subsequent chapters to discuss classification complexity in the context of a speech-based screening test. In the final part, the relative merits of task and feature engineering when designing a speech-based cognitive screening test are explored. Through an extensive classification analysis on a clinical speech dataset from patients with normal cognition and Alzheimer’s disease, the speech elicitation task is shown to have a large impact on test accuracy; carefully performed task and feature engineering are required for best results. A new framework for objectively quantifying speech elicitation tasks is introduced, and two methods are proposed for automatically extracting insights into the aspects of the speech elicitation task that are driving classification performance. The dissertation closes with recommendations for how to evaluate the obtained insights and use them to guide future design of speech-based screening tests.

Integrating agent-based models (ABMs) has been a popular approach for teaching emergent science concepts. However, students continue to find it difficult to explain the emergent process of natural selection. This study adopted an ontological framework–the Pattern, Agents, Interactions, Relations, and Causality (PAIR-C)–to guide the design of learning modules. This pre-posttest experimental study examines the effects of the PAIR-C module versus the Regular module on fostering students’ deep understanding of natural selection. Results show that students in the PAIR-C intervention group performed better in answering deep questions assessing the understanding of inter-level causal relationships than those in the Regular control group. Although students in both groups did not show significantly improved abilities in explaining the natural selection process for other contexts or significant differences in their abilities to explain other emergent phenomena, students in the intervention group demonstrated system-thinking perspectives and fewer misconceptions in their expressions compared to the control group. A close analysis of student misconceptions consolidates that the intervention group demonstrated drastically fewer categories and numbers of misconceptions while those in the control group did not show such drastic changes before and after the study. To precisely address misconceptions and further improve students’ learning outcomes, Epistemic Network Analysis was adopted to capture students’ misconception characteristics by examining the co-occurrences of different misconception categories as well as the relationship between misconceptions and PAIR-C features. The results of student learning outcomes and misconception characteristics collectively provide directions for improving the instructional design of the PAIR-C module. Furthermore, findings on student engagement levels during learning can also inform future design efforts. Overall, this project sheds light on applying an innovative framework to designing effective learning modules to teach emergent science concepts.

Goodness-of-fit test is a hypothesis test used to test whether a given model fit the data well. It is extremely difficult to find a universal goodness-of-fit test that can test all types of statistical models. Moreover, traditional Pearson’s chi-square goodness-of-fit test is sometimes considered to be an omnibus test but not a directional test so it is hard to find the source of poor fit when the null hypothesis is rejected and it will lose its validity and effectiveness in some of the special conditions. Sparseness is such an abnormal condition. One effective way to overcome the adverse effects of sparseness is to use limited-information statistics. In this dissertation, two topics about constructing and using limited-information statistics to overcome sparseness for binary data will be included. In the first topic, the theoretical framework of pairwise concordance and the transformation matrix which is used to extract the corresponding marginals and their generalizations are provided. Then a series of new chi-square test statistics and corresponding orthogonal components are proposed, which are used to detect the model misspecification for longitudinal binary data. One of the important conclusions is, the test statistic $X^2_{2c}$ can be taken as an extension of $X^2_{[2]}$, the second-order marginals of traditional Pearson’s chi-square statistic. In the second topic, the research interest is to investigate the effect caused by different intercept patterns when using Lagrange multiplier (LM) test to find the source of misfit for two items in 2-PL IRT model. Several other directional chi-square test statistics are taken into comparison. The simulation results showed that the intercept pattern does affect the performance of goodness-of-fit test, especially the power to find the source of misfit if the source of misfit does exist. More specifically, the power is directly affected by the `intercept distance' between two misfit variables. Another discovery is, the LM test statistic has the best balance between the accurate Type I error rates and high empirical power, which indicates the LM test is a robust test.

Innovations in undergraduate education have increased the prevalence of active learning courses, online education, and student engagement in the high-impact practice of undergraduate research, however it is unknown whether students with disabilities are able to engage in these innovative learning environments to the same extent that they are able to engage in more traditional learning environments. Universities, disability resource centers, and instructors are mandated to provide accommodations to students with disabilities for the purposes of prohibiting discrimination and ensuring equal access to opportunities for individuals with disabilities. Are accommodations being adapted and created for these new types of learning environments? This dissertation reports findings from four studies about the experiences of students with disabilities in these three learning environments, specifically examining the challenges students with disabilities encounter and the emerging recommendations for more effective accommodations. I find that students with disabilities experience challenges in each of these learning environments and that the current suite of accommodations are not sufficient for students with disabilities. I argue that institutions need to consider modifying student accommodations and the process for obtaining them to better support students with disabilities in these evolving learning environments. I also provide recommendations for the ways in which undergraduate science education can be made more accessible and inclusive of students with disabilities.

In this mixed-methods study, I sought to design and develop a test delivery method to reduce linguistic bias in English-based mathematics tests. Guided by translanguaging, a recent linguistic theory recognizing the complexity of multilingualism, I designed a computer-based test delivery method allowing test-takers to toggle between English and their self-identified dominant language. This three-part study asks and answers research questions from all phases of the novel test delivery design. In the first phase, I conducted cognitive interviews with 11 Mandarin Chinese dominant speakers and 11 Spanish speaking dominant undergraduate students while taking a well-regarded calculus conceptual exam, the Precalculus Concept Assessment (PCA). In the second phase, I designed and developed the linguistically adaptive test (LAT) version of the PCA using the Concerto test delivery platform. In the third phase, I conducted a within-subjects random-assignment study of the efficacy the LAT. I also conducted in-depth interviews with a subset of the test-takers. Nine items on the PCA revealed linguistic issues during the cognitive interviews demonstrating the need to improve the linguistic bias on the test items. Additionally, the newly developed LAT demonstrated evidence of reliability and validity. However, the large-scale efficacy study showed that the LAT did not appear to make a significant difference in scores for dominant speakers of Spanish or dominant speakers of Mandarin Chinese. This finding held true for overall test scores as well as at the item level indicating that the LAT test delivery system does not appear to reduce linguistic bias in testing. Additionally, in-depth interviews revealed that many students felt that the linguistically adaptive test was either the same or essentially the same as the non-LAT version of the test. Some participants felt that the toggle button was not necessary if they could understand the mathematics item well enough. As one participant noted, “It's math, It's math. It doesn't matter if it's in English or in Spanish.” This dissertation concludes with a discussion about the implications for test developers and suggestions for future direction of study.