Assessing College Readiness: Internal Structure Validity Evidence for Multistage Diagnostic Reading Assessment

April 25, 2025, Denver, CO

Oxana Rosca, Angela Lui, Kimberly Colvin, Heidi Andrade, Jason Bryer

Abstract This study provides internal structure validity evidence for the multistage, testlet-based DAACS Reading Assessment, a free and formative tool designed to support college readiness among diverse incoming undergraduate students. Using data from 4,523 first-attempt examinees across two institutions, we evaluated dimensionality, item properties, and fairness using modern psychometric methods. Parallel analyses and one-factor CFAs showed that 29 of 30 reading testlets exhibited essential unidimensionality. Bifactor 2PL models demonstrated significantly better fit than 1PL models, and local dependence was minimal across nearly all item pairs. After removing seven items with extreme or weak discrimination, the refined 173item model showed improved fit, stronger general-factor saturation, and stable theta score distributions. Item difficulty estimates indicated that the assessment was generally easy for newly enrolled students, consistent with expectations for a diagnostic, readiness-focused measure. Logistic regression DIF analyses produced preliminary findings due to limited sample size and statistical power, and these DIF results are currently being updated using additional data collected during the 2023 datac ollection cycle.

What this means for users Results indicate that DAACS Reading scores provide a consistent and accurate reflection of students’ reading proficiency, supporting their use for formative feedback that helps new college students understand their current skills and identify areas for improvement.

View the paper here https://www.researchgate.net/publication/394076325_Assessing_College_Readiness_Internal_Structure_Validity_Evidence_for_Multistage_Diagnostic_Reading_Assessment