We look forward to seeing you at our upcoming presentations

LOS ANGELES, WEDNESDAY, APRIL 8, TO SUNDAY, APRIL 12, 2026

                        American Educational Research Association (AERA)

Bryer, et al. (2026, April). Free will as an obstacle to randomized controlled trials. Paper presentation at annual meeting of AERA. [Division D - Section 2: Division D - Measurement and Research Methodologies / Division D - Section 2: Statistical Theory and Quantitative Methods]

Abstract:

College students’ responses to feedback from a diagnostic assessment of self-regulated learning

Thursday, April 9th, Los Angeles

Piet Wesling, Angela M. Lui, Joseph Garofalo, Diana Akhmedjanova, Heidi L. Andrade, Jason Bryer

Abstract:This study explores college students’ cognitive, emotional, and behavioral responses to feedback from an online diagnostic assessment of self-regulated learning skills. Using qualitative methods and a think-aloud protocol, we examined how seven students interpreted and reacted to personalized feedback, including descriptive text, dots as scores, and videos on suggested strategies. Findings revealed that students engaged cognitively by making judgments and attributions. Meaning-making frequently involved linking feedback to personal experiences. Emotional responses varied from appreciation to overwhelm. Few students made concrete decisions about next steps. Results underscore the complex interplay of interpretation, emotion, and motivation in feedback processing. Implications are discussed.

                        The National Council on Measurement in Education (NCME)

Bryer, J. (2026, April). Bootstrapping to Determine the Optimal Number of Clusters. Eboard presentation at NCME.

Abstract:

Students‘ Responses to Feedback on a Diagnostic Assessment of Self-Regulated Learning – A Theory of Action

April, Los Angeles

Piet Wesling, Angela M. Lui, Joseph Garofalo, Diana Akhmedjanova, Heidi L. Andrade

Abstract: The purpose of this study is to assess validity evidence for the formative feedback of a diagnostic assessment of self-regulated learning (SRL) skills. Because the diagnostic assessment system serves both measurement and intervention functions, we adopt a theory of action (ToA) approach to evaluate the alignment between key elements of the feedback system, including test components, action mechanisms, and intended and unintended impacts. Focusing specifically on the SRL feedback, we articulate the ToA for feedback use and examine interpretive claims regarding how students engage with, interpret, and respond to the feedback. Data sources include assessment system documentation, prior project publications, an open-ended survey of system developers, and think-aloud protocols and semi-structured retrospective interviews with undergraduate students (n = 7). Students completed the SRL survey and then reviewed personalized feedback while verbalizing their cognitive and affective responses. Using inductive coding, we analyzed students’ engagement with feedback to evaluate the extent to which empirical evidence supports the proposed interpretive claims and to identify important unintended consequences. Findings provide validity evidence based on feedback response processes and illuminate how learners make sense of and act on diagnostic SRL feedback. The study demonstrates the value of ToA as a framework for extending validity research beyond test items and scores to include feedback use and consequences, offering practical implications for both assessment design and educational validity theory.

View the paper and poster here: https://r.bryer.org/shiny/AERA2022/