Future Review Panel: Approaches to Studying Student Success in Higher Education Thursday, April 24, 11:25am to 12:10pm, Chicago This panel will highlight the work that is published in Future Review, our publication. Two of the first authors to publish in this journal will discuss their work on student success. The panel features both quantitative and qualitative research, and highlights the different ways of conceptualizing student success in higher education. Panelists will also discuss how their research can be used by practitioners in higher education, and why they chose Future Review as a venue for their research.
Efficacy of the Diagnostic Assessment and Achievement of College Students on Multiple Success Indicators Authors: Jason Bryer; Angela M. Lui; Heidi L. Andrade; David W Franklin; Timothy J. Cleary Abstract The purpose of this study is to examine the effects and predictive power of the Diagnostic Assessment and Achievement of College Skills (DAACS) on student success. DAACS is a no-stakes, open-source, formative assessment tool designed to measure newly enrolled college students’ reading, writing, mathematics, and self-regulated learning (SRL) skills, and provide them with feedback and resources to enhance their functioning and success.
Relationship Between Intraclass Correlation and Percent Rater Agreement Authors: Jason Bryer; Guher Gorgun Abstract Inter-rater reliability (IRR) is a critical component of establishing the reliability of measures when more than one rater is necessary. There are numerous IRR statistics available to researchers including percent rater agreement, Cohen’s Kappa, and several types of intraclass correlations (ICC). Several methodologists suggest using ICC over percent rater agreement (Hallgren, 2012; Koo & Li, 2016; McGraw & Wong, 1996; Shrout & Fleiss, 1979).
Validity Evidence of the Internal Structure of the DAACS Self-Regulated Learning Survey Authors: Angela M. Lui, David Franklin Jr., Diana Akhmedjanova, Guher Gorgun, Jason Bryer, Heidi L. Andrade, and Timothy Cleary Abstract The Diagnostic Assessment and Achievement of College Skills (DAACS) online system assesses newly enrolled college students’ skills in reading, writing, mathematics, and self-regulated learning, and provides individualized feedback and links to resources. The purpose of this study is to examine validity evidence regarding the internal structure of the DAACS Self-Regulated Learning (SRL) self-report survey.
Improving Interventions: Development of the Implementer Perception of Intervention Survey Authors: Jacqueline Slemp, Devora Panish, Erica Pawlo, Timothy Cleary Abstract The Implementer Perception of Intervention Survey (IPIS) was developed to evaluate academic support personnel’s perceptions of the usability, feasibility, and importance of an academic intervention for developmental, entry-level college students. Principal axis factoring of the IPIS yielded a two-factor structure. The factors were labeled perceived acceptability and perceived usability. Practitioners can adapt the IPIS for use in a variety of academic interventions and educational contexts.
Developing and Validating the DAACS Writing Assessment Authors: Heidi Andrade, Jason Bryer, and Robert Yagelski Abstract This paper describes an ongoing project in which on-demand writing is used as part of a diagnostic evaluation of newly admitted college students. The authors will present preliminary evidence of the validity and reliability of human and machine scored essays and the usefulness of the feedback provided to students on their writing and their self-regulated learning.
Distance Teaching and Learning Conference Although the issues of placement exams impact most less selective or open enrollment colleges, the efficacy of DAACS as an alternative to traditional placement exams was conducted at two large online distance education colleges: Excelsior College and Western Governors University. DAACS is an assessment framework that provides students with a formative view of their skills in self-regulation, reading, writing, and math. For administrators, DAACS provides valuable information to inform personalized interventions and more accurate risk modeling.
Rethinking Placement Exams: Diagnostic Assessment and Achievement of College Skills Abstract This session will introduce a new open source formative assessment developed as an alternative to tradition placement exams that measures self-regulated learning, reading, mathematics, and writing. Results from a pilot study conducted at two large online institutions will be discussed. The Diagnostic Assessment and Achievement of College Skills (DAACS) addresses the issues with traditional placement exams and remediation programs by providing students with immediate, detailed, formative feedback.
Development Of The Large Scale Diagnostic Assessments Of College Skills Authors: Jason Bryer, Excelsior College; Fusun Sahin, American Institutes for Research; Heidi Andrade, Angela Lui, Diana Akhmedjanova, & David Franklin, University at Albany, SUNY The Diagnostic Assessment and Achievement of College Skills (DAACS) is a technology-based, no-stakes diagnostic assessment system on academic and non-academic skills, with fast, actionable feedback. Development and piloting of these assessments at two online colleges is underway.