Examining the Efficacy, Predictive Power, and Cost Effectiveness of the Diagnostic Assessment and Achievement of College Skills

The CUNY School of Professional Studies (CUNY SPS) has been awarded a highly competitive and prestigious $3,789,074 million grant from the U.S. Department of Education’s Institute of Education Sciences (IES) to study the impact of online assessment tools on college students’ success.

CUNY SPS will lead a partnership with SUNY Albany, Rutgers University, SUNY Empire State College, and University of Maryland Global Campus (UMGC) to examine the efficacy, predictive power, and cost effectiveness of the Diagnostic Assessment and Achievement of College Skills (DAACS), a suite of technological and social supports designed to optimize student learning, on newly enrolled college students.

“This generous grant illustrates the innovative pedagogical approaches that the City University of New York is exploring in order to find new ways of supporting student success as the world of higher education adjusts to our post-pandemic reality,” said Chancellor Félix V. Matos Rodríguez. “While COVID is receding, many social, economic, and educational inequalities continue to exist. By making use of these online assessment tools, CUNY, and our public university partners, are pioneering new pathways to empower the very students who need it most. We thank the U.S. Department of Education, and our collaborators, for their support of this important project.”

“The Institute of Education Sciences grant is a highly prestigious award, and securing such funding is an extremely competitive process. We are delighted that this ambitious pedagogical project, spearheaded by faculty and staff at CUNY SPS, has been honored with this significant support from the U.S. Department of Education,” said CUNY SPS Dean John Mogulescu. “With this award, the project team will be able to determine the effectiveness of these innovative online assessment tools so that they may ensure the greatest possible outcomes of academic success for students entering college for the first time.”

The proposal, titled Examining the Efficacy, Predictive Power, and Cost Effectiveness of the Diagnostic Assessment and Achievement of College Skills, was submitted by Dr. Jason Bryer, assistant professor and associate director of data science and information systems at CUNY SPS, in collaboration with co-PIs Dr. Heidi Andrade at the University at Albany and Dr. Timothy Cleary at Rutgers University, as well as Lawrence MacLean, grants officer at CUNY SPS. Dr. Bryer will also lead this initiative, along with a project-funded research associate and project manager.

“We are very excited that IES is giving us the opportunity to test the efficacy of DAACS. With institutions reducing or eliminating their use of admissions and placement exams, DAACS represents a no-stakes alternative to these high-stakes assessments,” said Dr. Bryer. “Results of our initial study showed that students who utilized the feedback and resources were more likely to be successful in their first term of college. DAACS also provides institutions with valuable information about students’ strengths and weaknesses, along with tailored interventions and supports.”

Funded by the Department of Education in the Fund for the Improvement of Postsecondary Education First in the World grant program, DAACS is a suite of free, open source, online diagnostic assessments of key academic and self-regulated learning skills, accompanied by a collection of open education resources that are designed to help students successfully transition to college.

Students who use DAACs start by taking four free, online assessments of college readiness, including mathematics, reading, writing, and self-regulated learning. They then receive immediate personalized feedback, along with suggestions and links to websites they can use to boost their knowledge and skills without having to spend time and money on remedial coursework. Their DAAC results and data are also passed on to their academic advisors and institutional leaders to help provide individualized support and to identify those who might be at risk of academic failure.

As outlined in the proposal, the project will specifically compare the success of students who use DAACS to students who do not, as measured by a randomized controlled trial. Overall, the proposal predicts that students who use DAACS will be more successful in terms of staying in college, earning credits, and having higher GPAs. The project will also explore whether these effects may be standardized across different types of postsecondary institutions.

The very generous IES grant of $3,789,074 will cover 100% of the costs to fund the project, which will involve conducting a randomized controlled trial of approximately 36,000 newly enrolled undergraduates at three postsecondary institutions.

About the Institute of Education Sciences

The Institute of Education Sciences (IES) is the statistics, research, and evaluation arm of the U.S. Department of Education. The organization is independent and nonpartisan. Their mission is to provide scientific evidence on which to ground education practice and policy and to share this information in formats that are useful and accessible to educators, parents, policymakers, researchers, and the public. For more information, visit the IES website.