Use of AI for critical thinking artifact assessment

Abstract

The project pioneers an innovative approach to evaluating High-Impact Practices (HIPs) by leveraging advanced large language models (LLMs) to automate the grading of student critical thinking reflections. Building on research demonstrating LLMs’ capacity to achieve human-level accuracy in grading complex, multi-step explanations, this initiative addresses scalability challenges as the college expands its Experiential Learning and Critical Thinking (EXACT) Plan by embedding experiential learning and critical thinking into 70+ courses by 2029. The project will develop an AI algorithm trained to assess student reflections using validated rubrics for integrative learning and critical thinking, mirroring the interrater reliability of faculty committees.