Artificial Intelligence Assessment

AI Assessment Overview

In AI assessment, a software system infers problem-specific rules for automated scoring from examples of instructor grading of student assignments. 

Initially, AI techniques are applied to learn how an instructor grades a problem.  The instructor evaluates a sample set of student responses, and the system creates a computer model incorporating rules it inferred about the instructor’s grading decisions.  The model is then used to grade other students’ work.

Such techniques are applied to student work in a number of disciplines, for example, to mathematical problem-solving, to programming in computer science, and increasingly to essays.  

The strengths of AI assessment are efficiency, consistency in applying the same criteria across students, and immediate and detailed feedback on performance.  

Today, AI assessment is most useful as one part of an assessment process and for enhancing learning, rather than making final, authoritative, high-stakes decisions about student performance. Key considerations are sufficient transparency of the rules applied, human scoring establishing the validity of machine-generated scores, and ongoing quality control.

AI Assessment References

Williamson, D.M., et al.  (July 2010) Automated Scoring for Assessment of Common Core Standards, Educational Testing Service.

Zhang, M.  (March 2013) Contrasting Automated and Human Scoring of Essays.  R&D Connections, n21.  Educational Testing Service

Stanford Lytics Lab