Response Latencies and IRT Person Estimates Using CASI

Computer assisted self-interviewing (CASI) has been used around the world for a long time in order to achieve survey data collection with a reduced cost. CASI can also be useful when the intention is to reach groups of the population who are geographically dispersed, or who cannot be available to be interviewed during normal work hours. However, it has often been argued that CASI may potentially yield low-quality data if the respondents are not motivated to respond honestly and reliably, using their full potential and concentration.

This study proposes a new Item Response Theory (IRT) algorithm for the estimation of the person estimates when an attitude scale or an ability test is administered through a CASI. The algorithm weights less the responses that are given too soon after the stimuli are presented, so they may not be considered as legitimate and honest efforts for a truthful  response. The new method is based on the work of Wise and Kong (2005) and Mislevy and Bock (1982) and used a modified biweight IRT estimation method .

The application of the algorithm on empirical data (N=920) in the context of the mathematics National Curriculum in England gave indications of increased validity and usefulness of test results.

The possible practial consequences of the use of this algorithm for survey research are discussed.

Wise, S. L., & Kong, X. (2005). Response time effort: A new measure of examinee motivation in computer-based tests. Applied Measurement in Education, 18, 163-183.

Mislevy, R. J., & Bock, R. D. (1982). Biweight estimates of latent ability. Educational and Psychological Measurement, 42, 725-737.