Using just noticeable differences to interpret test scores.

Abstract

This study explored the value of obtaining a just noticeable difference (JND) for a test--the difference in scores needed before observers detect a difference in examinees' behavior--as a means of interpreting the practical meaning of scores. Classical psychophysical methods were adapted and applied to the scores of foreign teaching assistants (TAs) on an achievement test, the Test of Spoken English (TSE), and the ratings for English proficiency that the TAs received from their students. The JND for the TSE scores was substantial, as large as the standard deviation of the scores and much larger than the standard error of measurement and guidelines for the d index of effect size for mean differences, suggesting that both sets of standards may highlight score differences that are not practically significant. This study demonstrates the applicability of JNDs for evaluating scores on educational and psychologists' tests.