On measuring repeatability of data from self-administered questionnaires.

Abstract

It is usually necessary to estimate repeatability of a self-administered questionnaire from responses by subjects on just two occasions. It is shown that it is not possible to estimate false positive and false negative classification rates separately, but that an average correct classification can be calculated. This is recommended as the appropriate absolute measure of repeatability, whereas the kappa statistic, which can be interpreted as the ratio of between subject to total variation, is a measure of repeatability relative to prevalence of responses. Both measures should be calculated to give complete information on repeatability, analogous to absolute and relative measures used for continuous variables.