Incomplete data sets: coping with inadequate databases.

Abstract

Three problems arise in handling numerical values in databases: bad data, missing data, and sloppy data. The effects of bad data are mitigated by using statistical subterfuges such as robust statistics or outlier removal. Missing data are replaced by creating a substitute through interpolation or by using statistics appropriate to unbalanced designs. Sloppy, semiquantitative data are relegated to innocuous positions by using nonparametric, rank, or attribute statistics. These techniques are illustrated by the telephone directory, a database of carcinogenicity test results, and a database of precision parameters derived from method performance (collaborative) studies.