Validating standard setting with a modified nedelsky procedure through common item test equating.

Abstract

It is often impossible to validate cut scores set using judged item review methods due to the fact that many high stakes testing programs attempt to limit the number of common items across consecutively administered forms. However, over time, with a stable item pool, secondary links through other test administrations allow the use of common item equating to test the stability of the judged cut scores. In this study five forms of a basic science examination administered over a three year period in a national board testing program were analyzed to determine the stability of judged cut scores. The stability was determined by comparison of the judged cut scores with the equated cut scores derived by the Rasch common item equating technique. The results indicate cut scores derived from the modified Nedelsky procedure were within equating error of the Rasch equated cut scores over five administrations.