PubMed Health⌕ Search
Home/Archive/PubMed

PubMed · 16199446

Using factor analysis to evaluate checklist items.

Danette W McKinley·John R Boulet
View original source ↗Export metadata ↓

Abstract

BACKGROUND: Relatively little research has been published examining the use of factor analytic techniques to improve the psychometric qualities of performance assessments. The purpose of the current investigation is to illustrate the potential use of structural equation modeling (SEM) with scores from a standardized patient examination. METHOD: Checklist items for a single case were examined to compare two models within a SEM framework: a single, underlying "data gathering" construct and a two-factor model consisting of medical history interviewing and physical examination maneuvers. RESULTS: The chi2 statistics obtained were statistically significant for both the single factor (chi2=1645.8, p<.001) and the two-factor model, (chi2=1373.1; p<.001), indicating marginal model-data fit. A difference statistic was calculated (chi2=320.9; p<.001), suggesting that the two-factor model has better fit. CONCLUSIONS: The parameters estimated by each model could be used to evaluate checklist item performance. The results of this investigation illustrate the utility of this method in the analysis of checklist item data.

Explore related subjects

Clinical CompetenceEducational MeasurementFactor Analysis, StatisticalHumansMedical History TakingPhysical ExaminationPsychometricsSurveys and QuestionnairesUnited StatesJournal Article

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Danette W McKinley, John R Boulet. 2005. Using factor analysis to evaluate checklist items.. https://doi.org/10.1097/00001888-200510001-00027

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related citations

PubMed2007

A survey of the education of nurses working in mental health settings in 12 European countries.

BACKGROUND: Recent European policies emphasise that the training and education of health professionals are a major means of improving mental health services. Little comparative research has been undertaken to ascertain the degree of homogeneity in the training of nurses working in mental health settings across Europe. OBJECTIVE: The aim of this study was to investigate the training of nurses working in mental health settings in a range of European countries. Specifically, information regarding criteria for entry into training, type of qualification awarded and further training opportunities was to be explored. METHODS AND PARTICIPANTS: A specially designed questionnaire was designed to gather data from senior mental health nurses working in the mental health field in a sample of 12 countries from across Europe. RESULTS: Findings disclosed considerable disparity between countries in respect to nurse training, with few countries requiring a specialist nursing qualification to practice in the mental health field. There appears to be little homogeneity in terms of educational provision pre and post-qualification. CONCLUSIONS: More research is needed into how European states deliver nurse training and education, how students are prepared for various practice settings and the effect on outcomes for service users of different models of training.

Clinical Competence↗
PubMed2006

Is an Angoff standard an indication of minimal competence of examinees or of judges?

BACKGROUND: To establish credible, defensible and acceptable passing scores for written tests is a challenge for health profession educators. Angoff procedures are often used to establish pass/fail decisions for written and performance tests. In an Angoff procedure judges' expertise and professional skills are assumed to influence their ratings of the items during standard-setting. The purpose of this study was to investigate the impact of judges' item-related knowledge on their judgement of the difficulty of items, and second, to determine the stability of differences between judges. METHOD: Thirteen judges were presented with two sets of 60 items on different occasions. They were asked to not only judge the difficulty of the items but also to answer them, without the benefit of the answer key. For each of the 120 items an Angoff estimate and an item score were obtained. The relationship between the Angoff estimate and the item score was examined by applying a regression analysis to the 60 items (Angoff estimate, score) for each judge at each occasion. RESULTS AND CONCLUSIONS: This study shows that in standard-setting the individual judgement of the individual item is not only a reflection of the difficulty of the item but also of the inherent stringency of the judge and his/her subject-related knowledge. Considerable variation between judges in their stringency was found, and Angoff estimates were significantly affected by a judge knowing or not knowing the answer to the item. These findings stress the importance of a careful selection process of the Angoff judges when making pass/fail decisions in health professions education. They imply that judges should be selected who are not only capable of conceptualising the 'minimally competent student', but who would also be capable of answering all the items.

Clinical Competence↗
PubMed2006

Hospital visiting in obstetrics and gynecology: a tool for the advancement of training.

OBJECTIVE: To assess the improvement of obstetric and gynecologic training brought about by peer influence in Europe. METHODS: In 1996, the European Board and College of Obstetrics and Gynecology (EBCOG) initiated a visiting process by international and local peers to improve training and decrease differences in health care standards. RESULTS: A large number of visits of obstetrics and gynecology departments have been conducted across Europe at teaching hospitals by the Hospital Visiting Committee. Compliance with the structured approach of the visiting policy and problems met during these visits are reported. CONCLUSION: The program focuses on the continuous improvement of the competencies of all persons trained in the obstetrics and gynecology departments of teaching hospitals throughout Europe. It also increases the understanding of diversity in training methods and can gradually lead to the convergence of training and health care standards in Europe.

Clinical Competence↗