Evaluating and improving your classes.

Specific methods of mechanical ventilation management reduce mortality and lower health care costs. However, in the face of a predicted deficit of intensivists, it is unclear whether residency programs are training internists to provide effective care for patients who require mechanical ventilation. To evaluate these educational outcomes, we administered a validated 19-item case-based test and survey to resident physicians at 31 diverse U.S. internal medicine residency programs nationwide. Of 347 senior residents, 259 (75%) responded. The mean test score was 74% correct (SD, 14%; range, 37 to 100%). Important items representing evidence-based standards of critical care answered incorrectly were as follows: use of appropriate tidal volume in the acute respiratory distress syndrome (48% incorrect), identifying a patient ready for a weaning trial (38% incorrect), and recognizing indication for noninvasive ventilation (27% incorrect). Most accurately identified pneumothorax (86% correct) and increased intrathoracic positive end-expiratory pressure (93% correct). Better scores were associated with "closed" versus "open" intensive care unit organization (76 versus 71% correct, p = 0.001), resident perception of greater versus lesser ventilator knowledge (79 versus 71% correct, p = 0.001), and graduation from a U.S. versus international medical school (75 versus 69% correct, p = 0.033). Although overall training satisfaction correlated strongly with program use of learning objectives (r = 0.89, p < 0.0001), only 46% reported being satisfied with their mechanical ventilation training. We conclude that senior residents may not be gaining essential evidence-based knowledge needed to provide effective care for patients who require mechanical ventilation. Residency programs should emphasize evidence-based learning objectives to guide mechanical ventilation instruction.

Educational Measurement↗

PubMed2002

The In-Training Examination in Internal Medicine: an analysis of resident performance over time.

BACKGROUND: The In-Training Examination in Internal Medicine (IM-ITE) has been offered annually to all trainees in U.S. medical residency programs since 1988. Its purpose is to provide residents and program directors with an objective assessment of each resident's personal performance on a written, multiple-choice examination and the performance of the residency program compared with that of its peers. OBJECTIVE: To analyze trends in the demographic characteristics and scores of examinees during the first 12 years of administration of this examination. DESIGN: Descriptive analysis over time. SETTING: U.S. residency programs in internal medicine, 1988-2000. PARTICIPANTS: Residents at all levels of training in categorical, primary care, and medicine-pediatrics programs in the United States and Canada. The number of examinees increased from 7500 in 1988 to almost 18 000 in 2000. MEASUREMENTS: After calibration of the scores for each examination, test results were compared and analyzed for selected cohorts of residents over 12 years. RESULTS: More than 80% of residents in medicine training programs participate in the IM-ITE, most on an annual basis throughout their period of training. Test performance improves at a predictable rate with each year of training. Since 1995, international medical school graduates have persistently outperformed graduates of U.S. medical schools. Test results were affected by the timing of the examination, the time that was available to complete the examination, and the actual time that residents spent in internal medicine training before each examination. CONCLUSIONS: The IM-ITE scores generally improve with year of training time spent in internal medicine training before the examination and time permitted to complete the examination. These observations provide evidence that the IM-ITE is a valid measure of knowledge acquired during internal medicine training.

Educational Measurement↗