PubMed Health⌕ Search

Biomedical subjects

D Cleave-Hogg

Publications and source records attributed to D Cleave-Hogg.

12 recordsLinked to original sources
PubMed2004

High-fidelity patient simulation: validation of performance checklists.

BACKGROUND: Standardized scenarios can be used for performance assessments geared to the level of the learner. The purpose of this study was to validate checklists used for the assessments of medical students' performance using high-fidelity patient simulation. METHODS: Our undergraduate committee designed 10 scenarios based on curriculum objectives. Fifteen faculty members with undergraduate educational experience identified items considered appropriate for medical students' performance level and identified items that, if omitted, would negatively affect grades. Items endorsed by less than 20% of faculty were omitted. For remaining items, weighting was calculated according to faculty responses. Students managed at least one scenario during which their performance was videotaped. Two raters independently completed the checklists for three consecutive sessions to determine inter-rater reliability. Validity was determined using Cronbach's alpha with an alpha>or=0.6 and <or=0.9 considered acceptable internal consistency. Item analysis was performed by recalculating Cronbach's alpha with each item deleted to determine if that item contributed to a low internal consistency. RESULTS: 135 students participated in the study. Inter-rater reliability of the two raters determined on the third session was 0.97 and therefore one rater completed the remaining performance assessments. Cronbach's alpha for the 10 scenarios ranged from 0.16 to 0.93 with two scenarios demonstrating acceptable internal consistency with all items. Three scenarios demonstrated acceptable internal consistency with one item deleted. CONCLUSIONS: Five scenarios developed for this study were shown to be valid when using the faculty criteria for expected performance level.

Algorithms↗
PubMed2002

Comparison between medical students' experience, confidence and competence.

OBJECTIVES: This study was undertaken to determine whether or not breadth of clinical experience and student levels of confidence were indicators of competency on standardized simulator performance-based assessments. METHODS: All students (n=144) attending an educational session were asked to complete a 25-point questionnaire regarding specific clinical experiences and levels of confidence in their ability to manage patient problems. For enumeration of clinical experiences, students were asked to estimate the number of times a situation had been encountered or a skill had been performed. For level of confidence, each response was based on a 5-point Likert scale where 1=novice and 5=expert. Students then participated in a standardized simulated performance test. Median and range were calculated and data analysed using Spearman rank correlations. A P-value <0.05 was considered significant. Level of confidence data were compared to performance during clinical rotation and to marks in the anaesthesia final examination. RESULTS: A total of 144 students attended the session, completed the questionnaire and participated in the standardized test. There were wide ranges of experience and confidence in the 25 listed items. Analysis of data showed good correlation between clinical experience and level of confidence. There was no correlation between clinical experience, level of confidence and performance in a standardized simulation test. Neither was there any correlation between level of confidence and clinical grades or written examination marks. CONCLUSIONS: Clinical experience and level of confidence have no predictive value in performance assessments when using standardized anaesthesia simulation scenarios.

Canada↗
PubMed2001

The validity of performance assessments using simulation.

BACKGROUND: The authors wished to determine whether a simulator-based evaluation technique assessing clinical performance could demonstrate construct validity and determine the subjects' perception of realism of the evaluation process. METHODS: Research ethics board approval and informed consent were obtained. Subjects were 33 university-based anesthesiologists, 46 community-based anesthesiologists, 23 final-year anesthesiology residents, and 37 final-year medical students. The simulation involved patient evaluation, induction, and maintenance of anesthesia. Each problem was scored as follows: no response to the problem, score = 0; compensating intervention, score = 1; and corrective treatment, score = 2. Examples of problems included atelectasis, coronary ischemia, and hypothermia. After the simulation, participants rated the realism of their experience on a 10-point visual analog scale (VAS). RESULTS: After testing for internal consistency, a seven-item scenario remained. The mean proportion scoring correct answers (out of 7) for each group was as follows: university-based anesthesiologists = 0.53, community-based anesthesiologists = 0.38, residents = 0.54, and medical students = 0.15. The overall group differences were significant (P < 0.0001). The overall realism VAS score was 7.8. There was no relation between the simulator score and the realism VAS (R = -0.07, P = 0.41). CONCLUSIONS: The simulation-based evaluation method was able to discriminate between practice categories, demonstrating construct validity. Subjects rated the realism of the test scenario highly, suggesting that familiarity or comfort with the simulation environment had little or no effect on performance.

Aged↗
PubMed2001

A comparison of global ratings and checklist scores from an undergraduate assessment using an anesthesia simulator.

PURPOSE: To determine the correlation between global ratings and criterion-based checklist scores, and inter-rater reliability of global ratings and criterion-based checklist scores, in a performance assessment using an anesthesia simulator. METHOD: All final-year medical students at the University of Toronto were invited to work through a 15-minute faculty-facilitated scenario using an anesthesia simulator. Students' performances were videotaped and analyzed by two faculty using a 25-point criterion-based checklist and a five-point global rating of competency (1 = clear failure, 5 = superior performance). Correlations between global ratings and checklist scores, as well as specific performance competencies (knowledge, technical skills, and judgment), were determined. Checklist and global scores were converted to percentages; means of the two marks were compared. Mean reliability of a single rater for both checklist and global ratings was determined. RESULTS: The correlation between checklist and global ratings was.74. Mean ratings of both checklist and global scores were low (58.67, SD = 14.96, and 57.08, SD = 24.27, respectively); these differences were not statistically significant. For a single rater, the mean reliability score across rater pairs for checklist scores was.77 (range.58-.93). Mean reliability score across rater pairs for global ratings was.62 (.40-.77). Global ratings correlated more highly with technical skills and judgment (r =.51 and r =.53, respectively) than with knowledge. (r =.24) CONCLUSION: Inter-rater reliability was higher for checklist scores than for global ratings; however, global ratings demonstrated acceptable inter-rater reliability and may be useful for competency assessment in performance assessments using simulators.

Anesthesia↗
PubMed2000

Evaluation of medical students' performance using the anaesthesia simulator.

OBJECTIVES: A pilot project assessing clinical performance was undertaken using the Anaesthesia Simulation Centre at the University of Toronto. The purpose of this study was to determine the reliability of assessments of medical students' performance using the simulator as an evaluation tool, to compare these assessments to written and clinical evaluations and to elicit student opinion. Simulator assessments were performed at the completion of the anaesthesia rotation. DESIGN: Twenty-four of 177 University of Toronto medical students participated in a videotaped simulator session with an attending faculty. These 24 students were based at Sunnybrook Health Science Centre. During the session, each student worked through one of six predetermined cases involving discrete patient problems based on the list of core objectives. Five evaluators independently assessed each student's videotaped performance using standardized performance evaluation criteria and data were examined for inter-rater reliability. Clinical and written examination marks were compared to simulator assessments. A student questionnaire was administered and descriptive data obtained. SETTING: The University of Toronto. SUBJECTS: Medical students. RESULTS: The intraclass correlation coefficient (ICC) of inter-rater reliability was 0.87. There was poor correlation between the simulator and written examination marks (r=0.19, P=0.38) and between the simulator and clinical marks (r=0.04, P=0.87). The simulator experience was highly rated by students: learning experience 4.6 +/- 0.51, appropriate content 4.4 +/- 0.74, use as evaluation tool, 4.1 +/- 0.92 (1=poor, 5=excellent). CONCLUSIONS: Our pilot data suggest that the simulator is a reliable assessment method for medical students' performance. Further work may justify the inclusion of the simulator as an evaluation and education tool and expanded to incorporate learning objectives of other medical disciplines.

Anesthesiology↗
PubMed2000

A Canadian simulation experience: faculty and student opinions of a performance evaluation study.

One hundred and forty-three students and 18 faculty at the University of Toronto participated in a study of the anaesthesia simulator as an evaluation tool. Both student and faculty opinions regarding the experience were elicited using questionnaires with a five-point scale, 1 = strongly disagree, 5 = strongly agree. Faculty and student opinion were similar and positive with respect to the use of the simulator and matching of educational objectives, its use as a learning experience, its use as an evaluation tool and the need for familiarity with the tool before use as an assessment method. This study supports the use of the simulator as an evaluation tool based on faculty and student opinions provided that prior exposure to the environment is offered.

Anesthesiology↗
PubMed1997

Characteristics of good anaesthesia teachers.

PURPOSE: The Department of Anaesthesia undertook a qualitative study to a) reveal the characteristics of teachers who had been identified as "good," and b) explore the levels of epistemological development (defined as conceptualization of knowledge) that are evidenced. Changes in medical education curricula have focused attention on the ways in which medical teaching staff conceptualize the learning/teaching interactions and their ability to alter or modify their teaching styles. Teachers are often assessed or informally recognized as "good teachers," but there are few indicators to guide what is meant by the label in anaesthesia. METHODS: Teachers who had consistently received overall ratings of 4+ on a 5 point rating scale over a five year period were selected to be interviewed. Data were analyzed a) noting key teaching characteristics and patterns of teaching and b) within the framework of adult development theories. RESULTS: Good teachers in Anaesthesia all identified six characteristics necessary for good teaching. They were characterised by their "inquiry" approach to teaching, their complexity of thought and their functioning at higher relativistic/Commitment levels of epistemological development. CONCLUSION: Teaching in anaesthesia is depicted by the need to address multiple aspects of thinking and action. Good teachers are aware of these aspects and include techniques that offer residents opportunities to develop their thinking skills to deal with medical complexities as well as guiding learners to increase their knowledge. The interviewed teachers' revealed approaches to teaching and learning that indicated their own personal cognitive complexity and levels of development.

Adult↗
PubMed1991

Achievement of objectives: internal medicine fourth year clinical clerkship.

The clinical clerkship in Canadian Medical Schools is intended to provide senior medical students with opportunities to gain practical knowledge of clinical medicine, develop technical skills, learn to use judgement and experience first-hand clinical decision making. Assessment of rotations in internal medicine were undertaken in order to understand more fully the nature of medical clerkship experiences. We found that medical clerks in our programme were exposed to a high proportion of undifferentiated problems and an adequate case mix. They performed a wide range of technical procedures. There were, however, certain gaps in the clerks' experiences both in the specialty areas, and in therapeutic and technical procedures. Recognition of these gaps in the clerkship experiences suggest that further deliberation of goals and objectives is required and issues of achievement of competencies need to be addressed.

Adult↗
PubMed1984

Changes in undergraduate attitudes toward medical ethics.

To detect any change in medical students' attitudes toward medical ethics, students from the same class were given a questionnaire on their first day of medical school and again near the end of their fourth year of study. The results showed a strong shift away from the students' initial expectations that they would rely on specialists or scholarly sources in the future; the need for a medical ethics course in the curriculum, while still felt, was less important to them by the fourth year. The reasons for these changes were not apparent, for the students' levels of knowledge and perceptions of the role of ethics in medicine in the first and fourth years did not differ. It is recommended that medical school faculty actively reinforce the initially positive attitudes of students during clinical supervision.

Attitude of Health Personnel↗