PubMed Health⌕ Search

Biomedical subjects

C P van der Vleuten

Publications and source records attributed to C P van der Vleuten.

At least 19 recordsLinked to original sources
PubMed2006

Implementing an Objective Structured Clinical Examination (OSCE) in dental education: effects on students' learning strategies.

OBJECTIVES: To investigate the effect of an Objective Structured Clinical Examination (OSCE) on dental students' learning strategies and competence to manage periodontal diseases in patients. The implemented OSCE was expected to be superior to the existing Written Exam in fostering the acquisition of clinical competencies in terms of study strategies that are more oriented towards clinical practice, longer study time, greater clinical proficiency, and more realistic self-assessment. MATERIAL AND METHODS: After a clinical course in periodontology, 72 third year dental students were assessed summatively, either using a Written Exam or an OSCE (P-OSCE). The students were informed beforehand about the assessment formats. The self-assessed clinical competence, study time and strategies (i.e. practice with a manikin, peers and patient case) were evaluated by means of a questionnaire. After a comprehensive dental care course, all 72 students were assessed by an overall end-of-year OSCE, in which three periodontal stations were included 'measuring pockets', 'educating patients' and 'tracing an X-ray with bone-loss'. The competence of the previous Written Exam group and the P-OSCE group was investigated by determining the mean scores and pass-fail scores of three periodontal test-stations as well as the total score of the end-of-year OSCE. The degree of realistic self-assessment was studied by correlating the self-assessed competencies as evaluated by means of the questionnaire with the total score of the end-of-year OSCE. RESULTS: Self-assessed clinical competence, study time and study strategies showed no differences between the P-OSCE and the Written Exam-group. The clinical competence determined in the test-station 'measuring pockets' in the end of year overall OSCE was higher for the P-OSCE group (P = 0.05) when compared with the Written Exam group; the two groups performed equally well in the test station 'educating patients', whereas the performance in 'tracing an X-ray with bone-loss' was better in the Written Exam group. This group also had a higher total score in the end-of-year OSCE (P = 0.05). The degree of realistic self-assessment was higher in the P-OSCE group than in the Written Exam group: in the P-OSCE group the self-assessed clinical competencies correlated significantly with the total score of the overall end-of-year OSCE (P < or = 0.05). CONCLUSIONS: No effects of the implementation of an OSCE in undergraduate periodontal education were observed in study strategies, but the implementation of an OSCE in undergraduate periodontal education appears to stimulate learning, resulting in greater achievement of specific clinical competence and a greater level of realistic self-assessment.

Adult↗
PubMed2001

Do short cases elicit different thinking processes than factual knowledge questions do?

PURPOSE: To assess whether case-based questions elicit different thinking processes from factual knowledge-based questions. METHOD: 20 general practitioners (GPs) and 20 students solved case-based questions and matched factual knowledge-based questions while thinking aloud. Verbatim protocols were analysed. Five indicators were defined: extent of protocols; immediate responses; re-reading of information given in the stem or case after the question had been read; order of re-reading information, and type of consideration, i.e. 'true-false' type or 'vector', that is, a deliberation which has a magnitude and a direction. RESULTS: Cases elicited longer protocols than factual knowledge questions. Students re-read more given information than GPs. GPs gave an immediate response on twice as many occasions as students. GPs re-ordered the case information, whereas students re-read the information in the order it was presented. This ordering difference was not found in the factual knowledge questions. Factual knowledge questions mainly led to 'true-false' considerations, whereas cases elicited mainly 'vector' considerations. CONCLUSION: Short case-based questions lead to thinking processes which represent problem-solving ability better than those elicited by factual knowledge questions.

Education, Medical↗
PubMed2001

Solving problems with group work in problem-based learning: hold on to the philosophy.

INTRODUCTION: Problem-based learning (PBL) has gained a foothold within many schools in higher education as a response to the problems faced within traditional education. DISCUSSION: Working with PBL tutorial groups is assumed to have positive effects on student learning. Several studies provide empirical evidence that PBL stimulates cognitive effects and leads to restructuring of knowledge and enhanced intrinsic interest in the subject matter. However, staff members do not always experience the positive effects of group work which they had hoped for. When confronted with problems in group work, such as students who only maintain an appearance of being actively involved and students who let others do the work, teachers all too often implement solutions which can be characterized as teacher- directed rather than student-directed. Teachers tend to choose solutions which are familiar from their own experience during professional training, i.e. using the teacher-directed model. These solutions are not effective in improving group work and the negative experiences persist. CONCLUSION: It is argued that teachers should hold on to the underlying educational philosophy when solving problems arising from group work in PBL, by choosing actions which are consistent with the student-directed view of education in PBL.

Cognition↗
PubMed2001

Students' perceptions of time spent during clinical rotations.

Some rotations during clinical education are characterized by a high number of hours spent per week in the hospital because students panicipate in a hospital on-call system, i.e. hours beyond usual working hours, e.g. at night. However, students complain about spending too many hours in the hospital on non-instructive activities. This study was undertaken to investigate differences among rotations in time spent in hospital, in on-call hours, in self-study and in non-instructive activities and to investigate the relationship between time spent on the various activities and the overall effectiveness as perceived by students. A questionnaire was administered to students at the end of various clinical rotations. Rotations differ considerably in time spent in hospital, on call (i.e. beyond usual working hours), on self-study and on non-instructive activities. In some rotations students report spending on average 18-20% of their time in the hospital on non-instructive activities. Furthermore, the numbers of hours spent in hospital do not correlate with the overall effectiveness as perceived by students, unless the numbers of hours spent in non-instructive activities are taken into account. The effectiveness of a rotation does not automatically improve if students spend more hours in hospital. The number of hours spent on non-instructive activities should be reduced in some rotations and should be kept as low as possible. Further research is needed to find out which activities are perceived as non-instructive.

Journal Article↗
PubMed2001

Assessment in problem-based learning (PBL).

Problem-based learning (PBL), a relatively new approach to teaching, has been introduced in several of the medical schools in the Southeast Pacific region. This approach emphasises active, self-directed learning and puts the emphasis on the individual student and his/her role in the educational process. It is well known that assessment plays a large role in influencing student learning behaviour. Therefore, it is important that the assessment process not hamper learning or adversely affect attainment of the goals of the curriculum. If student behaviours are directed toward achieving success on the assessments, instructors' efforts to create a climate of self-directed learning and individual responsibility will be frustrated. This article provides information on the different classes of assessment vehicles available to assess the various domains of medical competence. We pay special attention to the critical relationship between the educational goals of problem-based learning and the different assessment instruments used. We briefly describe some assessment instruments and how it can relate to these goals in a positive manner. We conclude that no assessment instrument is ideal and that each has unique inherent deficiencies. As a consequence, a successful student assessment process requires use of several complementary instruments.

Clinical Competence↗
PubMed2000

The effect on reliability of adding a separate written assessment component to an objective structured clinical examination.

PURPOSE: To determine the effect on test reliability when a separate written assessment component is added to an objective structured clinical examination (OSCE). METHOD: Volunteers (n=38) from Maastricht Medical School were recruited to take a skills-related knowledge test in addition to their regular end-of-year OSCE. The OSCE scores of these volunteers did not differ from those of the other students of their class. Multivariate generalizability theory was used to investigate the combined reliability of the two test formats as well as their respective contributions to overall reliability. RESULTS: Combining the two formats has an added value. The loss of reliability due to the use of fewer stations in the OSCE can be fully compensated by lengthening the written test component. CONCLUSION: From the perspective of test reliability, it is possible to economize on the resources needed for performance-based assessment by adding a separate written test component.

Adult↗
PubMed2000

Quality assessment in general practice trainers.

INTRODUCTION: General practice trainers hold a key position in general practice training, especially through their provision of a role model. Their own competence in general practice care is important in this regard. The purpose of the study was to evaluate whether a quality assessment programme could identify the strengths and weaknesses of GP trainers in four main domains of general practice care. METHODS: The quality assessment programme comprised validated tests on four domains of general practice: general medical knowledge, knowledge of medical-technical skills, consultation skills and practice management. The criterion for the identification of relative strengths and weaknesses of GP trainers was a variation in the scores of trainers indicating higher and lower scores (strengths and weaknesses) within each domain. RESULTS: GP trainers (n=105) were invited to participate in the study and 90% (n=94) did so. The variation in scores allowed the indication of strengths and weaknesses. Main strengths were: general medical knowledge of the digestive system; knowledge of medical skills relating to the skin; consultation skills concerning empathy; practice management with regard to accessibility. Main weaknesses were: general medical knowledge of the neurological system; knowledge of the medical/technical skills relating to the endocrine metabolic and nutritional system; consultation skills regarding shared decision making; practice management involving cooperation with staff and other care providers. DISCUSSION: This first systematic evaluation of GP trainers identified their strengths and weaknesses. The weaknesses identified will be used in the improvement process as topics for collective improvement in the GP trainers' general curriculum and in individual learning plans.

Educational Measurement↗
PubMed2000

Reliability of the amsterdam clinical challenge scale (ACCS): a new instrument to assess the level of difficulty of patient cases in medical education.

INTRODUCTION: In problem-based medical curricula, consideration should be given to the level of difficulty of patient cases used for training and assessment. The Amsterdam Clinical Challenge Scale (ACCS) has been developed to assess the degree of difficulty of patient cases in a systematic and reproducible manner. To determine the reliability of the instrument two research questions were addressed: (1) How many judges are required, on the basis of the total score of the ACCS, to obtain a reliable estimate of the difficulty of a single case? (2) How many cases and/or how many judges are needed to reach an acceptable level of reliability of the total score of the ACCS? METHOD: Four judges scored 36 patient scripts reflecting a wide range of patient problems encountered in general practice. Each script was scored four times. In the reliability analysis, the generalizability theory was applied. RESULTS: The results show that the judges did, indeed, use the whole range of difficulty ratings. When the ACCS is applied to a single case, eight or more judges are needed to reach an acceptable level of reliability. When more cases are involved, fewer judges are needed; for 10 or more cases one judge will be sufficient. CONCLUSIONS: Given the typical length, for example of an objective structured clinical examination, the ACCS makes it possible to provide a reliable estimate of the level of difficulty of such a test with only a limited number of judges.

Clinical Competence↗
PubMed2000

A comparison of standard-setting procedures for an OSCE in undergraduate medical education.

PURPOSE: To compare four standard-setting procedures for an objective structure clinical examination (OSCE). METHODS: A 12-station OSCE was administered to 84 students in each of the final (fourth-) year medical classes of 1996 and 1997 at Dalhousie University Faculty of Medicine. Four standard-setting procedures (Angoff, borderline, relative, and holistic) were applied to the data to establish a cutoff score for a pass/fail decision. RESULTS: The procedures yielded highly inconsistent results. The Angoff and borderline procedures gave similar results; however, the relative and holistic methods gave widely divergent results. The Angoff procedure yielded results reliable enough to use in decision making for a high-stakes examination, but would have required more judges or more stations. CONCLUSIONS: The Angoff and borderline procedures provide reasonable and defensible approaches to standard setting and are practical to apply by non-psychometricians in medical schools. Further investigation of the other procedures is needed.

Education, Medical, Undergraduate↗
PubMed2000

Sequential testing in the assessment of clinical skills.

PURPOSE: Educators who use standardized-patient-based (SP-based) tests may save resources by using sequential testing. In this approach, students take a short screening test; only those who fail take a second test. This study investigated whether sequential testing increases efficiency with only a minor decrease of validity. METHOD: In 1994-95, first- through fourth-year (Group 1) and sixth-year (Group 2) medical students at the University of Maastricht took SP-based tests. Each test took two days. In a simulation experiment based on the data from those tests, the authors considered the first day as the screening test and the second day as the second test. They investigated efficiency and validity as a function of the cutoff score of the screening test. They developed and evaluated a new method to determine the optimum cutoff score of the screening test, a method based on minimization of the loss represented by the (weighted) numbers of false positives and negatives in the screening test. RESULTS: The negative predictive value (probability that a student would fail the complete test if he or she had failed the screening test) was low (<60%), while the positive predictive value was high (>96%). Accordingly, stringent pass/fail cutoff scores in the screening test (75% for Group 1 and 80% for Group 2) produced optimum results. Using those cutoff values, only 26% and 11% of 'the students would have had to take the complete test to get a "true" score, while only 0.2% and 0.0% of the students who passed the screening test went on to fail the complete test (false positives). CONCLUSIONS: In a sequential SP-based test, the pass/fail cutoff score of the screening test should be stringent. This can considerably reduce testing time (30% to 40%), while keeping the percentage of false positives at an acceptably low level of less than 0.2%. As an alternative to receiver operator characteristic analysis, minimization of the loss function was found to be an appropriate method to determine the optimum cutoff value of the screening test.

Clinical Competence↗
PubMed1999

The effect of a 'don't know' option on test scores: number-right and formula scoring compared.

OBJECTIVES: In multiple-choice tests using a 'don't-know' option the number of correct minus incorrect answers was used as the test score (formula scoring) in order to reduce the measurement error resulting from random guessing. In the literature diverging results are reported when comparing formula scoring and number-right scoring, the scoring method without the don't-know option. DESIGN: To investigate which method was most appropriate, both scoring methods were used in true-false tests (block tests) taken at the end of a second- and third-year educational module (block). The students were asked to answer each item initially by choosing from the response options true, false or don't know, and secondly to replace all don't-know answers by a true-false answer. SETTING: Maastricht University, The Netherlands. SUBJECTS: Medical students. RESULTS: The correct scores for the don't-know answered items were found to be 4.5% and 5.9%, respectively, higher than expected with pure random guesswork. This represents a source of bias with formula scoring, because students who were less willing to guess obtained lower scores. The average difference in the correct minus incorrect score for the two scoring methods (2.5%, P < 0.001, and 3.4%, P < 0. 001, respectively) indicates the size of the bias (compare: the standard deviation of the score equals 11%). Test reliability was higher with formula scoring (0.72 vs. 0.66 and 0.74 vs. 0.66), but the difference decreased when the test was restricted to items which were close to the core content of the block (0.81 vs. 0.77, resp. 0. 75 vs. 0.70). CONCLUSIONS: In deciding what scoring method to use, less bias (number-right scoring) has to be weighed against higher reliability (formula scoring). Apart from these psychometric reasons educational factors must be considered.

Education, Medical, Undergraduate↗
PubMed1999

The impact of student-generated learning issues on individual study time and academic achievement.

OBJECTIVES: The aim of this study was twofold. The first question concerns the way students make use of the learning issues they generate (as strict guidelines or as global guidelines) and whether this changes across years of training. The second question concerned the relationship between the way students make use of learning issues and the time spent on individual study and achievement on two tests of knowledge. DESIGN: A questionnaire was developed, containing seven items that measured to what extent students study strictly according to the student-generated learning issues and six items that measured to what extent students study beyond the student-generated learning issues. The questionnaire also contained one question in which students had to estimate the mean time spent on individual study. Achievement was measured by two forms of tests of knowledge, a block test assessing course content and a progress test assessing long-term functional knowledge. SETTING: Medical School of Maastricht University, the Netherlands. SUBJECTS: Medical students (response=69%) from the problem-based curriculum at the Maastricht University. RESULTS: During their first year students study strictly according to the content of the learning issues, whereas in later years students studied more according to their own learning needs and interests. In addition, students who tended to study beyond the generated learning issues spent more time on individual study and achieved better on both tests. CONCLUSIONS: Students in a problem-based curriculum seem to become better self-directed learners during the years of training.

Education, Medical, Undergraduate↗
PubMed1999

Reliability and credibility of an angoff standard setting procedure in progress testing using recent graduates as judges.

INTRODUCTION: Progress testing is an assessment method that samples the complete domain of knowledge that is considered pertinent to undergraduate medical education. Because of the comprehensive nature of this test, it is very difficult to set a passing score. We obtained a progress test standard using an Angoff procedure with recent graduates as judges. This paper reports on the reliability and credibility of this approach. METHODS: The Angoff procedure was applied to a sample of 146 progress test items. The items were judged by a panel of eight recently graduated students. Generalizability theory was used to investigate the reliability as a function of the number of items and judges. Credibility was judged by comparing the pass/fail rates resulting from the standard arrived at by the Angoff procedure with those obtained using a relative and a fixed standard. RESULTS: The results indicate that an acceptable error score can be achieved, yielding a precision within one percentage on the scoring scale, by using 10 judges on a full-length progress test (i.e. 250 items). The pass/fail rates associated with the Angoff standard came closest to those of the relative standard, which takes variations in test difficulty into account. A high correlation was found between item-Angoff estimates and the item P-values. CONCLUSION: The results of this study suggest that the Angoff procedure, using recently graduated students as judges, is an appropriate standard setting method for a progress test.

Education, Medical, Undergraduate↗
PubMed1999

Profiles of effective tutors in problem-based learning: scaffolding student learning.

OBJECTIVES: Research on tutoring in problem-based learning has not focused so far on the variation in tutoring and how this variation can be interpreted by conceptions about effective tutoring. DESIGN: This study focuses on the profiles of tutors generated by means of an instrument, the so-called Tutor Intervention Profile (TIP), and tries to determine which profiles are more or less effective. The TIP contains four dimensions of tutor behaviour: (1) elaboration; (2) directing the learning process; (3) integration of knowledge; and (4) stimulating interaction and individual accountability. SETTING: The medical school of the University of Maastricht, The Netherlands. SUBJECTS: Sixty-seven tutors who run 67 tutorial groups across three units (courses) in the academic year 1996-97. RESULTS: It appeared that high, average and low performing tutors differ in their performance on each of the four dimensions of the TIP. Several different profiles of tutor performance could be distinguished, which were more or less effective. One group of tutors demonstrated a tutor intervention profile that was characterized as relying more on the use of expert knowledge, whereas another group of tutors was characterized as relying more on their abilities to stimulate the learning process in the tutorial group. The tutor intervention profile that was perceived by students as most effective showed high scores on each of the four dimensions, as expected. Notably, a tutor stressing the learning process in the tutorial group was perceived as more effective than a tutor stressing content (expert tutor). This is especially true for a relatively poor scoring tutor. CONCLUSIONS: The results of this study are consistent with research on human tutoring and research on tutoring in problem-based learning.

Educational Measurement↗
PubMed1998

Tutor intervention profile: reliability and validity.

An instrument has been developed to provide tutors with feedback about their performance in problem-based tutorial groups. This questionnaire consists of 33 statements reflecting the behaviour of the tutor in the tutorial group when students analyse the case and generate learning issues and the behaviour of the tutor when students report back to the tutorial group about their self-study. The statements are grouped on theoretical dimensions found in the literature about co-operative learning and problem-based learning. This study focuses on the validity and the reliability of this instrument. Confirmatory factor analysis showed that a four-factor model showed a weak fit, whereas each separate factor fitted the data reasonably well. The four factors are: (1) elaboration; (2) directing the learning process; (3) integration of knowledge and (4) stimulating interaction and individual accountability. These factors were strongly correlated with the overall judgement of the tutor at the end of the unit, so the concurrent validity of this instrument was high. Generalizability studies indicated that the rating scales provide reliable information with a sample size of student responses falling within the range of real tutorial group sizes. The results of this study show that the instrument for providing tutors with feedback is valid and reliable. This questionnaire can be used in staff development activities and also provides needs assessment for faculty development.

Education, Medical, Undergraduate↗