Carol F Whitfield | PubMed Health

PubMed2002

Correlation of problem-based learning facilitators' scores with student performance on written exams.

Problem-based learning (PBL) is widely used in medical education. In some cases, facilitators assign a grade to reflect a student's performance in small-group sessions. In our PBL track, facilitators were asked to assess student knowledge base independent of their group participatory skills. To determine if facilitators' grades were correlated with student performance in written exams, a retrospective study of data from our PBL track was undertaken. Data from 156 students and 107 facilitators in six years of a PBL track at Penn State College of Medicine was analyzed by the Pearson correlation after pairing facilitator grades with written exam grades for each of the eight blocks of the curriculum. Exam reliability and validity were assessed by Cronbach's alpha and correlation with USMLE I board scores. The mean alpha was 0.549 +/- 0.221. The mean correlation with USMLE scores was 0.558 +/- 0.151. Facilitators' scores for knowledge were positively associated with students' exam grades. The corresponding significant Pearson correlation coefficients were between 0.342-0.622. However, the coefficients of determination showed that the correlation was not significant. Coefficients of determination showed that the knowledge scores explained only 12 to 39% of the variance in exam scores. Overestimation by facilitators was significantly (p < 0.0001) greater for students in the bottom 25% of the class by exam score than for students in the top 25% of the class. On the basis of this study, we concluded that facilitator assessment of student knowledge base is not useful.

Curriculum↗

PubMed2002

Differences between students in problem-based and lecture-based curricula measured by clerkship performance ratings at the beginning of the third year.

BACKGROUND: Problem-based learning (PBL) is being incorporated into more medical curricula, but its influence on subsequent clinical performance remains unclear. PURPOSE: To determine if PBL leads to better scores for fund of knowledge or clinical problem-solving skills in required clerkships taken early in the 3rd year at Penn State College of Medicine. METHODS: Data were collected from 6 class years, for clinical clerkship subscores completed during the first 4 months of the 3rd year, of students completing 1 or 2 years in a PBL or traditional track. Clerkship scores were analyzed as individual clerkships and as the average across clerkships for each student. Statistical analysis included a comparison of clerkship scores between the 2 tracks; using a 2-sample t test, and calculation of effect sizes. A multiple regression model was also employed to adjust for age, gender, race, preadmission grade point average, and Medical College Admission Test (MCAT). RESULTS: Mean scores of individual clerkships taken by problem-based or lecture-based students differed significantly in some clerkships, but the effect size was small. The effect sizes for fund of knowledge for the 6 clerkships ranged from 0.20 to 0.41; for clinical problem-solving skills, they ranged from 0.26 to 0.39. These differences between the problem-based and lecture-based students were of the same magnitude as the difference at the start of medical school on the MCAT, namely d = 0.31. There was a trend toward higher effect sizes in students having 2 rather than 1 year of PBL, and in later iterations of the track. CONCLUSION: PBL effect size on students' scores for fund of knowledge and clinical problem-solving skills was small to moderate in various years.

Clinical Clerkship↗