Criterion-related measures in education.
Explore the source record for details and available documents.
SEARCH · PubMed Health
Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
BACKGROUND: Evaluating residency programs requires objective assessment tools, but few are readily available. The purpose of this study was to measure education by correlating resident test scores with several measurements of educator performance. MATERIALS AND METHODS: The study group included residents and educators from a single residency program. We performed a retrospective analysis of scores from the Orthopaedic In-Training Examination collected during a 6-year period. Resident examination scores were indexed by dividing program averages by national averages to determine yearly score trends and then were correlated with educator attendance and teaching hours. Subspecialty scores were ranked to gauge residency strengths and weaknesses. Teaching hours devoted to subspecialties were compared with test scores to measure curricular emphases and to appraise teaching efficiency. RESULTS: Yearly average examination scores were proportional to national averages (P < 0.001). However, of 3436 possible educator-score associations, only 15 scores correlated highly (r > 0.9) with educators, and only 26 were significant (P < 0.05). Trend analysis put subspecialty scores in yearly perspective. Ranking was inaccurate until scores were indexed to the national average. In 2002, the distribution of 238 teaching hours ranged from 4 to 48 h for subspecialties, and 9 of 12 subspecialties were emphasized disproportionately to the examination. Teaching efficiency varied more than 10-fold by subspecialty. CONCLUSIONS: The creation of a score index helped to identify and address imbalances between teaching hours devoted to subspecialties and resident needs as evidenced by low In-Training examination scores. The present study improved educator accountability by correlating measurements of teaching and learning.
Measuring the quality of the medical education imparted in a school or faculty requires a conceptual frame of reference that take account of the socioeconomic and political context and the structure and standards of the health services in the country considered. Quality must be judged by the extent to which medical training responds not only to current but also to future needs and expectations. Hence the determination of quality also requires a prospective study. A study is made of the situation in the medical schools of Latin America with a view to establishing how medical education was and is in the Region and those characteristics are taken as a yardstick for a school that can meet the demands of the future, preferably toward the year 2000, the horizon year for the world health goals. The author builds a detailed "objective-image" model of medical education in the year 2000, from which he derives four attributes or indicators that can be used to evaluate the education process in any institution. These indicators relate to four salient aspects of medical education: context, structure, function and integration.
The Dundee Ready Educational Environment Measure (DREEM) is a measure of students' perceptions of the educational environment, but its performance in evaluating the educational environment in the residency setting has not yet been described. This study aimed at describing the psychometric performance of DREEM applied to medical residents. DREEM was applied to 97 residents from 12 training programs on four specialties in six institutions in three Brazilian cities. Psychometric measures included factor analysis, Cronbach's alpha coefficients, item-to-total correlations, t-test comparisons of scores between genders, institutions, specialties, and programs, correlations with the global score of the Quality of School Life Scale (concurrent validity), and test-retest reliability. Generalizability theory procedures were applied to a random subset of data. Programs (8) were the objects of measure, while institutions (6), specialties (4), raters-within-programs (40), and items-on-the-scale (50) were facets. Variance components, generalizability (G) and dependability (D) coefficients were calculated. Cronbach's alpha was 0.93. DREEM showed high discriminant and concurrent validities. Test-retest reliability was moderate. Interactions between programs, raters and items accounted for 68% of the total variance. G and D coefficients were 0.95 and 0.67, respectively. The instrument proved to be useful for relative comparisons at both resident and program level.
Most sexuality education curricula developed the past 20 years were not thoroughly evaluated. This study provides results from a content analysis of 10 sexuality education curricula for junior and senior high school students. Nine nationally available sexuality education curricula and one curriculum guide comprised the sample. The basis for analysis was the Guidelines for Comprehensive Sexuality Education, developed by the Sex Information and Education Council of the U.S. (SIECUS) and an instrument developed to measure bias in the curricula. Trained coders found that Sex Respect and Teen Aid addressed less than half the topics suggested by the SIECUS guidelines. Several of the curricula contained gender and sexual orientation bias. Certain key concepts such as "Sexual Behavior" and "Society and Culture" were not adequately addressed by most of the curricula. Findings indicate that of 10 curricula, only six are considered acceptable for educating junior and senior high school students.
Students' perceptions of their educational environment have been studied at all levels of the education system, from primary through post-secondary education. Recent imperatives towards enhanced quality assessment monitoring at a time when health professions education is increasingly committed to student-centred teaching and learning have stimulated a revival of interest in this field. This paper reports a body of research in health professions institutions around the world based on the Dundee Ready Educational Environment Measure (DREEM), a reliable, validated inventory that claims to be generic to undergraduate health professions education and non-culturally specific.
The study aims at measuring doctors' performance while giving therapeutic instructions and testing the effectiveness of an educational technique for affective objectives. For this purpose doctors' performance was analysed into nine components. Twenty-five trainees in internal medicine were observed and rated regarding the nine components, while instructing 40 patients with a chronic disease. Both doctors and patients were then questioned regarding the verbal components of the doctors' performance and the answers were compared to assess the degree of concordance. One to 3 months later the patients were interviewed to assess whether they complied with the instructions or not. Eighteen months later a 4-hour seminar on affective objectives was held, involving listening and small-group discussion on tape-recorded doctor-patient consultations. After completion of the seminar, 25 doctors instructed 39 patients and were assessed as above. The findings suggest that the doctors' performance regarding the nine components did not predict patient compliance. However, sufficient doctor-patient concordance regarding the verbal components of the behaviour significantly predicted patient compliance. Following the seminar, significant improvement was noted in doctors' performance, in doctor-patient communication and in patient compliance. It is concluded that observing the doctors' behaviour independently of the patient does not predict patients' compliance; the analysis of the total doctors' behaviour into components is valid in predicting patient compliance if it is used in order to assess doctor-patient communication; and a programme based on audiotape-assisted education is both simple and effective in improving the trainees' behaviour and communication skills.
OBJECTIVE: There are no accepted and practical measures of the relative clinical and educational activities of pediatricians who work in an academic health science center. Such measures are necessary for justification of existing and future human resource plans and evaluation of the activities and performance of physicians. The limited literature on the measurement of physician workload usually focuses on a specific subspecialty group and does not account for such issues as indirect patient care, such as telephone calls or e-mail consultations; variables that affect the delivery of clinical care, including patient acuity and complexity; and the presence of students during the patient care activities. After completing a pilot study that assessed the educational workload of faculty members, we adapted existing personal digital assistant technology and software to document clinical and educational activities. METHODS: Twenty full-time physicians from 4 subspecialty pediatric divisions participated in a 2-week evaluation project in May through June 2005. Clinical activities, with and without trainees, and educational activities were collected with the use of personal digital assistants. Software allowed an individualized division-specific drop-down menu. Information that was collected included clinical (location of activity, diagnosis, and time requirement) and educational activities. After completion of a 2-week data collection period, each physician was asked to complete a 5-question evaluation form. RESULTS: The project was completed successfully with capture of additional clinical and educational activities. A 5-question evaluation form was completed by 70% of the participants at the end of the 2-week data collection. Data on clinical and educational activities were analyzed qualitatively and graphed. CONCLUSIONS: This method of workload data collection added significant information in capturing activities that are not measured in traditional workload evaluations for either clinical activities, such as e-mail, telephone, and patient information review, or educational endeavors, including mentoring and educational lectures and presentations.
The present study was designed to determine whether 4 measures of children's spontaneous speech and language differed according to the educational level of the children's mothers. Spontaneous language samples from 240 three-year-old children were analyzed to determine mean length of utterance in morphemes (MLUm), number of different words (NDW), total number of words (TNW), and percentage of consonants correct (PCC). A norm-referenced, knowledge-dependent measure of language comprehension, the Peabody Picture Vocabulary Test-Revised (PPVT-R), was also included for purposes of comparison with the spontaneous measures. Three levels of maternal education were compared: less than high school graduate, high school graduate, and college graduate. Trend analyses showed statistically significant linear trends across educational levels for MLUm, NDW, TNW, and PPVT-R; the trend for PCC was not significant. The relationship of maternal education and other sociodemographic variables to measures of children's language should be examined before using such measures to identify children with language disorders.
The National High Blood Pressure Education Program has been designed to translate the results of basic and clinical research to medical practice through a program of education for the public, patients, and health professionals. It has continuously used health-education principals of media development, patient education, social networking, community organizations, theories of social change, and program evaluation and measurement to reach its objectives. After 15 years, public knowledge regarding blood pressure and its sequelae has improved dramatically.
A multidimensional Rasch model was applied to two instruments measuring abilities in two related areas of a university general education curriculum. Grades from related courses were also calibrated using the Rasch model. Thus, course grades, test items, and persons were all placed on the same metric. Incorporating grades within the metric provided additional meaning to the measures; instructors could see which items were matched to students in a particular grade range for a course. This could help not only in interpreting items but also in interpreting grades. Test items and grades fit the model reasonably well, with adequate person separation reliability.
Explore the source record for details and available documents.
Clinical practice guidelines have introduced a new dimension to facility-level performance measurement--one which requires caregivers to change the way they have traditionally evaluated patient care practices. To successfully integrate guideline-derived evaluation tools into their performance measurement activities, practitioners and quality management professionals face new education and training challenges.