PubMed Health⌕ Search

Biomedical subjects

J Norcini

Publications and source records attributed to J Norcini.

11 recordsLinked to original sources
PubMed2001

Setting performance standards for medical practice: a theoretical framework.

BACKGROUND: The assessment of performance in the real world of medical practice is now widely accepted as the goal of assessment at the postgraduate level. This is largely a validity issue, as it is recognised that tests of knowledge and in clinical simulations cannot on their own really measure how medical practitioners function in the broader health care system. However, the development of standards for performance-based assessment is not as well understood as in competency assessment, where simulations can more readily reflect narrower issues of knowledge and skills. This paper proposes a theoretical framework for the development of standards that reflect the more complex world in which experienced medical practitioners work. METHODS: The paper reflects the combined experiences of a group of education researchers and the results of literature searches that included identifying current health system data sources that might contribute information to the measurement of standards. CONCLUSION: Standards that reflect the complexity of medical practice may best be developed through an "expert systems" analysis of clinical conditions for which desired health care outcomes reflect the contribution of several health professionals within a complex, three-dimensional, contextual model. Examples of the model are provided, but further work is needed to test validity and measurability.

Clinical Competence↗
PubMed2000

A review of the evaluation of clinical teaching: new perspectives and challenges.

PURPOSE: This article discusses the importance of the process of evaluation of clinical teaching for the individual teacher and for the programme. Measurement principles, including validity, reliability, efficiency and feasibility, and methods to evaluate clinical teaching are reviewed. CONTEXT: Evaluation is usually carried out from the perspective of the learner. This article broadens the evaluation to include the perspectives of the teacher, the patient and the institutional administrators and payers in the health care system and recommends evaluation strategies. RESULTS: Each perspective provides specific feedback on factors or attributes of the clinical teacher's performance in the domains of medical expert, professional, scholar, communicator, collaborator, patient advocate and manager. Teachers should be evaluated in all domains relevant to their teaching objectives; these include knowledge, clinical competence, teaching effectiveness and professional attributes. CONCLUSIONS AND IMPLICATIONS: Using this model of evaluation, a connection can be made between teaching and learning about all the expected roles of a physician. This can form the basis for systematic investigation into the relationship between the quality of teaching and the desired outcomes, the improvement of student learning and the achievement of better health care practice. It is suggested that the extent of effort and resources devoted to evaluation should be commensurate with the value assigned to the evaluation process and its outcomes.

Education, Medical↗
PubMed2000

Clinical teaching: maintaining an educational role for doctors in the new health care environment.

CONTEXT AND OBJECTIVES: Good clinical teaching is central to medical education but there is concern about maintaining this in contemporary, pressured health care environments. This paper aims to demonstrate that good clinical practice is at the heart of good clinical teaching. METHODS: Seven roles are used as a framework for analysing good clinical teaching. The roles are medical expert, communicator, collaborator, manager, advocate, scholar and professional. RESULTS: The analysis of clinical teaching and clinical practice demonstrates that they are closely linked. As experts, clinical teachers are involved in research, information retrieval and sharing of knowledge or teaching. Good communication with trainees, patients and colleagues defines teaching excellence. Clinicians can 'teach' collaboration by acting as role models and by encouraging learners to understand the responsibilities of other health professionals. As managers, clinicians can apply their skills to the effective management of learning resources. Similarly skills as advocates at the individual, community and population level can be passed on in educational encounters. The clinicians' responsibilities as scholars are most readily applied to teaching activities. Clinicians have clear roles in taking scholarly approaches to their practice and demonstrating them to others. CONCLUSION: Good clinical teaching is concerned with providing role models for good practice, making good practice visible and explaining it to trainees. This is the very basis of clinicians as professionals, the seventh role, and should be the foundation for the further development of clinicians as excellent clinical teachers.

Australia↗
PubMed1998

Association between licensing examination scores and resource use and quality of care in primary care practice.

CONTEXT: Clinical competence is a determinant of the quality of care delivered, and may be associated with use of health care resources by primary care physicians. Clinical competence is assumed to be assessed by licensing examinations, yet there is a paucity of information on whether scores achieved predict subsequent practice. OBJECTIVE: To determine if licensing examination scores were associated with selected aspects of quality of care and resource use in initial primary care practice. DESIGN: Prospective cohort study of recently licensed family physicians, followed up for the first 18 months of practice. SETTING: The Quebec health care system. PARTICIPANTS: A total of 614 family physicians who passed the licensing examination between 1991 and 1993 and entered fee-for-service practice in Quebec. MAIN OUTCOME MEASURES: All patients seen by physicians were identified by the universal health insurance board and all health services provided to these patients were retrieved for the 18 months prior to (baseline) and after (follow-up) the physicians' entry into practice. Medical service and prescription claims files were used to measure rates of resource use (specialty consultation, symptom-relief prescribing compared with disease-specific prescribing) and quality of care (inappropriate prescribing, mammography screening). Baseline data were used to adjust for differences in practice population. RESULTS: Study physicians saw a total of 1116389 patients, of whom 113535 (10.2%) were elderly and 83391 (7.5%) were women aged 50 to 69 years. Physicians with higher licensing examination scores referred more of their patients for consultation (3.8/1000 patients per SD increase in score; 95% confidence interval [CI], 1.2-7.0; P = .005), prescribed to elderly patients fewer inappropriate medications (-2.7/1000 patients per SD increase in score; 95% CI, -4.8 to -0.7; P=.009) and more disease-specific medications relative to symptom-relief medications (3.9/1000 patients per SD increase in score; 95% CI, 0.3 to 7.4; P= .03), and referred more women aged 50 to 69 years (6.6/1000 patients per SD increase in score; 95% CI, 1.2-11.9; P = .02) for mammography screening. If patients of physicians with the lowest scores had experienced the same rates of consultation, prescribing, and screening as patients of physicians with the highest scores, an additional 3027 patients would have been referred, 179 fewer elderly patients would have been prescribed symptom-relief medication, 912 more elderly patients would have been prescribed disease-specific medication, 189 fewer patients would have received inappropriate medication, and 121 more women would have received mammography screening. CONCLUSIONS: Licensing examination scores are significant predictors of consultation, prescribing, and mammography screening rates in initial primary care practice.

Adult↗
PubMed1994

Does feedback on examination performance help directors of internal medicine residencies evaluate the medical knowledge of their residents against national norms?

BACKGROUND: As part of the admission process to the American Board of Internal Medicine's certifying examination in internal medicine, training program directors evaluate residents in several components of clinical competence, including medical knowledge. Research suggested that these ratings had different meanings across programs. A report comparing certifying examination performance and ratings of medical knowledge at the program and national levels was developed and sent to program directors after the 1988 through 1992 examinations. The present study investigated whether feedback helped program directors identify where their residents ranked nationally. METHOD: Subjects were first-time takers of the 1986 through 1992 certifying examinations in internal medicine who took the examination in the year they completed training and who received ratings of 4 through 9 in medical knowledge. All subjects were from programs contributing examinees in all seven study years and that received feedback in 1988 through 1991. Year-by-year distributions of program mean percentages of examinees receiving each rating of medical knowledge (4 through 9) were generated. Program means for equated examination scores and ratings of medical knowledge were computed for each year. Correlations between program mean scores and ratings were also computed. RESULTS: The distributions of the ratings were stable across the study years. Mean scores declined while mean ratings were unchanged. At the same time, correlations between scores and ratings increased. The biggest one-year change was from 1989 to 1990 (.49 to .57). CONCLUSION: Since equated scores are directly comparable, declining mean scores but unchanged mean ratings suggest that the standards applied by program directors drifted downward. The increasing correlations suggest that program directors improved in their abilities to evaluate residents relative to a common standard. It is not clear what effect the feedback had on program directors' evaluations. It is encouraging, however, to see a higher level of agreement among program directors on the meaning of the ratings.

Certification↗
PubMed1992

Accuracy and reproducibility of visual coronary stenosis estimates using information from multiple observers.

The reliability of visual estimation of severity of coronary artery stenosis may be improved using data from multiple independent observers. Data were collected from the results of a video format examination used on an experimental basis in 1987 by the American Board of Internal Medicine to test 61 candidates for certification in cardiovascular diseases. Twenty arteriographic cases were presented in a standardized format. Each artery was viewed in multiple projections including angled views. Each view was shown in both real time and slow motion, and each case was seen twice in its entirety. The observers rated stenosis severity on a four-point scale ranging from 1-4. A two-way repeated measures analysis of variance was performed on the tabulated results, yielding variance components for the arteriographic data (signal), the differences among observers, and the observer by case interaction (both considered noise). These components then allowed calculation of 68 and 95% confidence intervals, the signal-to-noise ratio, and the reproducibility coefficient for any number of observers. When a single observer was considered, reproducibility was low, with 95% confidence intervals of +/- 0.9 points, corresponding to approximately +/- 22% diameter stenosis. However, when data of three observers were averaged, the 95% confidence interval decreased to +/- 0.52 points (13% stenosis), signal-to-noise ratio rose to 12.2, and reproducibility coefficient was 0.92. Relatively small increments in these values were noted when data from a fourth or fifth observer were added. In comparison to a computer-assisted quantitative method, 86% accuracy was found for the results of averaged subjective determinations of stenosis severity.(ABSTRACT TRUNCATED AT 250 WORDS)

Constriction, Pathologic↗
PubMed1989

The first certifying examination in geriatric medicine.

On April 22, 1988, the first Certifying Examination in Geriatric Medicine was administered jointly by the American Board of Internal Medicine and the American Board of Family Practice to 4,282 diplomates (ABIM = 2,202; ABFP = 2,080). This paper addresses both an analysis of the examination and the relationship between performance on that examination and a group of characteristics of the examinees, collected as part of the registration process. The pass rate was 56%. Performance on the examination was positively correlated with scores on the general certifying examinations and with training in geriatric medicine. Data provided by the candidates in an addendum to the application were also available for analysis and were used to derive correlations with groups of questions. The performance of candidates was positively correlated with seeing large numbers of elderly in hospitals, nursing homes, or home settings, working in a University Hospital, teaching and research, and the size of the community in which the candidate practiced. Physicians from long-term care settings did exceptionally well. Working in a solo practice setting was negatively correlated with performance on the examination as was working in a for-profit setting.

Certification↗