PubMed Health⌕ Search

Biomedical subjects

Geoffrey R Norman

Publications and source records attributed to Geoffrey R Norman.

At least 19 recordsLinked to original sources

The role of biomedical knowledge in diagnosis of difficult clinical cases.

Although biomedical knowledge is believed to be of little value in diagnosis of routine clinical cases, studies of clinical reasoning have found that physicians revert to use of basic biomedical knowledge when faced with challenging clinical problems. The current paper presents two experiments that empirically examine the role of biomedical knowledge in diagnosis of difficult cases by novice diagnosticians. Novices are taught to diagnose a series of artificial diseases using either knowledge of causal mechanisms or a list of clinical features. In Experiment 1, participants are then tested on two types of clinical challenges: (1) case summaries with irrelevant findings; (2) cases using unfamiliar terminology. Participants with an understanding of underlying mechanisms out performed their counterparts on both types of cases. In Experiment 2, participants are tested 1 week after initial training. Participants with knowledge of causal mechanisms were found to do better on cases with unfamiliar terminology. The results of the two studies provide additional support for the critical role of biomedical knowledge in diagnosis of difficult clinical cases.

Analysis of Variance↗

Validation of the SETOC instrument -- Student evaluation of teaching in outpatient clinics.

PURPOSE: There is a paucity of evaluation forms specifically developed and validated for outpatient settings. The purpose of this study was to develop and validate an instrument specifically for evaluating outpatient teaching, to provide reliable and valid ratings for individual and group feedback to faculty, and to identify outstanding teachers in that setting. METHOD: By literature reviews and pilot studies at the Faculties of Health Sciences, McMaster University (Canada) and Aga Khan University (AKU-Pakistan), a 15-item instrument, Student Evaluation of Teaching in Outpatient Clinics (SETOC), was created with five subscales: "Establishing Learning-Milieu, Clinical-Teaching, General-Teaching, Clinical-Competence, and Global-Rating." Seven-point Likert-type rating scales were used. Students also nominated three "best" outpatient teachers. PARTICIPANTS: 87 faculty members (79%) rated by all 224 third to fifth-year students (clerks) at outpatient departments of the AKU hospital over a one-year period. ANALYSES: Repeated measures generalizability studies, correlations, concurrent validity of SETOC scores with best teacher nominations. RESULTS: Inter-rater G-coefficient and internal consistency of SETOC student ratings were 0.92 and 0.98. Average inter-item and inter-subscale correlations were 0.79 and 0.86. Comparing SETOC scores against "Best Teacher" nominations, sensitivity, specificity, positive and negative predictive values were greater than 0.84. Student ratings ranged from unsatisfactory (fourteen instructors) to outstanding (four instructors). Mean-scores for Learning-Milieu, Clinical-Teaching and General-Teaching were lower than those for Clinical-Competence and Global-Rating (p=0.000 for all). CONCLUSIONS: The SETOC elicited reliable and valid student ratings that can provide specific feedback to individual faculty with weak or outstanding teaching skills, and identify overall group shortcomings for faculty development.

Education, Medical↗

Using a sampling strategy to address psychometric challenges in tutorial-based assessments.

INTRODUCTION: Tutorial-based assessment, despite providing a good match with the philosophy adopted by educational programmes that emphasize small group learning, remains one of the greatest challenges for educators working in this context. The current study was performed in an attempt to assess the psychometric characteristics of tutorial-based evaluation upon adopting a multiple sampling approach that requires minimal recording of observations. METHOD: After reviewing the literature, a simple 3-item evaluation form was created. The items were "Professional Behaviour," "Contribution to Group Process," and "Contribution to Group Content." Explicit definition of these items was provided on an evaluation form. Twenty five tutors in five different programmes were asked to use the form to evaluate their students (N=169) after every tutorial over the course of an academic unit. Each item was rated using a 10-point scale. RESULTS: Cronbach's alpha revealed an appropriate internal consistency in all five programmes. Test-retest reliability of any single rating was low, but the reliability of the average rating was at least 0.75 in all cases. The construct validity of the tool was supported by the observation of increasing ratings over the course of the academic unit and by the finding that more senior students received higher ratings than more junior students. CONCLUSION: Consistent with the context specificity phenomenon, the adoption of a "minimal observations often" approach to tutorial-based assessment appears to maintain better psychometric characteristics than do attempts to assess tutorial performance using more comprehensive measurement tools.

Curriculum↗

Innovations in problem-based learning: what can we learn from recent studies?

This article aims at discussing the six papers included in this special issue on innovations in Problem-based learning (PBL). The papers address different aspects related to the implementation and the development of PBL. This discussion article highlights the relevance of the theme explored by each of the papers, the contributions emerging from the study to what is already known about that topic, and its limitations, particularly those that suggest directions for future research. Emphasis is given to new insights brought by the papers for better understanding tutorial group processes and self-study phase in PBL. The contributions provided by the papers are discussed in the light of pertinent literature and also in relation to their companion articles in this issue when indicated.

Cooperative Behavior↗

"Precision" and "accuracy": two terms that are neither.

OBJECTIVE: In some publications, the terms "precision" and "accuracy" are used as if they were synonyms for "reliability" and "validity." METHODS AND RESULTS: This article shows that these terms are neither precise nor accurate when used in this way. Scales can demonstrate high test-retest or interrater reliability (i.e., they are "precise") but still be unreliable in certain circumstances; and "imprecise" scales can still show good reliability. Further, "accuracy" as a synonym for validity reflects an outdated conceptualization of validity, which has been superseded by one that emphasizes that validity tells us what conclusions can be drawn about a person based on a test result. CONCLUSION: The article ends with a call for the use of the more traditional terms as better reflecting the process of scale development and the uses to which they are put.

Health Status Indicators↗

Standard setting in a small scale OSCE: a comparison of the Modified Borderline-Group Method and the Borderline Regression Method.

When setting standards, administrators of small-scale OSCEs often face several challenges, including a lack of resources, a lack of available expertise in statistics, and difficulty in recruiting judges. The Modified Borderline-Group Method is a standard setting procedure that compensates for these challenges by using physician examiners and is easy to use making it a good choice for small scale OSCEs. Unfortunately, the use of this approach may introduce a new challenge. Because a small scale OSCE has a small number of examinees, there may be few examinees in the borderline range, which could introduce an unintentional bias. A standard setting method called The Borderline Regression Method will be described. This standard setting method is similar to the Modified Borderline-Group Method but incorporates a linear regression approach allowing the cut score to be set using the scores from all examinees and not from a subset. The current study uses confidence intervals to analyze the precision of cut scores derived from both approaches when applied to a small scale OSCE.

Bias↗

The value of basic science in clinical diagnosis.

BACKGROUND: The role of basic science knowledge in clinical diagnosis is unclear. There has been no experimental demonstration of its value in helping students recall and organize clinical information. This study examines how causal knowledge may lead to better recall and diagnostic skill over time. METHOD: Undergraduate medical students learned either four neurological or rheumatic disorders. One group learned a basic science explanation for the symptoms. The other learned epidemiological information. Both were then tested with the same set of clinical cases immediately after learning and one week later. RESULTS: On immediate test, there was no difference in accuracy (70% for both groups). However, one week later, performance in the epidemiology group dropped to 51%; the basic science group only dropped to 62%. CONCLUSIONS: Basic science knowledge relating causal knowledge to disease symptoms can improve diagnostic accuracy after a delay.

Education, Medical, Undergraduate↗

Speed kills? Speed, accuracy, encapsulations and causal understanding.

BACKGROUND: The role of basic science, which provides causal explanations for clinical phenomena in medical education, is poorly understood. Schmidt has postulated that expert clinicians maintain this knowledge in 'encapsulated' form, indexed by words or phrases describing the processes. In the present paper we show that students who learn causal explanations have a more coherent understanding of the relation between diseases and clinical features which, in turn, influences recognition of words or phrases describing 'encapsulated knowledge' and the ability to maintain performance under speeded conditions. HYPOTHESES: In comparison to students who simply learn the features of 4 diagnostic categories, students who learn a causal explanation will: (a) recognise words describing encapsulated knowledge more accurately and (b) maintain or improve diagnostic performance under speeded conditions. METHODS: Two studies were conducted involving 4 'pseudo-endocrinology' diseases and undergraduate psychology students. One group learned signs and symptoms alone; the second group also learned a causal explanation. In study 1, they were then given a recognition memory task. In study 2, they were asked to diagnose new cases either (i) as quickly as possible or (ii) taking their time. RESULTS: In study 1, while there was no difference in recognising old words (90% versus 91%), the causal group was better able to recognise encapsulated and novel consistent words (50% versus 41%) (P = 0.02). In study 2 there was an interaction; causal students performed better under speeded conditions (71% versus 66%) but worse under thoroughness conditions (67% versus 73%), as predicted. CONCLUSIONS: Causal understanding leads to more coherent understanding of clinical conditions, which in turn leads to expert-like behaviour.

Clinical Competence↗

The relation between the minimally important difference and patient benefit.

A critical issue in the examination of the effects of treatments on health-related quality of life is how to determine whether a particular change is clinically relevant. One approach is the so-called anchor-based method derived from patient or clinician estimates of minimal change (the Minimally Important Difference or MID). At issue, however, is whether this criterion provides a meaningful way to differentiate between beneficial and ineffective treatments. In this paper, I show that the likelihood that a patient will benefit from treatment, or alternatively, the number of patients in a given cohort who will benefit from treatment, can be predicted with considerable precision from the Effect Size, and the particular choice of MID bears almost no relation to the projected benefit. To examine the relation between the threshold of minimal difference, the effect size of treatment, and the likelihood that a patient will benefit from treatment, a simulation based on a normal distribution was used to compute the proportion of patients benefiting for various values of the ES and the MID. The agreement of the simulation with empirical data from four studies of asthma and respiratory disease was examined. The simulation showed a near-linear relationship between ES and the likelihood of benefit, which was nearly independent of the value of the MID. Agreement of the simulation with the empirical data was excellent. Introducing moderate skew into the distributions had minimal impact on the relationship. The proportion of patients who will benefit from treatment can be directly estimated from the effect size, and is nearly independent of the choice of MID. Effect size- and anchor-based approaches provide equivalent information in this situation. There appears to be little utility in the notion of the MID as an absolute indicator of clinically important treatment effects.

Data Interpretation, Statistical↗

The value of basic science in clinical diagnosis: creating coherence among signs and symptoms.

BACKGROUND: We investigated whether learning basic science mechanisms may have mnemonic value in helping students remember signs and symptoms, in comparison with learning the relation between symptoms and diagnoses directly. PURPOSE: To compare 2 approaches to learning diagnosis: learning how features of various conditions relate to underlying pathophysiological mechanisms and learning the conditional probabilities of features and diseases. METHODS: Undergraduate students (n = 36) were taught 4 disorders (upper motor neuron lesion, lower motor neuron lesion, neuromuscular junction disease and muscular disease), either using basic science explanations or (symptom x disease) probabilities. They were tested with diagnostic cases immediately after learning and 1 week later. RESULTS: On the immediate test, there was no difference in the results. One week later, the accuracy of the mechanism group remained at 0.52, but the performance of the probability group had dropped to 0.43. CONCLUSIONS: Knowledge of basic science may have value in clinical diagnosis by helping students recall or reconstruct the relationships between features and diagnoses.

Clinical Competence↗

How can I know what I don't know? Poor self assessment in a well-defined domain.

As the rapidity with which medical knowledge is generated and disseminated becomes amplified, an increasing emphasis has been placed on the need for physicians to develop the skills necessary for life-long learning. One such skill is the ability to evaluate one's own deficiencies. A ubiquitous finding in the study of self-assessment, however, is that self-ratings are poorly correlated with other performance measures. Still, many educators view the ability to recognize and communicate one's deficiencies as an important component of adult learning. As a result, two studies have been performed in an attempt to improve upon this status quo. First, we tried to re-define the limits within which self-assessments should be used, using Rosenblit and Keil's argument that calibration between perceived and actual performance will be better within taxonomies that are regularly tested (e.g., factual knowledge) compared to those that are not (e.g., conceptual knowledge). Second, we tried to norm reference individuals based on both the performance of their colleagues and their own historical performance on McMaster's Personal Progress Inventory (a multiple choice question test of medical knowledge). While it appears that students are able to (a) make macro-level self-assessments (i.e., to recognize that third year students typically outperform first year students), and (b) judge their performance relatively accurately after the fact, students were unable to predict the percentage of questions they would answer correctly with a testing procedure in which they have had a substantial amount of feedback. Previous test score was a much better predictor of current test performance than were individuals' expectations.

Adult↗

Using comprehensive feature lists to bias medical diagnosis.

Clinicians routinely report fewer features in a case than they subsequently agree are present. The authors report studies that assess the effect of considering a more comprehensive description than physicians usually offer. These comprehensive descriptions were generated from photographs of dermatology and internal medicine and were complete and accurate. Groups of clinicians of varying expertise were asked to offer a diagnosis based solely on the comprehensive verbal description. This initial exercise decreased the subsequent diagnostic acumen of experienced participants with the photographs relative to a group that initially diagnosed from the photographs. Reasons that the initial consideration of a list of features, all of which are present in the photograph, would decrease diagnostic accuracy are discussed.

Dermatology↗