PubMed Health⌕ Search

Biomedical subjects

D B Swanson

Publications and source records attributed to D B Swanson.

At least 55 records · Page 3Linked to original sources
PubMed1990

Charting the winds of change: evaluating innovative medical curricula.

The increased interest, in North America and around the world, in problem-based and community-oriented medical curricula has sparked interest in the evaluation of these innovative programs. In January 1989, the Josiah Macy Jr. Foundation sponsored a conference to consider designs for evaluation studies and the potential distinctive outcomes of the innovative curricula that might be foci of these studies. After defining an "innovative curriculum," the participants identified seven characteristics of "important evaluation studies," particularly endorsing studies that compare curricula as whole entities. The participants then identified 26 areas where differences between graduates of innovative and traditional curricula might be expected, and five equally important areas where differences are not expected. Distinctive outcomes of innovative curricula were anticipated in areas such as interpersonal skills, continuing learning, and professional satisfaction. Overall, these recommendations are offered to stimulate creative evaluations of the growing number of innovative programs in medical education.

Clinical Clerkship↗
PubMed1990

An assessment of the clinical skills of fourth-year students at four New England medical schools.

This paper describes a collaborative effort among five New England medical schools to assess important clinical skills of fourth-year medical students graduating in the class of 1988; results are presented from the four schools that provided sufficient data. Faculty from each school developed 36 patient cases representing a variety of common ambulatory-care problems. Over the course of a day, each student, on average, interacted with 16 different standardized patients, who were nonphysicians trained to accurately and consistently portray a patient in a simulated clinical setting. The students obtained focused histories, performed relevant physical examinations, and provided the patients with education and counseling. At each school, the performance of a small number of the students fell below standards set by the faculty. These deficiencies were not detected with the evaluation strategies currently being used. Although the use of standardized patients should not substitute for the process of faculty observing students as they interact with real patients, it appears that standardized patients can provide faculty with important information, not readily available from other sources, about students' performances of essential clinical activities and the levels of their clinical skills.

Ambulatory Care↗
PubMed1989

The Medical Education Evaluation Program of the state of Ohio.

In October 1988, seven foreign medical graduates participated in the first administration of the examination devised by the Medical Education Evaluation Program (MEEP) mandated by the State Medical Board of Ohio. The MEEP was established to provide an objective evaluation of an applicant's clinical competencies; passing the MEEP examination was intended to certify that the applicant's clinical skills were comparable to those of a medical student graduating from a school accredited by the Liaison Committee on Medical Education. An applicant who successfully passed the MEEP examination and fulfilled the other Ohio licensure requirements would be eligible to take the Federation Licensing Examination (FLEX) and apply for an unrestricted license to practice medicine in Ohio. The paper describes the origin and development of the MEEP examination and the testing modalities selected (multiple-choice examinations and the use of standardized patients). Four fundamental areas were tested; these are named and described, along with the method for calculating scores for each area and the criteria for passing the different components of the examination. Although the small sample size prohibited meaningful data analysis for the performance of the first group of MEEP candidates, the MEEP examination appears to meet psychometric standards of certifying and licensing examinations, based on data from comparable tests taken by beginning fourth-year medical students in New England and NBME Part III examination examinees. Some potential pitfalls of the MEEP examination are mentioned, as well as the fact that it presents a challenge to boards of medical examiners of other states to implement performance-based assessments of physicians who graduate from non-accredited medical schools.

Clinical Competence↗
PubMed1989

Training and experience of examiners.

Variation in the accuracy of examiner judgements is a source of measurement error in performance-based tests. In previous studies using doctor subjects, examiner training yielded marginal or no improvement in the accuracy of examiner judgments. This study reports an experiment on accuracy of scoring in which provision of training and background of examiners are systematically varied. Experienced teaching staff, medical students and lay subjects were randomly assigned to either training or no-training groups. Using detailed behavioural check-lists, they subsequently scored videotaped performance on two clinical cases, and accuracy of their judgments was appraised. Results indicated that the need for and effectiveness of training varied across groups: it was least needed and least effective for the teaching staff group, more needed and effective for medical students, and most needed and effective for the lay group. The accuracy of the lay group after training approached the accuracy of untrained teaching staff. Trained medical students were as accurate as trained teaching staff. For teaching staff and medical students training also influenced the nature of errors made by reducing the number of errors of commission. It was concluded that training varies in effectiveness as a function of medical experience and that trained lay persons can be utilized as examiners in performance-based tests.

Clinical Competence↗
PubMed1988

Psychometric characteristics of the objective structured clinical examination.

The objective structured clinical examination (OSCE) is increasingly being used as a method of clinical assessment yet its measurement characteristics have not been well documented. Evidence is accumulating that many OSCEs may be too short to achieve reliable results. This paper reports detailed psychometric analyses of OSCEs which were administered as part of a well-established final-year examination. Generalizability theory guided investigation of test reliability. At the present test length the OSCE components showed low reliabilities relative to written components. Satisfactory reliabilities could potentially be achieved if test length was increased to approximately 6 hours, a time which would create significant logistic problems for most medical schools. Several strategies for dealing with this practical problem have been explored. Firstly, it was shown that more careful selection of stations based on their psychometric characteristics can significantly improve reliability. Secondly, where rater availability is a limiting factor to increasing test length, more can be gained by using one rater per station and having more stations than using two raters per station. Finally, OSCE scores can, with advantage, be combined with other test scores which are obtained by using less resource-intensive methods. By adopting such strategies, a reliable assessment of clinical competence could be obtained in about 4 hours of testing time which was equally divided between an OSCE constructed of practical and clinical stations and a written test.

Clinical Competence↗
PubMed1988

Reliability (generalizability) of the Maastricht Skills Test.

This study investigated the reproducibility of scores on the University of Limburg Skills Test, using data drawn from tests administered to medical students in all years of training during 1984-85, 1985-86, and 1986-87. Generalizability analyses indicated that, while inter-rater reliability is good, a minimum of three to four hours of testing time is required to obtain reproducible scores due to content-specificity of examinee performance.

Clinical Competence↗
PubMed1988

Evaluating diagnostic pattern recognition: the psychometric characteristics of a new item format.

The issues related to measuring pattern recognition are similar to measuring other clinical skills; how can testing time be used most efficiently to obtain reliable and valid scores? How should tests be constructed to obtain scores that validly reflect individual performance in making diagnoses? Generalizability analyses indicated that performance in one topic area does not predict performance in other areas very well. For example, students who were relatively expert in diagnosing patients with headaches tend not to be expert in diagnosing patients with chest pain, joint pain, etc. Therefore, to evaluate diagnostic pattern recognition skills in general, it is preferable to sample more presenting complaints with fewer items directed at each one, rather than to sample more items within a small number of presenting complaints. Approximately one hour of testing time would be required to generate a reasonably reliable score (ie, with a generalizability coefficient greater than 0.80). For diagnostic or remedial purposes, performance can be examined by content area to determine specific areas of weakness for individual students. The next phase of this study will be directed at determining whether there are benefits to using the current matching format with a relatively long list of response alternatives rather than a traditional multiple choice format with five choices. It is hypothesized that the shorter list differentially benefits the lower ability students and the more junior students. Efforts will also be directed to determining the applicability of the item format to other content areas such as diagnostic testing and therapy.

Clinical Competence↗
PubMed1987

Ensuring the clinical competence of medical school graduates through standardized patients.

There are substantial problems with the clinical training provided to medical students and with the assessment procedure used by medical schools to ensure that students have acquired the clinical skills necessary for graduate medical education. These skills are not evaluated carefully nor systematically at any point in training or licensure. This article describes the use of standardized patients to help resolve some of these shortcomings. Standardized patients are non-physicians highly trained to function in the multiple roles of patient, teacher, and evaluator while realistically replicating a patient encounter. They are effective teachers of interviewing and physical examination skills. They can help to provide a controlled exposure to common ambulatory and difficult patient communication situations. Initial studies indicate the promise of this approach for ensuring the competence of medical school graduates.

Clinical Clerkship↗
PubMed1987

A diagnostic fourth-year performance assessment.

The University of Massachusetts Medical Center, Worcester, attempted to develop a standardized, performance-based test battery aimed directly at assessing the critical aspects of clinical competence required for graduation from medical school. The battery used a blend of standardized patient-based and written test materials and was designed to yield a profile of scores, providing a "diagnosis" of student strengths and weaknesses on a skill-by-skill basis. Results indicate that a stable, reproducible assessment of clinical skills can be achieved in a one- to two-day test battery, depending on the specific skills measured. The resulting score profile provides faculty with important information about the clinical competence of students that is not readily available from other sources, thus improving the breadth and accuracy of student assessment. A long-term goal is that performance-based testing techniques will be incorporated into the licensure process to evaluate clinical skills and ensure the competence of graduating physicians.

Academic Medical Centers↗
PubMed1986

Assessing clinical skills of residents with standardized patients.

Current techniques do not provide a reproducible, reliable, or valid basis for assessing clinical skills. The need for large-scale direct observation and standardized assessment procedures has precluded development of better techniques. A project using standardized patients presenting with common clinical problems evaluated the skills of 336 internal medicine residents at 14 New England residency programs in 1289 standardized patient and resident encounters. Results indicated that reproducible assessment of the clinical skills could be achieved in approximately 1 day of testing time using standardized patients. Resident performance improved with years of training, and senior residents and those from programs with stronger reputations performed better and were more homogeneous in ability. Low correlations between standardized-patient-based measures of clinical skills and other evaluation techniques suggested that standardized patients provided unique information. Reactions of residents and faculty to standardized-patient-based evaluations were favorable.

Clinical Competence↗
PubMed1985

Reliability, validity and efficiency of multiple choice question and patient management problem item formats in assessment of clinical competence.

Despite a lack of face validity, there continues to be heavy reliance on objective paper-and-pencil measures of clinical competence. Among these measures, the most common item formats are patient management problems (PMPs) and three types of multiple choice questions (MCQs): one-best-answer (A-types); matching questions (M-types); and multiple true/false questions (X-types). The purpose of this study is to compare the reliability, validity and efficiency of these item formats with particular focus on whether MCQs and PMPs measure different aspects of clinical competence. Analyses revealed reliabilities of 0.72 or better for all item formats; the MCQ formats were most reliable. Similarly, efficiency analyses (reliability per unit of testing time) demonstrated the superiority of MCQs. Evidence for validity obtained through correlations of both programme directors' ratings and criterion group membership with item format scores also favoured MCQs. More important, however, is whether MCQs and PMPs measure the same or different aspects of clinical competence. Regression analyses of the scores on the validity measures (programme directors' ratings and criterion group membership) indicated that MCQs and PMPs seem to be measuring predominantly the same thing. MCQs contribute a small unique variance component over and above PMPs, while PMPs make the smallest unique contribution. As a whole, these results indicate that MCQs are more efficient, reliable and valid than PMPs.

Clinical Competence↗
PubMed1984

A needs assessment in continuing professional education.

This paper describes a systematic approach to an assessment of the learning needs of practicing nurses, and a computer-based solution to marketing the resulting CE offerings. Demographic, educational and employment information was collected in a data base survey from more than five thousand professional nurses in a thirteen county region of upper New York State. Based upon three mini-surveys, a questionnaire assessing learning needs was then developed and sent to a large subpopulation of these nurses. An analysis of fifteen hundred responses to the questionnaire identified continuing nursing education priorities in the region. Responses to the data base survey and the learning needs questionnaire were merged into a computerized data base management system, which is used to select topics, design courses, and communicate with nurses in the region regarding continuing education. This approach to needs assessment, program planning and dissemination of continuing education information should be useful in solving analogous problems in other areas of continuing professional education.

Data Collection↗