Biomedical subjects

Dale J Carlson

Publications and source records attributed to Dale J Carlson.

4 recordsLinked to original sources

PubMed2006

Developing a valid evaluation for interpersonal and communication skills.

OBJECTIVES: The Accreditation Council for Graduate Medical Education requires that residency programs evaluate the acquisition of six general competencies, including Interpersonal and Communication Skills (ICS). To develop a 360-degree method for accomplishing this, a semantic-differentiation matrix for various communication traits for nurses to evaluate physician ICS was developed. The authors sought to determine whether this evaluation method could discriminate between more experienced medical communicators (faculty) and less experienced medical communicators (residents). METHODS: A 98-item questionnaire measured several communication dimensions by using an eight-element semantic-differentiation scale. In addition, global assessment ratings assessed nursing perceptions of physician ICS skills. This process was repeated for various clinical scenarios. RESULTS: There were 26 nurse evaluators, 19 emergency medicine (EM) residents (EM2 and EM3), and 30 EM faculty. Each physician received five independent evaluations (total, 245 evaluations). All EM residents (EM2 and EM3) were compared with the EM faculty. All eight items on the semantic-differentiation scale were compared. Likewise, the global assessment scores were compared. In every category, the faculty scored higher (Mann-Whitney U: p < 0.001). CONCLUSIONS: An evaluation process integrating a semantic-differentiation matrix was applied to various clinical scenarios (as well as global assessment items) and demonstrated discriminatory results. Faculty physicians scored significantly higher than resident physicians. The ability to provide discriminatory results is a requisite in the development of a valid evaluation process. The described semantic-differentiation matrix and global assessment questions may be valid measurements of ICS.

Communication↗

PubMed2004

Quantitative validation of a general competency composite assessment evaluation.

OBJECTIVES: The authors sought to modify and validate a composite assessment evaluation process that assesses resident acquisition of the Accreditation Council for Graduate Medical Education (ACGME) general competencies (GCs). METHODS: This study critically analyzed the evaluation process used in a multicenter study (150 emergency medicine resident evaluations) to determine whether the procedure was psychometrically valid. For each GC, principal component analysis (PCA) was used to determine whether certain evaluation items could be eliminated, as well as to determine the magnitude of variability explained by up to three linear combinations or "principal components." The factor proportions (factor loadings) of various eigenvectors were measured to determine the degree of variability (determined by the square of the factor proportion) within a data or item set. The factor proportions essentially measure the length of the eigenvector as determined from a correlation matrix. RESULTS: The first three principal components are reported as factor proportion sum (% of total variability) as follows: patient care 0.91 (83%), medical knowledge 0.87 (76%), practice-based learning and improvement 0.90 (81%), interpersonal and communication skills 0.84 (71%), professionalism 0.74 (55%), and systems-based practice 0.80 (64%). PCA showed that evaluating certain traditional categories such as medical knowledge seemed to capture a single element, whereas professionalism appeared to measure a more complex, multidimensional phenomenon. CONCLUSIONS: By using a structured development process, the authors were able to create valid evaluation items for determining resident acquisition of the ACGME GCs.

Clinical Competence↗

PubMed2002

Using standardized oral examinations to evaluate general competencies.

Emergency medicine residency programs are required by the Accreditation Council for Graduate Medical Education (ACGME) to formally evaluate each resident with oral and written examinations. The Michigan State University Emergency Medicine Residency Program in Lansing conducts monthly standardized oral examinations (SOEs) as part of each resident's evaluation. Recently, the ACGME has advanced six areas, termed "general competencies," that should be acquired during graduate medical education. According to the ACGME, these competencies should be included in the educational process of all residency programs. In promulgating these competencies, the ACGME did not provide examples of core content, strategies for implementation, or methods of evaluation; rather, individual residency programs are required to develop their own methods. The authors describe a modification of an existing SOE strategy that assesses residents' knowledge, skills, experiences, and attitudes as reflected in the general competencies.

Educational Measurement↗

PubMed2002

Evaluating systems-based practice in emergency medicine.

The Accreditation Council for Graduate Medical Education has required that training programs initiate an evaluation process to assess resident acquisition of the newly promulgated general competencies (GCs). Certain GCs (e.g., systems-based practice, problem-based learning and improvement) are somewhat more challenging to define and measure than others. Systems-based practice essentially captures the interactions of the emergency medicine resident that expand beyond isolated contact with the patient. Evaluating these various interactions is readily accomplished using a detailed ordinal evaluation form that measures commonly occurring easily identified actions. Examples of measurable items and the method by which they can be integrated into an evaluation device are presented.

Educational Measurement↗