PubMed Health⌕ Search

Biomedical subjects

Reed G Williams

Publications and source records attributed to Reed G Williams.

18 recordsLinked to original sources
PubMed2006

Surgeon communication behaviors that lead patients to not recommend the surgeon to family members or friends: Analysis and impact.

BACKGROUND: This study analyzes specific elements of physician communication that lead patients to not recommend surgeons to family members or friends (FMoFs). METHODS: Patients completed questionnaires after surgery clinic encounters. Questionnaires addressed whether surgeons used optimal communication behaviors and whether patients would recommend the surgeon. RESULTS: A total of 1,514 questionnaires were completed for 39 surgeons. Patients reported the following communication lapses: failure to ask whether the patient had questions (6.9% of occasions), failure to sit down (6.5%), use of words patients could not understand (5%), failure to educate patients about their condition (4.3%), failure to introduce themselves (4%), lack of interest in patients as persons (2.4%), and inadequacies in answering questions (2%). Surgeons omitted at least one of these optimal behaviors in 16.3% of encounters. Surgeons were not recommended in 1.7% of encounters. Twelve surgeons (31%) were not recommended on at least 1 occasion. Behaviors omitted most commonly in encounters where patients wouldn't recommend surgeons included failure to show interest in the patient (52%), explain their medical condition (52%), invite questions (40%), and answer questions (36%). CONCLUSIONS: Extrapolating these results to 1,618 patient visits/surgeon/year, results in the following number of patients annually who do not recommend their surgeons: 15 for failure to adequately explain their medical condition, 15 for failure to show interest in them, 11 for failure to ask if the patient had questions, and 10 for failure to answer questions. Considering the ripple effect due to the number of a patient's FMoFs, surgeons should be aware of the significant impact of even occasional lapses in optimal communication behaviors.

Adolescent↗
PubMed2006

See one, do one, teach one--exploring the core teaching beliefs of medical school faculty.

This paper explores the core teaching beliefs of medical school faculty and establishes whether these beliefs differ among basic science, clinical, and instruction specialist faculty. One hundred and twenty-five medical school teachers who were members of professional organizations dedicated to the improvement of medical school teaching completed a Q-sort of 56 statements reflecting their core teaching beliefs. The statements described beliefs about motivation, knowledge and skill acquisition, retention, feedback, transfer, teacher characteristics, and teaching strategies. Q-sorts were completed by 37 basic scientists (30% of respondents), 59 clinicians (47%) and 29 instruction specialists (23%) working in medical schools. Fifty-two participants were classroom teachers (42%), 66 were classroom and clinical teachers (53%), and seven reported that they do not teach (6%). The Q-sort results indicate how medical school faculty members differ in their core beliefs about teaching and learning. Thirty-two respondents (26%) focused on the student as a person first. Eight (6%) were content oriented. Thirty-four (27%) were performance oriented; their focus was on having students learn and apply knowledge and skills to accomplish clinical tasks. Fifty-one respondents (41%) were found to have a blend of these viewpoints. Respondents' type of training or type of teaching did not provide a reliable indication of core teaching beliefs classification.

Culture↗
PubMed2006

An investigation of medical student reactions to feedback: a randomised controlled trial.

BACKGROUND: Medical educators have indicated that feedback is one of the main catalysts required for performance improvement. However, medical students appear to be persistently dissatisfied with the feedback that they receive. The purpose of this study was to evaluate learning outcomes and perceptions in students who received feedback compared to those who received general compliments. METHODS: All subjects received identical instruction on two-handed surgical knot-tying. Group 1 received specific, constructive feedback on how to improve their knot-tying skill. Group 2 received only general compliments. Performance was videotaped before and after instruction and after feedback. Subjects completed the study by indicating their global level of satisfaction. Three faculty evaluators observed and scored blinded videotapes of each performance. Intra-observer agreement among expert ratings of performance was calculated using 2-way random effects intraclass correlation (ICC) methods. Satisfaction scores and performance scores were compared using paired samples t-tests and independent samples t-tests. RESULTS: Performance data from 33 subjects were analysed. Inter-rater reliability exceeded 0.8 for ratings of pre-test, pre-intervention and post-intervention performances. The average performance of students who received specific feedback improved (21.98 versus 15.87, P<0.001), whereas there was no significant change in the performance score in the group who received only compliments (17.00 versus 15.39, P=0.181) The average satisfaction rating in the group that received compliments was significantly higher than the group that received feedback (6.00 versus 5.00, P=0.005). DISCUSSION: Student satisfaction is not an accurate measure of the quality of feedback. It appears that satisfaction ratings respond to praise more than feedback, while learning is more a function of feedback.

Adult↗
PubMed2005

Assuring the reliability of resident performance appraisals: more items or more observations?

BACKGROUND: The tendency to add items to resident performance rating forms has accelerated due to new ACGME competency requirements. This study addresses the relative merits of adding items versus increasing number of observations. The specific questions addressed are (1) what is the reliability of single items used to assess resident performance, (2) what effect does adding items have on reliability, and (3) how many observations are required to obtain reliable resident performance ratings. METHODS: Surgeon ratings of resident performance were collected for 3 years. The rating instrument had 3 single items representing clinical performance, professional behavior, and comparisons to other house staff. Reliability analyses were performed separately for each year, and variance components were pooled across years to compute overall reliability coefficients. RESULTS: Single-item resident performance rating scales were equivalent to multiple-item scales using conventional reliability standards. Increasing the number of rating items had little effect on reliability. Increasing the number of observations had a much larger effect. CONCLUSIONS: Program directors should focus on increasing the number of observations per resident to improve performance sampling and reliability of assessment. Increasing the number of rating items had little effect on reliability and is unlikely to assess new ACGME competencies adequately.

Clinical Competence↗
PubMed2005

Feasibility, reliability and validity of an operative performance rating system for evaluating surgery residents.

BACKGROUND: Resident evaluation traditionally involves global assessments including clinical performance, professional behavior, technical skill, and number of procedures performed. These evaluations lack objective assessment of operative skills. We describe an operative performance rating system (OPRS) designed to provide objective operative performance ratings using a sentinel procedure format. METHODS: Ten-item procedure-specific rating instruments were developed. Items included technical skills, operative decision making, and general items. A 1 to 5 (5 = excellent) scale was used for evaluation. Six procedures had sufficient forms returned to allow evaluation. Inter-rater reliability was determined by having faculty evaluators view 2 videotaped operations. RESULTS: Return rates for the Internet-based form were full-time faculty (92%), volunteer faculty (27%), and overall (67%). Reliability, (average interitem correlation), and total procedures evaluated were excisional biopsy, 0.90, (0.48), 77; open inguinal herniorraphy, 0.94, (0.62), 51; laparoscopic cholecystectomy, 0.95, (0.64), 75; small-bowel and colon resection, 0.92, (0.58), 30; parathyroidectomy, 0.70, (0.19), 30; and lumpectomy, 0.92, (0.51), 38. Years of training accounted for 25% to 57% of the variation in scores. Inter-rater variability was observed; however, the average rater agreement was reliable. CONCLUSIONS: Internet-based management made obtaining the data feasible. The OPRS complements traditional evaluations by providing objective assessment of operative decision-making and technical skills. Interitem correlations indicate the average rating of items provides a reliable indicator of resident performance. The OPRS is useful in tracking resident development throughout postgraduate training and offers a structured means of certifying operative skills.

Biopsy↗
PubMed2005

Forecasting residents' performance--partly cloudy.

The authors offer a practical guide for improving the appraisal of a resident's performance. They identify six major factors that compromise the process of observing, measuring, and characterizing a resident's current performance, forecasting future performance, and making decisions about the resident's progress. Factors that compromise any of these steps lead to individual and collective uncertainty and decrease faculty confidence when making decisions on a resident's progress. The six factors, addressed in order of importance, are inaccuracies due to (1) incomplete sampling of performance, (2) rater memory constraints, (3) hidden performance deficits of the resident, (4) lack of performance benchmarks, (5) faculty members' hesitancy to act on negative performance information, and (6) systematic rater error. The description of each factor is followed by a number of specific suggestions on what residency programs can do to eliminate or minimize the impact of these factors. While this article is couched in the context of the performance evaluation of residents, everything included pertains to measuring and appraising medical students' and practicing physicians' clinical performance as well.

Academic Medical Centers↗
PubMed2005

Improving resident performance assessment data: numeric precision and narrative specificity.

PURPOSE: To evaluate the use of a systems approach for diagnosing performance assessment problems in surgery residencies, and intervene to improve the numeric precision of global rating scores and the behavioral specificity of narrative comments. METHOD: Faculty and residents at two surgery programs participated in parallel before- and-after trials. During the baseline year, quality assurance data were gathered and problems were identified. During two subsequent intervention years, an educational specialist at each program intervened with an organizational change strategy to improve information feedback loops. Three quality-assurance measures were analyzed: (1) percentage return rate of forms, (2) generalizability coefficients and 95% confidence intervals of scores, and (3) percentage of forms with behaviorally specific narrative comments. RESULTS: Median return rates of forms increased significantly from baseline to intervention Year 1 at Site A (71% to 100%) and Site B (75% to 100%), and then remained stable during Year 2. Generalizability coefficients increased between baseline and intervention Year 1 at Site A (0.65 to 0.85) and Site B (0.58 to 0.79), and then remained stable. The 95% confidence interval around resident mean scores improved at Site A from baseline to intervention Year 1 (0.78 to 0.58) and then remained stable; at Site B, it remained constant throughout (0.55 to 0.56). The median percentage of forms with behaviorally specific narrative comments at Site A increased significantly from baseline to intervention Years 1 and 2 (50%, 57%, 82%); at Site B, the percentage increased significantly in intervention Year 1, and then remained constant (50%, 60%, 67%). CONCLUSIONS: Diagnosing performance assessment system problems and improving information feedback loops improved the quality of resident performance assessment data at both programs.

Adult↗
PubMed2005

The effects of group dynamics on resident progress committee deliberations.

BACKGROUND: Prior investigations suggest that resident progress decisions by committee provide a broader perspective on performance and result in less grade inflation. There is countervailing concern that group dynamics may compromise quality of progress decisions. PURPOSE: To determine whether and how group dynamics compromise decision making about resident progress. METHODS: Researchers recorded and analyzed participant comments during a resident progress committee meeting. End-of-rotation (EOR) evaluations were analyzed and compared to progress committee meeting results. RESULTS: EOR and progress committee comments were similar in content. The ratio of specific to general comments was higher for EOR evaluations (5:1) than for progress committee meetings (2:1). EOR evaluation comments provided more supporting evidence for assertions. Individual progress committee participants did not dominate discussion or sway decision making. Participant progress committee meeting comments were consistent with their EOR comments failing to support the presence of progress committee meeting "feeding frenzies." CONCLUSIONS: Results suggest that progress committee meeting group dynamics do not seriously compromise the validity of resident progress decisions.

Employee Performance Appraisal↗
PubMed2004

Variables influencing medical student learning in the operating room.

BACKGROUND: The operating room (OR) is an important venue where surgeons do much of medical student teaching and yet there has been little work evaluating variables that influence learning in this unique environment. We designed this study to identify variables that affected medical student learning in the OR. METHODS: We developed a questionnaire based on surgery faculty observations of learning in the OR. The medical students completed the questionnaire on 114 learning episodes in the OR. Pearson correlation coefficient was used to establish the strength of association between various variables and the student's overall perception of learning. RESULTS: The students evaluated 27 variables that might impact their learning in the OR. Strong correlations were identified between the attending physician's attitude, interactions and teaching ability in the OR and the environment being conducive to learning. CONCLUSIONS: Surgical faculty behavior is a powerful determinant of student perceptions of what provides for a favorable learning environment in the OR.

Behavior↗
PubMed2004

Prognostic value of resident clinical performance ratings.

BACKGROUND: This study investigated the concurrent and predictive validity of end-of-rotation (EOR) clinical performance ratings. STUDY DESIGN: Surgeon EOR ratings of residents were collected and compared with end-of-year (EOY) progress decisions and to EOR and EOY confidential judgments of resident ability to provide patient care without direct supervision. RESULTS: Eighty percent to 85% of EOR ratings were Excellent or Very Good. Five percent or fewer were Fair or Poor. Almost all residents receiving Excellent or Very Good EOR ratings also received positive EOR judgments about ability to provide patient care without direct supervision. Residents rated Fair or Poor received negative EOR judgments about ability to provide patient care without direct supervision. As the cumulative percent of Good, Fair, and Poor EOR ratings increased, the number of residents promoted without stipulations at the end of the year decreased and the percentage of faculty members who judged the residents capable of providing effective patient care without direct supervision at the end of the year declined. All residents receiving 40% or more EOR ratings below Very Good had stipulations associated with their promotion. CONCLUSIONS: Despite use of descriptive anchors on the scale, clinical performance ratings have no direct meaning. Their meaning needs to be established in the same manner as is done in setting normal values for diagnostic tests, ie, by establishing the relationship between EOR ratings and practice outcomes.

Clinical Competence↗
PubMed2004

Do individual attendings' post-rotation performance ratings detect residents' clinical performance deficiencies?

PURPOSE: To determine whether attending physicians' post-rotation performance ratings and written comments detect surgery residents' clinical performance deficits. METHOD: Residents' performance records from 1997-2002 in the Department of Surgery, Southern Illinois University School of Medicine, were reviewed to determine the percentage of times end-of-rotation performance ratings and/or comments detected deficiencies leading to negative end-of-year progress decisions. RESULTS: Thirteen of 1,986 individual post-rotation ratings (0.7%) nominally noted a deficit. Post-rotation ratings of "good" or below were predictive of negative end-of-year progress decisions. Eighteen percent of residents determined to have some deficiency requiring remediation received no post-rotation performance ratings indicating that deficiency. Written comments on post-rotation evaluation forms detected deficits more accurately than did numeric ratings. Physicians detected technical skills performance deficits more frequently than applied knowledge and professional behavior deficits. More physicians' post-rotation numeric ratings contradicted performance deficits than supported them. More written comments supported deficits than contradicted them in the technical skills area. In the applied knowledge and professional behavior areas, more written comments contradicted deficits than supported them. CONCLUSIONS: A large percentage of performance deficiencies only became apparent when the attending physicians discussed performance at the annual evaluation meetings. Annual evaluation meetings may (1) make patterns of residents' behavior apparent that were not previously apparent to individual physicians, (2) provide evidence that strengthens the individual attending's preexisting convictions about residents' performance deficiencies, or (3) lead to erroneous conclusions. The authors believe deficiencies were real and that their findings can be explained by a combination of reasons one and two.

Clinical Competence↗
PubMed2003

Who are the surgery clerkship directors and what are their educational needs?

BACKGROUND: The surgery clerkship director is a key individual in the surgery department's educational mission and yet there has been no prior effort to describe this group or identify their learning needs. The purpose of this study was to develop a demographic profile and an educational needs assessment for surgery clerkship directors. METHODS: A survey instrument was designed based on existing literature and distributed to surgery clerkship directors in the United States and Canada. RESULTS: Surveys were returned from 108 subjects (77%). The majority of clerkship directors strongly agree that directing is a positive experience but express concern that the job demands may impede their professional careers. The perceived educational needs identified related primarily to the development and management of the student education curriculum. CONCLUSIONS: Surgery clerkship directors are experienced academic surgeons who report high levels of satisfaction. They identify a number of important educational needs of the position and express concern about the requirements of the position on their academic careers.

Adult↗
PubMed2003

Self-assessment in simulation-based surgical skills training.

BACKGROUND: Simulation-based training provides minimal feedback and relies heavily on self-assessment. Research has shown medical trainees are poor self-assessors. The purpose of this study was to examine trainees' ability to self-assess technical skills using a simulation-trainer. METHODS: Twenty-one medical students performed 10 repetitions of a simulated task. After each repetition they estimated their time and errors made. These were compared with the simulator data. RESULTS: Task time (P < 0.0001) and errors made (P < 0.0001) improved with repetition. Both self-assessment curves reflected their actual performance curves (P < 0.0001). Self-assessment of time did not improve in accuracy (P = 0.26) but error estimation did (P = 0.01) when compared with actual performance. CONCLUSIONS: Novices demonstrated improved skill acquisition using simulation. Their estimates of performance and accuracy of error estimation improved with repetition. Clearly, practice enhances technical skill self-assessment. These results support the notion of self-directed skills training and could have significant implications for residency training programs.

Clinical Competence↗
PubMed2003

Addressing the new competencies for residents' surgical training.

In July 2001 the Accreditation Council for Graduate Medical Education (ACGME) charged U.S. residency training programs to implement a curriculum and evaluation plan covering six competencies. The authors describe the curriculum and evaluation strategy of the first surgical training program developed to meet the competencies, and list each competency and the teaching method and measurement instruments used. Implementation began July 1, 2001, and the program was fully operational on July 1, 2002. Meeting the curriculum challenges required modification of the existing curriculum and the addition of new instructional units. Nine additional evaluation instruments were needed. The largest investment was in planning and implementation, a one-time development cost. Staff workload increased by 252 hours; this is expected to be a continuing annual requirement. Faculty workload increased by two hours per resident and each resident's workload increased by 112 hours per year (2.3 hours per week). The transition was smoother than expected. Faculty and residents' buy-in was crucial. Faculty and residents were alerted to upcoming changes at the beginning of the year in a grand rounds presentation on the ACGME competencies and the approach to meeting requirements. Updates were presented periodically. The authors recommend that residency programs engaged in similar efforts make effective use of instruments developed elsewhere and collaborate with other programs rather than develop everything locally. The program's benefits include time savings and the availability of validity data and norms to inform decision making on residents' and program progress.

Clinical Competence↗
PubMed2003

The impact of a program for systematically recognizing and rewarding academic performance.

PURPOSE: To describe an academic performance incentive system (APIS) and faculty perception of it; explore the impacts of incentive level, faculty rank, clinical practice volume, and administrative responsibility on academic productivity; and describe the APIS's use in maintaining congruence between department mission and activities. METHOD: A list of teaching, research, and academic service activities was developed, which full-time faculty (n = 33) used to report activities. Clinical faculty members received incentive income based on credits earned. APIS initially distributed 1% of practice plan receipts (subsequently increased to 3% and then 5%). Productivity was measured by differences in APIS points achieved. Satisfaction of all faculty participants was measured by survey. RESULTS: Faculty members (n = 20) who participated throughout averaged 22 credits per month (nine to 42 credits), and quarterly incentive bonuses ranged from 145 US dollars to 6,128 US dollars. Average credits earned per month were 24 for the 1% incentive, 23 for the 3% incentive, and 20 for the 5% incentive. Faculty members with administrative responsibilities were as productive academically as were their non-administrative counterparts. Senior faculty members were as productive as junior faculty. Faculty members who were more productive clinically were more productive academically. Seventy percent of respondents reported they were either very satisfied or somewhat satisfied with the APIS. Seventy-eight percent felt that the APIS accurately reflected their academic productivity. Most respondents (81%) felt that the amount of money allocated to the incentive system was appropriate (15% felt it should be increased and one respondent recommended reduction). CONCLUSIONS: The APIS system has been well accepted by faculty and allows for data-driven discussion of the department's mission and activities.

Efficiency↗
PubMed2003

Cognitive, social and environmental sources of bias in clinical performance ratings.

BACKGROUND: Global ratings based on observing convenience samples of clinical performance form the primary basis for appraising the clinical competence of medical students, residents, and practicing physicians. This review explores cognitive, social, and environmental factors that contribute unwanted sources of score variation (bias) to clinical performance evaluations. SUMMARY: Raters have a 1 or 2-dimensional conception of clinical performance and do not recall details. Good news is reported more quickly and fully than bad news, leading to overly generous performance evaluations. Training has little impact on accuracy and reproducibility of clinical performance ratings. CONCLUSIONS: Clinical performance evaluation systems should assure broad, systematic sampling of clinical situations; keep rating instruments short; encourage immediate feedback for teaching and learning purposes; encourage maintenance of written performance notes to support delayed clinical performance ratings; give raters feedback about their ratings; supplement formal with unobtrusive observation; make promotion decisions via group review; supplement traditional observation with other clinical skills measures (e.g., Objective Structured Clinical Examination); encourage rating of specific performances rather than global ratings; and establish the meaning of ratings in the manner used to set normal limits for clinical diagnostic investigations.

Clinical Competence↗