A practice oral examination rating scale--inter-observer reliability.

W O McCormick

doi:10.1177/070674378102600407

Home/Archive/PubMed

PubMed · 7296435

A practice oral examination rating scale--inter-observer reliability.

W O McCormick

View original source ↗Export metadata ↓

Abstract

Three videotaped practice clinical, oral examinations were rated by 9, 9 and 11 raters respectively. Raters were all certified specialists engaged in training residents for the certification examinations. In two of the simulated examinations considerable differences in rating scores, to the extent of pass/fail disagreement, were found. The significance of the findings, including the possibility that one examiner may "contaminate" another, is discussed. Further work is essential to develop reliable instruments for rating certification examinations, whatever their format, as the Royal College policies evolve.

Explore related subjects

Canada Certification Humans Internship and Residency Interview, Psychological Psychiatry Specialty Boards Videotape Recording Journal Article

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

W O McCormick. 1981. A practice oral examination rating scale--inter-observer reliability.. https://doi.org/10.1177/070674378102600407

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related citations

PubMed2026

Investing in Canada's nursing workforce: a comprehensive review to inform policy innovations and directions.

BACKGROUND: Health systems worldwide face persistent health workers challenges including nursing shortages, workforce strain, and inequities. In Canada, these challenges have prompted renewed national and provincial reforms to strengthen recruitment, retention, leadership, and sustainability. This paper compares nursing workforce policy directions across Canada, and international jurisdictions to inform policy and planning. METHODS: A cross-country comparative analysis of policies building on a comprehensive national funded review that included an umbrella review of 69 systematic reviews, a comparative policy review of nursing workforce strategies in five jurisdictions, and validation through national horizon-scanning and policy dialogues (n >100). Evidence was analyzed across system, organizational, and individual levels. RESULTS: At the system level, international jurisdictions demonstrate comprehensive, legislated approaches integrating data, governance, and multi-year funding have advanced key nursing strategies. In Canada, the advances show the importance of strategies to have national and provincial/territorial alignment emphasizing leadership, flexibility, and inclusion as key levers. Organizational and individual-level reforms such as mentorship, leadership development, and wellness initiatives are expanding but remain variably evaluated. Experts identified national workforce data strategies and policy integration with embedded evaluation as key enablers to inform scalability and sustainability of implemented strategies. CONCLUSIONS: Canada's nursing workforce reforms are advancing toward coordinated, equity-driven, and evidence-informed strategies. Continued investment in evaluation, leadership, and national integrated data systems along with integrating nursing workforce planning within broader intersectoral planning will consolidate these gains and position Canada as an international leader in sustainable nursing workforce policy.

Canada↗

PubMed2007

A likelihood approach to estimating sensitivity and specificity for binocular data: application in ophthalmology.

Binocular data typically arise in ophthalmology where pairs of eyes are evaluated, through some diagnostic procedure, for the presence of certain diseases or pathologies. Treating eyes as independent and adopting the usual approach in estimating the sensitivity and specificity of a diagnostic test ignores the correlation between eyes. This may consequently yield incorrect estimates, especially of the standard errors. The paper proposes a likelihood-based method of accounting for the correlations between eyes and estimating sensitivity and specificity using a model for binocular or paired binary outcomes. Estimation of model parameters via maximum likelihood is outlined and approximate tests are provided. The efficiency of the estimates is assessed in a simulation study. An extension of the methodology to the case of several diagnostic tests, or the same test measured on several occasions, which arises in multi-reader studies, is given. A further extension to the case of multiple diseases is outlined as well. Data from a study on diabetic retinopathy are analysed to illustrate the methodology.

Canada↗

PubMed2007

Accounting for expected attrition in the planning of community intervention trials.

Trials in which intact communities are the units of randomization are increasingly being used to evaluate interventions which are more naturally administered at the community level, or when there is a substantial risk of treatment contamination. In this article we focus on the planning of community intervention trials in which k communities (for example, medical practices, worksites, or villages) are to be randomly allocated to each of an intervention and a control group, and fixed cohorts of m individuals enrolled in each community prior to randomization. Formulas to determine k or m may be obtained by adjusting standard sample size formulas to account for the intracluster correlation coefficient rho. In the presence of individual-level attrition however, observed cohort sizes are likely to vary. We show that conventional approaches of accounting for potential attrition, such as dividing standard sample size formulas by the anticipated follow-up rate pi or using the average anticipated cohort size m pi, may, respectively, overestimate or underestimate the required sample size when cluster follow-up rates are highly variable, and m or rho are large. We present new sample size estimation formulas for the comparison of two means or two proportions, which appropriately account for variation among cluster follow-up rates. These formulas are derived by specifying a model for the binary missingness indicators under the population-averaged approach, assuming an exchangeable intracluster correlation coefficient, denoted by tau. To aid in the planning of future trials, we recommend that estimates for tau be reported in published community intervention trials.

Canada↗