SEARCH · PubMed Health

Results for “Observer Variation”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 649 records · Page 36Linked to original sources

PubMed1993

The design of observer agreement studies with binary assessments.

We discuss the design of observer agreement studies with binary assessments, with particular emphasis on the need for adequate sample size and the use of replicate observations. First, we present a method and tables for determining the sample size required for ensuring a desired precision for the estimate of the probability of disagreement between two observers. Second, for studies including replicate observations, we present a statistical model that allows estimation of the magnitude of within- and between-observer variation. We then derive sample sizes guaranteeing a specified precision for these estimates, present tables of these sample sizes and give examples of their use.

Confidence Intervals↗

PubMed2002

Brain SPECT: a controlled, blinded assessment of intra-reader and inter-reader agreement.

A detailed assessment of intra- and inter-reader variation in the interpretation of brain SPECT scans has been performed. A random sample was selected from scans performed at a community/teaching hospital in Seattle. Scans were interpreted independently by three experienced readers who were blinded to all patient information. Forty-eight scans were interpreted twice by each reader, for a total of 288 readings. Readers recorded detailed assessments of individual lesions and overall impressions using a standardized reporting form. Intra-observer agreement as reflected in per cent agreement for severity scores ranged from 65% to 100%. Intra-observer agreement on the 'overall impression' was very good for Alzheimer's pattern (kappa=0.73-1.00), and fair to good for the 'heterogeneous pattern' (kappa=0.30-0.63). Inter-observer agreement, as reflected in per cent agreement, ranged from 29% to 100%. Inter-observer agreement about the 'overall impression' was fair to moderate for Alzheimer's pattern (kappa=0.24-0.54) and was poor for the descriptors 'heterogeneous' and 'normal'. It is concluded that brain SPECT has great potential value in many important conditions. This study demonstrates a need for further work in the areas of pattern definition and reduction of observer variation.

Adult↗

PubMed1998

Staging lymphoma with CT: comparison of contiguous and alternate 10 mm slice techniques.

PURPOSE: To compare two computed tomography (CT) techniques, contiguous 10 mm and alternate 10 mm slices of the chest, abdomen and pelvis, for initial staging of lymphoma. SUBJECTS AND METHODS: Fifty-two consecutive patients referred at initial diagnosis for lymphoma staging by CT were examined with contiguous 10 mm slices of the chest, abdomen and pelvis. Oral contrast was administered, but no intravenous contrast. Two sets of films for each examination were printed, one with 10 mm contiguous slices and one with 10 mm alternate slices. The two sets of films for each patient were reviewed separately in a randomized order and blinded fashion by two independent observers. Discrepancies were reviewed by a third observer and subsequently resolved by consensus. RESULTS: Staging assessments by both techniques were concordant in 51 of 52 cases: one patient was recorded as stage II on the alternate slice technique, but stage III with contiguous slices but this difference did not affect management. Inter-technique agreement was very good (kappa=0.97). The staging assessments showed discrepancies between the two observers in 12 of the 52 cases which were independent of technique (kappa=0.71; good agreement). Consensus review showed them to result from differences in perception in seven cases and differences in interpretation of abnormalities in five cases. CONCLUSION: Our findings support the use of an alternate 10 mm slice technique in the staging of lymphoma at initial diagnosis. Observer variation was greater than the difference between techniques.

Adolescent↗

PubMed1992

Inter-observer variance in ultrasonographical assessment of Schistosoma mansoni-related morbidity in young schoolchildren.

49 Sudanese schoolchildren aged 6-9 years with Schistosoma mansoni infection were ultrasonographically examined by two independent observers in a double-blind fashion. The first observer recorded normal appearance of the liver in 23 cases, whereas the second observer recorded the appearance as normal in 33 cases. There were 23 concordant observations. For Grade I periportal fibrosis (PF), 13 observations were concordant. PF Grade II was rarely observed (2 versus 3 cases), and Grade III was not recorded at all. In total, 38 out of 49 observations were concordant (77.5%). These preliminary data from two ultrasound observers, from observations on a limited number of patients, can be seen as an indication of a potential inter-observer variation of around 20% for the distinction between the absence of PF and a low level of PF.

Child↗

PubMed1994

The tussometer: accuracy and reproducibility.

Tussometry is a new non-invasive technique for objectively assessing laryngeal function by analysis of the airflow waveform produced by a maximum effort voluntary cough manoeuvre. We describe the technique and present the calibration data. The tussometer has been calibrated for flows of up to 1100 litre min-1 using a flowmeter with a quoted accuracy of +/- 1.75%. The variables measured (cough peak flow rate (CPFR) and peak velocity time (PVT)) were found to be reproducible; the within-subject variability for CPFR was found to be 23.9% and for PVT 9%. There was no inter-observer variation. We found that the size of the mask used did not influence the readings obtained, providing an adequate seal was achieved.

Calibration↗

PubMed2004

Scintigraphic assessment of slow transit constipation with special reference to right- or left-sided colonic delay.

OBJECTIVE: Subtotal colectomy and ileorectal anastomosis for slow transit constipation has several side-effects. The motor abnormality in some patients may be segmental which could motivate a limited resection of the colon. Therefore a diagnostic tool to identify a segmental colonic motor dysfunction is needed. The aim of this study was to evaluate a scintigraphic method to assess colonic transit with special reference to right- or left-sided delay. METHODS: Twenty-three constipated patients (19 women, mean age 50 years) with slow colonic transit on radio-opaque marker studies and 13 healthy individuals (11 women, mean age 46 years) were studied. All subjects were examined with oral (111)Indium-DTPA scintigraphy. The scintigraphic results for patients and controls were presented as geometric centre of radioactivity and percent activity over time in the right, the left and the recto-sigmoid colon. The inter-observer variation in the interpretation of the scans was also evaluated. RESULTS: There was no difference in transit time between the groups of patients and controls in the right colon whereas the patients had a significant delay in the left colon (P < 0.05). Two patients had a marked delay in the right colon followed by relatively rapid transit in the left colon. The inter-observer correlation was good comparing the right, the left and the recto-sigmoid colon (r = 0.58-0.98, P < 0.01-0.001). CONCLUSION: The results indicate that colonic scintigraphy with oral (111)Indium-DTPA may help to select patients for a left or, in a few cases, a right hemicolectomy for slow transit constipation.

Adult↗

PubMed1990

Measurement techniques for melanoma: a statistical comparison.

Inter- and intra-observer variation in measuring the depth of invasion of malignant melanomas was assessed using three different techniques: eye-piece graticule, stage Vernier, and projection image analysis. Significant variation was found for all methods but was least pronounced with the stage Vernier. It is recommended that this should be the preferred technique for routine use.

Analysis of Variance↗

PubMed1996

Low-grade transitional cell carcinoma of the urinary bladder: application of select cytologic criteria to improve diagnostic accuracy [corrected].

The cytologic diagnosis of low-grade transitional cell carcinoma of the bladder is difficult, and the reported sensitivity of a positive diagnosis ranges from 0 to 73%. Using regression analysis, our laboratory previously reported the criteria of increased nuclear/cytoplasmic ratios, irregular nuclear membranes, and cytoplasmic homogeneity as indicative of low-grade transitional cell carcinoma. To examine the validity of these criteria, six observers examined 88 bladder-wash specimens (39 transitional cell carcinomas and 49 benign) and, using the selected criteria, graded each wash for the probability of malignancy. Diagnostic accuracy was measured using the receiver operating characteristic curve and the likelihood ratio. Overall observer accuracy was 76%, the sensitivity of a definitive negative diagnosis was 82%, and the specificity of a definitive positive diagnosis was 96%. We conclude that key cytologic criteria can be learned and effectively applied with high accuracy. Observer variation in diagnostic categories might reflect different confidence levels and probabilities of transitional cell carcinoma.

Carcinoma, Transitional Cell↗

PubMed1999

Accuracy of strategies for monitoring fetal heart rate in labor.

We investigated whether the accuracy of auscultation could be improved with the use of a heart rate meter. Six fetal heart rate (FHR) traces were presented in a random sequence to 30 subjects using a customized computer program in each of three modalities: auscultation by counting alone, auscultation with the aid of an FHR meter, and visual assessment. The following characteristics were assessed: baseline rate, baseline variability, periodic change, and interpretation of the trace. For baseline rate, counting was associated with consistent underestimation of the FHR, which became more evident as the heart rate increased. The variation observed with each method was greatest with counting. For baseline variability, the proportion of correct responses using a meter was comparable to visual assessment, whereas counting was significantly less effective in 4 of 6 traces. For periodic change, the use of a meter was superior to counting in 4 of 6 traces, but both were inferior to visual assessment in 4 of 6 traces. In the interpretation of the trace, the use of a meter was again superior to counting, but both were inferior to visual assessment. Discrepancies were most marked in the least reassuring traces. Assessment of the FHR is significantly more accurate with the aid of a heart rate meter, and reduces interobserver variation.

Cardiotocography↗

PubMed2006

Shade selection performed by novice dental professionals and colorimeter.

The objective of this study was to test inter-observer variability in shade selection for porcelain restorations, using three different shade guides: Vita Lumin Vacuum, Vita 3D-Master and Procera. Nineteen young dental professionals acted as observers. The results were also compared with those of a digital colorimeter (Shade Eye Ex; Shofu, Japan). Regarding repeatability, no significant differences were found between the three shade guides, although repeatability was relatively low (33-43%). Agreement with the colorimetric results was also low (8-34%). In conclusion, shade selection shows moderate to great inter-observer variation. In teaching and standardizing the shade selection procedure, a digital colorimeter may be a useful educational tool.

Adult↗

PubMed1996

Chest radiograph abnormalities in very low birthweight survivors of chronic neonatal lung disease.

OBJECTIVE: To determine whether the neonatal chest radiograph (CXR) at 28 days in very low birthweight (VLBW) infants who develop chronic neonatal lung disease (CNLD) predicts oxygen therapy duration or CXR abnormalities in early childhood. Also, to assess the inter-observer reliability of the radiologists scoring the CXR. METHODOLOGY: Clinically well survivors of CNLD (n = 46) had neonatal CXR scored (mean age 28.5 days) and compared with current CXR (mean age 40 months). The CXR were scored independently and 'blindly' by two paediatric radiologists using a standardized scoring system (range 0-10). RESULTS: There was no correlation between neonatal CXR scores and current CXR scores for either radiologist. There was no association between CXR severity scores and duration of oxygen therapy for either neonatal or current CXR. Radiologist A scored the current CXR significantly more abnormal than radiologist B [medians (range): 3 (1-6) vs 1 (0-5), P < 0.001] with reasonable correlation (r = 0.593, P < 0.005) but worse than chance agreement (kappa = -0.034). The median scores for the neonatal CXR were similar [1.5 (0-8) vs 2 (0-8), P = 0.789] and again there was good correlation (r = 0.760, P < 0.0005) although poor individual agreement (kappa = 0.243) between radiologists. CONCLUSIONS: Follow-up CXR abnormalities in VLBW infants with CNLD are usually minor and are not predictive of the duration of oxygen therapy that will be required nor of the CXR appearance in early childhood. Considerable inter-observer variation exists in the interpretation of the CXR in CNLD.

Bronchopulmonary Dysplasia↗

PubMed2000

Abnormalities on chest radiographs following radiation therapy for breast cancer.

The aim of this study was to study pulmonary radiological abnormalities with chest radiography following different radiotherapy (RT) techniques for breast cancer with respect to regions and density, and their association with pulmonary complications and reduction in vital capacity (VC). Chest radiographs were performed 5 months following local or loco-regional RT in 167 breast cancer patients. The radiological abnormalities were analysed with a classification system originally proposed by Arriagada and evaluated according to increasing density (0-3) and affected lung regions (apical-lateral, basal-lateral, central-parahilar). The highest-density grades in each region were added together to form scores ranging from 0 to 9. The patients were monitored for RT-induced pulmonary complications. The VC was measured prior to and 5 months following RT. An independent evaluation of 51 patients was performed by a second radiologist to control the reproducibility of the classification system. Increasing scores were associated with loco-regional RT and pulmonary complications (P < 0.001). The mean reduction of VC for patients scoring 0-3 (-30 ml) vs 4-9 (-161 ml) was not statistically significant (P = 0.10). Scores of 4-9 were more frequently observed in older patients (P < 0.001). The independent evaluations by two radiologists revealed good agreement (P < 0.001) and no systematic inter-observer variation. Radiological abnormalities on chest radiographs, scored according to Arriagada, can be used as an objective end point for RT-induced pulmonary side effects in breast cancer.

Breast Neoplasms↗

PubMed2002

Sampling error and intraobserver variation in liver biopsy in patients with chronic HCV infection.

OBJECTIVES: Needle liver biopsy has been shown to have a high rate of sampling error in patients with diffuse parenchymal liver diseases. In these cases, the sample of liver tissue does not reflect the true degree of inflammation, fibrosis, or cirrhosis, despite an adequate sample size. The aim of this study was to determine the rate and extent of sampling error in patients with chronic hepatitis C virus infection, and to assess the intraobserver variation with the commonly used scoring system proposed by Scheuer and modified by Batts and Ludwig. METHODS: A total of 124 patients with chronic hepatitis C virus infection underwent simultaneous laparoscopy-guided biopsies of the right and left hepatic lobes. Formalin-fixed paraffin-embedded sections were stained with hematoxylin and eosin and with trichrome. The slides were blindly coded and randomly divided among two hepatopathologists. Inflammation and fibrosis were scored according to the standard grading (inflammation) and staging (fibrosis) method based on the modified Scheuer system. Following the interpretation, the slides were uncoded to compare the results of the right and left lobes. Fifty of the samples were blindly resubmitted to each of the pathologists to determine the intraobserver variation. RESULTS: Thirty of 124 patients (24.2%) had a difference of at least one grade, and 41 of 124 patients (33.1%) had a difference of at least one stage between the right and left lobes. In 18 patients (14.5%), interpretation of cirrhosis was given in one lobe, whereas stage 3 fibrosis was given in the other. A difference of two stages or two grades was found in only three (2.4%) and two (1.6%) patients, respectively. Of the 50 samples that were examined twice, the grading by each pathologist on the second examination differed from the first examination in 0% and 4%, and the staging differed in 6% and 10%, respectively. All observed variations were of one grade or one stage. CONCLUSIONS: Liver biopsy samples taken from the right and left hepatic lobes differed in histological grading and staging in a large proportion of chronic hepatitis C virus patients; however, differences of more than one stage or grade were uncommon. A sampling error may have led to underdiagnosis of cirrhosis in 14.5% of the patients. These differences could not be attributed to intraobserver variation, which appeared to be low.

Adult↗

PubMed1996

Interrater agreement in the measurement of QRS interval in tricyclic antidepressant overdose: implications for monitoring and research.

STUDY OBJECTIVE: To assess interobserver agreement when experienced clinicians measure QRS-interval duration in tricyclic antidepressant (TCA) overdose. METHODS: We studied the admission ECGs of 231 patients with ICA poisoning. Three of the authors, with experience in the management of TCA poisoning, independently measured QRS intervals manually. Each rater was blinded to patient outcome and the measurements made by the other raters. Our main outcome measure was agreement among raters, particularly as it applies to clinically used QRS cutoff points for the determination of treatment and disposition. RESULTS: Agreement on the measurement of QRS intervals was good (intraclass correlation coefficient, 60; 95% confidence interval [CI], 53 to .66) for transformed data and .82 (95% CI, 7B to .85) for raw data. When assigning patients to categories of QRS interval, the raters agreed on 169 of 231 patients (73%) (weighted kappa = 83; P < .0001). However, the raters did not agree unanimously on whether the QRS interval was less than 100 milliseconds or 100 milliseconds or greater in 45 of 231 of patients (19.5%) (kappa = 69; P < .0001). CONCLUSION: Reliance on the manually measured QRS interval to determine management in TCA poisoning is not justified because of substantial observer variation in the determination of whether the interval falls below the clinical cutoff point of 100 milliseconds. Agreement is sufficient to make the measurements useful as part of the overall assessment of toxicity.

Antidepressive Agents, Tricyclic↗

PubMed1998

International comparisons of professional assessments in orthodontics: Part 1--Treatment need.

An international survey has been undertaken to assess variation in professional assessment of orthodontic treatment need. Ninety-seven orthodontists from nine countries examined a standard sample of 240 dental study casts to judge the need for orthodontic treatment in terms of dental health, dental esthetics, and deviation from normal. Participants also indicated their assessment of treatment complexity and whether they would recommend treatment. Practitioners' subjective assessments were found to be only moderately reliable overall. Of the cases examined, 76% [corrected] obtained a consensus of 80% or greater on the need for treatment. Logistic regression equations were derived to explain the observed variation in treatment decisions. It was found that occlusal traits from the dental casts were highly predictive of the treatment decision, and that the practitioners' country of origin and payment methods are important influences on prescribing behavior. The treatment decisions were predicted with an accuracy of 84% by using occlusal score values for the dental esthetics, the degree of upper arch crowding, the presence of crossbite, the anterior overbite, and the buccal segment sagittal relationship. It is suggested that these traits may form the basis for an internationally validated index of treatment need for use in clinical audit and orthodontic research.

Adult↗

PubMed1991

[Reproducibility of a rapid test for HIV antibody].

This study evaluated three rapid test kits for HIV antibody detection. Thirty consecutive blood specimens from the infectious diseases outpatient clinic were examined by three house officers who tested each specimen twice in a blind and randomized experiment. HIV-Chek (Du Pont) and HIV-1/HIV-2 TestPack (Abbott) were easy to handle and gave good intra- and interobserver agreement and good agreement with routine ELISA. Recombigen (BioScience) was difficult to interpret and gave poorer intra- and inter-observer variation. We suggest that HIV-Chek and TestPack can be used for assessment of patients' HIV antibody status in emergency circumstances with only minimal instruction of the staff.

AIDS Serodiagnosis↗

PubMed2001

Reassessment of lymphocytic atypia in the diagnosis of mycosis fungoides.

The manifestations of mycosis fungoides in its early stage may mimic clinically and histologically those of many benign inflammatory dermatoses. Therefore, the diagnosis of mycosis fungoides remains a major challenge for dermatologists and dermatopathologists. For many years, it has been proposed that atypical lymphocytes within the epidermis constitute one of the diagnostic features in mycosis fungoides. Presence of dermal atypical lymphocytes remains controversial as a diagnostic criterion. We reassessed the feasibility of applying lymphocytic atypia within epidermis and dermis as diagnostic criteria discriminating between mycosis fungoides and spongiotic dermatitis. Thirty cases of mycosis fungoides and 30 cases of spongiotic dermatitis were retrieved from archival hematoxylin and eosin-stained histologic sections. Punch biopsy sections were examined by light microscopy; epidermal and dermal lymphocytes were photographed at 1000x (oil immersion). A total of 92 ektachrome slides (35 mM) were developed, coded, and ordered randomly. For each slide, cells were interpreted as typical or atypical lymphocytes by seven pathologists. Atypical epidermal lymphocytes were judged to be present in 9 +/- 2 out of 16 (56%) cases of mycosis fungoides photographed as compared with 8 +/- 3 out of 16 (50%) in spongiotic dermatitis. Dermal lymphocytic atypia was thought to be present in 14 +/- 6 out of 30 (47%) patients with mycosis fungoides. Thirteen +/- 6 out of 30 (43%) patients with non-mycosis fungoides also displayed dermal lymphocytic atypia. No statistical significance was observed in these comparisons (t test, P >.05). Furthermore, atypia of lymphocytes was deemed to be present in 41, 38, 59, 70, 23, 47, and 40 out of 92 slides examined by the investigators, suggesting that observer variation is a very significant factor in our present study. We conclude that it is not possible to distinguish mycosis fungoides from spongiotic dermatitis merely based on lymphocytic atypia within epidermis or dermis.

Dermatitis↗

PubMed1997

Estimates of the sources of variation (variance components) of bioelectric impedance and anthropometric measurements in an epidemiological case-control study of breast cancer.

OBJECTIVE: This study investigated the variability of anthropometric measurements and body fat estimated by bioelectric impedance analysis. Subsequently the methods were applied in a case-control study to investigate the association with breast cancer. DESIGN, SUBJECTS: The study group included 50 consecutive cases and 75 age-matched controls from the same area. The variation was investigated in 50 healthy women from the control group, who were repeatably measured using standardised measurement procedures, and the variation between-subjects, within-subjects, between-observers, and within-observers were estimated. RESULTS, CONCLUSIONS: The study showed that the variance components between-subjects were 64-99% of the total variance. The variables of skinfold thicknesses were characterised by having the highest relative observer variation and having many unavailable values that were out of the range of the Harpenden callipers. The mean body fat by bioelectric impedance analysis was 31.2%, and the total coefficient of variation 23%, while the variance components related to subject time, observer and measurement were 98.4%, 1.1%, 0.2%, and 0.4%, respectively. The body fat was significantly correlated with the variables of skinfold thicknesses. We decided to exclude the variables of skinfold thicknesses from the case-control study, and for the other variables to measure each subject only at one time by one observer. The case-control part of the study indicated a non-significant increase in body weight in the postmenopausal breast cancer patients (mean difference 3.6 kg; confidence interval from -0.9 kg to 8.0 kg). Similarly the body fat tended to be higher in the breast cancer patients (mean difference 1.2%; confidence interval from -1.6% to 4.0%).

Adult↗