SEARCH · PubMed Health

Results for “Reliability”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2Linked to original sources

PubMedDate not supplied

Validity and reliability of nursing workload measurement systems: review of validity and reliability theory.

Nursing workload measurement systems (WMSs) are used in inpatient and outpatient settings for staffing, scheduling, and budgeting. The nurse administrator can use WMS data to make wise decisions in these key areas providing the data are reliable and valid. Unfortunately, in most institutions, attention to issues of reliability and validity occurs only at system implementation and then the systems are left unattended. This article provides an overview of validity and reliability as it relates to WMSs. Part Two of this article will demonstrate how validity and reliability theory can be operationalized in an ongoing program for maintaining WMS reliability and validity.

Humans↗

PubMed1995

Reliability of reliability coefficients in the estimation of asymmetry.

Although promising to provide insight into the interaction between genotype and environment, investigations into fluctuating asymmetry suffer from a lack of standardization in the reporting of measurement error. In the present paper we show, using both anthropometric and odontometric data, that the use of the reliability coefficient calculated for a bilateral measurement provides no indication of the reliability of the corresponding asymmetry estimate, because reliability of asymmetry depends on the relationship between measurement error and the difference between sides. Thus, we suggest that future investigations either provide reliability coefficients for asymmetry estimates specifically, or use methods that account for measurement error.

Anthropometry↗

PubMed2003

Reliability of a standardized reading chart system: variance component analysis, test-retest and inter-chart reliability.

BACKGROUND: The purpose of this study was to analyze statistically the test-retest and inter-chart reliability of three standardized Radner Reading Charts with respect to reading acuity, maximum reading speed, critical print size and LogRAD/LogMAR ratio and to quantitatively identify the sources of variability. METHODS: At two testing sessions (3 to 4 weeks apart), 36 volunteers read monocularly all three Radner Reading Charts in randomized order (orthogonal Latin square design). The subjects were grouped according to their distance LogMAR visual acuity (group 1, 0.1 or better; group 2, 0.12-0.4; group 3, 0.42-0.8). Based on these 216 reading tests, crude correlations were calculated between the two testing sessions and between the three charts, and "variance component analyses" were performed for all variables. RESULTS: For all visual acuity groups and variables, the test-retest and the inter-chart reliability was high. The variance component analyses showed that the differences between individuals predominantly influenced the variance, whereas the effect of the test replication was low. The three charts caused minimal variability, indicating equality for clinical examinations. CONCLUSION: The standardized Radner Reading Charts provide clinically reliable and reproducible results for individuals with normal eyesight and for patients with visual impairment. These findings indicate that reading test systems, which consider the current international standards for visual acuity measurements (EN ISO 8596, NAS-NRC) and the psychophysical requirements for controlling optical item interactions, can provide reliable measures for clinical and scientific analyses of reading performance.

Aged↗

PubMed2006

The California Verbal Learning Test--second edition: test-retest reliability, practice effects, and reliable change indices for the standard and alternate forms.

The California Verbal Learning Test - second edition (CVLT-II) is one of the most widely used neuropsychological tests in North America. The present study evaluated the 1-month test-retest reliability and practice effects associated with the standard and alternate forms of the CVLT-II in a sample of 195 healthy adults. Eighty participants underwent repeat assessment using the standard form of the CVLT-II on both occasions, whereas the remaining 115 individuals received the standard form at baseline and the alternate form at follow-up. Consistent with prior research, results revealed generally large test-retest correlation coefficients for the primary CVLT-II measures in both the standard/standard (range=0.80-0.84) and standard/alternate (range=0.61-0.73) cohorts. Despite exhibiting slightly lower test-retest reliability coefficients, participants in the alternate form group displayed notably smaller practice effects (Cohen's d range=-0.01 to 0.18) on the primary indices relative to individuals who received the standard form on both occasions (Cohen's d range=0.27-0.61). Reliable change indices were also generated and applied to primary CVLT-II variables to determine the base rates of significant improvements (range=2-10%), declines (range=0-7%), and stability (range=85-97%) in performance over time. Overall, findings from this study support the test-retest reliability of the standard and alternate forms of the CVLT-II in healthy adults and may enhance the usefulness of this test in longitudinal neuropsychological evaluations.

Adolescent↗

PubMed2001

Increasing the reliability of the fluid/crystallized difference score from the Kaufman Adolescent and Adult Intelligence Test with reliable component analysis.

The unreliability of difference scores is a well documented phenomenon in the social sciences and has led researchers and practitioners to interpret differences cautiously, if at all. In the case of the Kaufman Adult and Adolescent Intelligence Test (KAIT), the unreliability of the difference between the Fluid IQ and the Crystallized IQ is due to the high correlation between the two scales. The consequences of the lack of precision with which differences are identified are wide confidence intervals and unpowerful significance tests (i.e., large differences are required to be declared statistically significant). Reliable component analysis (RCA) was performed on the subtests of the KAIT in order to address these problems. RCA is a new data reduction technique that results in uncorrelated component scores with maximum proportions of reliable variance. Results indicate that the scores defined by RCA have discriminant and convergent validity (with respect to the equally weighted scores) and that differences between the scores, derived from a single testing session, were more reliable than differences derived from equal weighting for each age group (11-14 years, 15-34 years, 35-85+ years). This reliability advantage results in narrower confidence intervals around difference scores and smaller differences required for statistical significance.

Age Distribution↗

PubMed2001

Study of interlaboratory reliability and reproducibility of estrogen and progesterone receptor assays in Europe. Documentation of poor reliability and identification of insufficient microwave antigen retrieval time as a major contributory element of unreliable assays.

Immunohistochemical assays for estrogen receptors (ERs) and progesterone receptors (PRs) have not been surveyed for technical validity. In the present study, the reliability of the immunohistochemical assay for ER and PR was evaluated using data from 105 laboratories participating in external quality assessment (EQA) during a 2-year period. Technical variables associated with reliable immunostaining were analyzed. The efficiency of the antigen retrieval step was identified as the single most important contributory factor influencing the overall reproducibility of the assays. Reliable assays were found in 24 (36%) of 66 laboratories participating in continual EQA, including the majority of centers known to have clinically validated results. Inadequate assay sensitivity, with subsequent weak staining, was the main cause of poor and variable results by laboratories using microwave antigen retrieval; too short a heating time was identified as the principal contributory factor. Extension of the heating time resulted in significant improvement regardless of all other variables in the immunohistochemical protocol. Continual participation in EQA is an effective means for identifying and ameliorating variables that influence the reliability of immunohistochemical assays for predictive markers, thereby assisting in technical validation and standardization.

Antigens↗

PubMed1992

Assessing the likelihood of reliable workplace behavior: further contributions to the validation of the Employee Reliability Inventory.

This paper summarizes a number of studies in which the validity of the Employee Reliability Inventory, a preemployment screening instrument designed to assess the likelihood of reliable and productive workplace behavior, was examined. Criterion-related studies compared the scores of a broadly diverse group of job applicants with those obtained from an array of criterion and comparison groups, for whom there was documented evidence of reliable or unreliable behavior. Criterion-related evidence indicates that the six scales are effective in differentiating a variety of criterion groups with unreliable behavior from a number of different job applicant comparison groups. Construct-related evidence for the validity of an emotional adjustment scale is reported as well. The issue of response distortion in preemployment inventories is discussed, and data are reported which indicate that scores on all six scales appear to be functionally free from the potentially confounding effects of response distortion. These results are consistent with the original validation and cross-validation findings, which supported the validity of the six scales when assessing the likelihood of reliable behavior in a population of job applicants.

Adult↗

PubMed2003

Combining reliability coefficients: possible application to meta-analysis and reliability generalization.

Formulae for combining reliability coefficients from any number of samples are provided. These formulae produce the exact reliability one would compute if one had the raw data from the samples. Needed are the sample means, standard deviations, sample sizes, and reliability coefficients. The formulae work for coefficient alpha, KR-20, retest, alternate-forms, split-half, interrater (intraclass), Gilmer-Feldt, Angoff-Feldt, validity, and other coefficients. They may be particularly useful for meta-analytic and reliability generalization studies.

Humans↗

PubMed2005

Combining reliability coefficients: toward reliability generalization of the conceptions of mathematics questionnaire.

The combined coefficient alpha from studies reporting the reliability of scores using the Conceptions of Mathematics Questionnaire were computed. Five studies comprising 898 participants were evaluated. A test of differences among the independent coefficients alpha was statistically significant (chi42= 10.38, p = .04) for the Fragmented and (chi42 = 11.58, p = .02) for the Cohesive subscales. Post hoc comparisons showed the difference (F129,299 = 1.50, p=.003) was between Australia and Nigeria for the former and (F155,157 = 1.54, p= .004) between South Africa and the United States for the latter alpha values. A one-way analysis of variance, testing for homogeneity among means within each subscale, indicated that these were homogeneous because the measure of the strength of association accounted for 10% of variability. As reliability coefficients were from homogeneous samples and alpha values were not different, the combined reliability is the best estimate of the population reliability for each subscale.

Australia↗

PubMed1997

Intraexaminer reliability, interexaminer reliability, and mean values for nine lower extremity skeletal measures in healthy naval midshipmen.

Physical therapists, athletic trainers, and other health care professionals commonly evaluate static postures and dynamic motions in patients with skeletal malalignments of the lower extremity. Accurate clinical evaluation of these malalignments depends on reliable and valid measures as well as established normal values. The purpose of this study was to document the intraexaminer and interexaminer reliability and mean values for nine static lower extremity skeletal measures in healthy Naval midshipmen. The measures studied were femoral torsion, ankle dorsiflexion, tibial length, leg length discrepancy, genu varus/valgus, medial talonavicular joint bulge, rearfoot angle, arch angle, and foot type classification. Intraexaminer and interexaminer reliability ICCs ranged between .65 and .97, and percentage agreements ranged between 88.8 and 94.4%. Mean absolute difference values all were within acceptable limits for clinical measurement. A foot type classification scheme was suggested. The results of this study indicate that the measures investigated are reliable when conducted on healthy Naval midshipmen by experienced examiners. Suggestions for further research are offered.

Adolescent↗

PubMedDate not supplied

[Reliability of the Spanish version of the Camberwell Assessment of Needs (CAN) (Spanish version of CAN Reliability Study)].

UNLABELLED: The Camberwell Assessment of Needs questionnaire (CAN) was developed in the Institute of Psychiatry (London) in the 1990's and constitutes one of the most widely accepted instruments to assess needs of people suffering from long term psychotic disorders. This assessment instrument was translated into Spanish by the Andalusian Group of Research in Mental Health. This paper describes both the methods and most significant results on the reliability of the Spanish version of the CAN. METHODS: 246 patients with a diagnosis of schizophrenia were interviewed as well as staff involved in their care. Kappa index was used to study concordance among the results obtained from interviews to the patients and those obtained from the staff. We calculated both inter-rater reliability and test-retest reliability. RESULTS: the mean score of needs was 6.47 in the patients sample and 6.49 in the staff sample. The level of agreement amongst patients and staff interviewed was moderate/strong. Inter-rater agreement was high (r=0.986 (patients) / 0.976 (staff), p<0.01). Similarly, test-retest agreement was r=0.791 in the patients group and r=0.766 in the staff sample (p<0.01). CONCLUSIONS: this study suggests that the Spanish version of the CAN is a highly reliable instrument to evaluate needs of people suffering from long-term psychotic disorders. It also is easy and fast to administer. However, we believe it may have some transcultural difficulties in certain areas as responses might be dependent on socio-cultural context.

Adult↗

PubMed1977

Reliability of psychiatric diagnosis. II. The test/retest reliability of diagnostic classification.

In a study of interrater diagnostic reliability, 101 psychiatric inpatients were independently interviewed by physicians using a structured interview. Newly admitted patients were randomly selected and examined by one of three psychiatrists. A second psychiatrist reexamined the same patient about 24 hours later. Interviews from the two examinations were evaluated independently and diagnoses were made on the basis of objective criteria. The degree of diagnostic agreement for the two examinations were calculated using the kappa statistic. Agreement was found to be high as compared to other studies in the psychiatric literature, despite the fact that in most previous investigations diagnoses were not made independently. The results were also compared to studies of reliability of medical judgments. Possible reasons for the high interrater reliability are discussed and include the use of a structured interview and objective diagnostic criteria.

Attitude of Health Personnel↗

PubMed2005

Validity and reliability of existing and modified clinical methods of measuring femoral and tibiofibular torsion in healthy subjects: use of different reference axes may improve reliability.

Previous studies have evaluated reliability and validity of clinical measurements of femoral and tibiofibular torsion, however, the results are conflicting, warranting an alternative method. The authors developed modified methods of measuring femoral and tibiofibular torsion using different reference axes, and validity and reliability of the modified and existing methods were examined in healthy subjects. Reference values were established using magnetic resonance imaging (MRI) for the validity study. There were significant differences between all clinical methods and corresponding MRI outcomes (P <0.05), whereas Pearson's correlation coefficients between the clinical methods and MRI ranged from 0.67-0.92. For modified methods, intraclass correlation coefficients were 0.89-0.97, whereas those of existing methods were 0.75-0.81. Modified methods were more precise, with standard error of measurements (SEM) of 1.5-2.9 degrees , compared to existing methods (SEM=4.0-5.1 degrees). These results indicate that clinical methods available currently do not accurately measure true torsion of the femur and tibia. Nevertheless, the clinical methods may be useful for screening and descriptive purposes as indices of true torsion, and use of different reference axes could improve their reliability.

Adolescent↗

PubMedDate not supplied

Reliability of life event assessments: test-retest reliability and fall-off effects of the Munich Interview for the Assessment of Life Events and Conditions.

This paper presents the findings of two independent studies which examined the test-retest reliability and the fall-off effects of the Munich Life Event List (MEL). The MEL is a three-step interview procedure for assessing life incidents which focuses on recognition processes rather than free recall. In a reliability study, test-retest coefficients of the MEL, based on a sample of 42 subjects, were quite stable over a 6-week interval. Stability for severe incidents appeared to be higher than for the less severe ones. In the fall-off study, a total rate of 30% fall-off was noted for all incidents reported retrospectively over an 8-year period. A more detailed analysis revealed average monthly fall-off effects of 0.36%. The size of fall-off effects was higher for non-severe and positive incidents than for severe incidents. This was particularly evident for the symptomatic groups. Non-symptomatic males reported a higher overall number of life incidents than females. This was partly due to more frequent reporting of severe incidents. The findings of the fall-off study do not support the common belief that the reliability of life incident report is much worse when the assessment period is extended over a period of several years as compared to the traditional 6-month period.

Adult↗

PubMed2004

Reliability of internal prediction/estimation and its application. I. Adaptive action selection reflecting reliability of value function.

This article proposes an adaptive action-selection method for a model-free reinforcement learning system, based on the concept of the 'reliability of internal prediction/estimation'. This concept is realized using an internal variable, called the Reliability Index (RI), which estimates the accuracy of the internal estimator. We define this index for a value function of a temporal difference learning system and substitute it for the temperature parameter of the Boltzmann action-selection rule. Accordingly, the weight of exploratory actions adaptively changes depending on the uncertainty of the prediction. We use this idea for tabular and weighted-sum type value functions. Moreover, we use the RI to adjust the learning coefficient in addition to the temperature parameter, meaning that the reliability becomes a general basis for meta-learning. Numerical experiments were performed to examine the behavior of the proposed method. The RI-based Q-learning system demonstrated its features when the adaptive learning coefficient and large RI-discount rate (which indicate how the RI values of future states are reflected in the RI value of the current state) were introduced. Statistical tests confirmed that the algorithm spent more time exploring in the initial phase of learning, but accelerated learning from the midpoint of learning. It is also shown that the proposed method does not work well with the actor-critic models. The limitations of the proposed method and its relationship to relevant research are discussed.

Acclimatization↗

PubMed1999

Reliability and limitations of echocardiographic measurement of left ventricular mass for risk stratification and follow-up in single patients: the RES trial. Working Group on Heart and Hypertension of the Italian Society of Hypertension. Reliability of M-mode Echocardiographic Studies.

OBJECTIVE: To investigate the clinical reliability of repeated measurements of left ventricular mass in a single patient. DESIGN: We used test-retest reliability analysis, within-class correlation and interval of agreement measures. METHODS: Two M-mode tracings (three consecutive cycles) were recorded in the same session and 3-10 days apart (5+/-2 days; mean +/- SD) in 261 participants (age 45+/-13 years, body mass index 24.7+/-3.6 kg/m2; 131 hypertensive and 130 normotensive; 50% of each group women) in 16 centres in Italy. The two tracings were read by two observers in each centre, after classification by a three-order quality score (1 = poor, 2 = sufficient, 3 = optimal). RESULTS: The average quality score was 2.11+/-0.71 (21% poor, 50% sufficient, 29% optimal). Left ventricular mass values ranged from 56 to 419 g (170+/-61 g). On the same day, within-observer 90% interval of agreement between tracing 1 and tracing 2 was -28 to +22 g (-17 to +11% of tracing 1). For day-to-day test-retest within-observer variability (average three cycles), the 90% interval of agreement was -30 to +35 g (-18 to +18%). This variability decreased to -13 to +12% at the 80% interval of agreement and -12 to +11% at the 75% interval of agreement. The 90% interval of agreement of test- retest between-observer variability was -26 to 30 g (-19 to +15%). A negligible regression toward the mean was identified. Categorical consistency of retest in the identification of hypertensive patients with left ventricular hypertrophy, classified in the first study, was 87% (k = 0.87). CONCLUSIONS: Measurement of left ventricular mass in single patients allows reliable risk stratification on the basis of the presence of left ventricular hypertrophy. The probability of a true change in left ventricular mass over time is maximized for a single-reader difference greater than 18% of the initial value, although differences of 10-13% might also have clinical relevance.

Adolescent↗

PubMed1986

Optimum split-half reliabilities for the Rorschach: projective techniques are more reliable than we think.

A new technique for optimizing split-half reliability estimates yielded substantial and comparable increases in indexes of internal reliability for some major Rorschach variables across two diverse samples. Practical suggestions for applying this method to other projective tests were advanced. The inadvisability of computing odd-even reliability coefficients without regard to split-half distributional anomalies is addressed.

Adult↗

PubMed2005

Reliability and comparison of acromion assessment techniques on X-ray and magnetic resonance imaging (reliability of acromion assessment techniques).

OBJECTIVE: To determine the reliability and correlation of plain radiography and magnetic resonance imaging (MRI) in the assessment of acromion morphology. MATERIALS AND METHODS: Acromion morphology was assessed using the lateral acromion angle (LAA) and the acromion-humeral interval (AHI). Thirty patients who had X-rays and MRI for impingement syndrome were included. Six blinded observers assessed the acromion morphology subjectively and objectively. RESULTS: Neither acromion assessment technique demonstrated a positive correlation (kappa and intraclass coefficient < 0.55) between X-ray and MRI. Both techniques were reliable (kappa and intraclass coefficient > 0.55) when measured objectively by experienced observers. CONCLUSION: The LAA and the AHI are both reliable acromion assessment techniques on X-ray and MRI when measured objectively and by experienced observers.

Acromion↗