Risk prediction models: recalibration.
Explore the source record for details and available documents.
Calibration↗
Biomedical subjects
Kathy Rowan
Publications and source records attributed to Kathy Rowan.
The Safety Attitudes Questionnaire: psychometric properties, benchmarking data, and emerging research.
BACKGROUND: There is widespread interest in measuring healthcare provider attitudes about issues relevant to patient safety (often called safety climate or safety culture). Here we report the psychometric properties, establish benchmarking data, and discuss emerging areas of research with the University of Texas Safety Attitudes Questionnaire. METHODS: Six cross-sectional surveys of health care providers (n = 10,843) in 203 clinical areas (including critical care units, operating rooms, inpatient settings, and ambulatory clinics) in three countries (USA, UK, New Zealand). Multilevel factor analyses yielded results at the clinical area level and the respondent nested within clinical area level. We report scale reliability, floor/ceiling effects, item factor loadings, inter-factor correlations, and percentage of respondents who agree with each item and scale. RESULTS: A six factor model of provider attitudes fit to the data at both the clinical area and respondent nested within clinical area levels. The factors were: Teamwork Climate, Safety Climate, Perceptions of Management, Job Satisfaction, Working Conditions, and Stress Recognition. Scale reliability was 0.9. Provider attitudes varied greatly both within and among organizations. Results are presented to allow benchmarking among organizations and emerging research is discussed. CONCLUSION: The Safety Attitudes Questionnaire demonstrated good psychometric properties. Healthcare organizations can use the survey to measure caregiver attitudes about six patient safety-related domains, to compare themselves with other organizations, to prompt interventions to improve safety attitudes and to measure the effectiveness of these interventions.
Ambulatory Care Facilities↗
Assessment and optimization of mortality prediction tools for admissions to pediatric intensive care in the United kingdom.
OBJECTIVE: To assess the Pediatric Risk of Mortality (PRISM, PRISM III-12, and PRISM III-24) systems and the Pediatric Index of Mortality (PIM and PIM2) systems for use in comparing the risk-adjusted mortality of children after admission for pediatric intensive care in the United Kingdom. METHODS: All PICUs in the United Kingdom were invited to participate. Predicted probability of PICU mortality was calculated using the published algorithms for PIM, PIM2, and PRISM and compared with observed mortality. These scores, along with PRISM III-12 and PRISM III-24, whose algorithms are not published, were optimized for the United Kingdom. RESULTS: Of 26 PICUs in the United Kingdom, 22 (85%) were recruited, and sufficient prospective data were collected from 18 (69%) units on 10,197 (98%) of 10,385 admissions between March 2001 and February 2002. All published tools were found to have poor calibration but provided good discriminatory power. After estimation of UK-specific coefficients, only PIM2, PRISM III-12, and PRISM III-24 had satisfactory calibration. All models provided good discriminatory power. Funnel plots for all of the recalibrated models indicated that the risk-adjusted mortality for all units was consistent with random variation. CONCLUSIONS: PIM2, PRISM III-12, and PRISM III-24 all were found to be suitable for use in a UK PICU setting. All tools provided similar conclusions in assessing the distribution of risk-adjusted mortality in UK PICUs. It now is important that these tools be used to monitor outcome and improve the quality of pediatric intensive care within the United Kingdom.
Child↗
Recalibration of risk prediction models in a large multicenter cohort of admissions to adult, general critical care units in the United Kingdom.
OBJECTIVE: To assess the performance of published risk prediction models in common use in adult critical care in the United Kingdom and to recalibrate these models in a large representative database of critical care admissions. DESIGN: Prospective cohort study. SETTING: A total of 163 adult general critical care units in England, Wales, and Northern Ireland, during the period of December 1995 to August 2003. PATIENTS: A total of 231,930 admissions, of which 141,106 met inclusion criteria and had sufficient data recorded for all risk prediction models. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: The published versions of the Acute Physiology and Chronic Health Evaluation (APACHE) II, APACHE II UK, APACHE III, Simplified Acute Physiology Score (SAPS) II, and Mortality Probability Models (MPM) II were evaluated for discrimination and calibration by means of a combination of appropriate statistical measures recommended by an expert steering committee. All models showed good discrimination (the c index varied from 0.803 to 0.832) but imperfect calibration. Recalibration of the models, which was performed by both the Cox method and re-estimating coefficients, led to improved discrimination and calibration, although all models still showed significant departures from perfect calibration. CONCLUSIONS: Risk prediction models developed in another country require validation and recalibration before being used to provide risk-adjusted outcomes within a new country setting. Periodic reassessment is beneficial to ensure calibration is maintained.
APACHE↗
Outcome at 6 months after admission for pediatric intensive care: a report of a national study of pediatric intensive care units in the United kingdom.
OBJECTIVE: The goal was to measure, by using the Health Utilities Index, the health status of children 6 months after admission to PICUs in the United Kingdom. METHODS: All PICUs in the United Kingdom were invited to participate. Children who were > or = 6 months of age at admission and were discharged alive from participating units during a 1-year period were eligible for this study. Children with completed consent forms who had survived to 6 months after admission received the Health Utilities Index questionnaire. RESULTS: Admission data on 7214 admissions to 22 units were collected between March 2001 and February 2002. Of those patients, 6786 survived to unit discharge, and consent for follow-up contact was obtained for 2642. At 6 months after admission, the mortality rate was estimated at 11.1% and 2034 children were still alive and contactable. Of those children, Health Utilities Index questionnaires were returned by 1455. No impairment was indicated for 767, 951, 940, 919, 962, and 939 children with respect to the Health Utilities Index sensation, cognition, emotion, pain, mobility, and self-care attributes, respectively. The mean +/- SE Health Utilities Index utility score was 0.73 +/- 0.01, with 397 children (27.3%) in full health. CONCLUSIONS: Death after pediatric intensive care in the United Kingdom is uncommon, making assessment of health status important. At 6 months after admission, there is significant morbidity. To assess the impact of strategies to improve or to optimize care on longer-term outcomes, standardized collection of data on preexisting comorbidities and illness severity is required.
Child↗
The incremental cost effectiveness of withdrawing pulmonary artery catheters from routine use in critical care.
OBJECTIVE: The objective of this study was to conduct an economic evaluation to identify any differences in the expected costs and outcomes between patients treated with pulmonary artery catheters (PACs) and those without, in order to better inform healthcare decision makers. METHOD: The evaluation was carried out alongside a clinical trial investigating the use of PACs in intensive care units (ICUs) in the UK. It was conducted from the perspective of the UK NHS, in which PACs are an established intervention. Treating patients without using a PAC was characterised as the new intervention. The primary outcome measure was QALYs. The secondary outcome measure was hospital mortality. NHS costs per patient were calculated for the financial year 2002/03. The bootstrap method was used to characterise the uncertainty of the results and to construct cost-effectiveness acceptability curves. RESULTS: The cost per QALY and per life gained from the withdrawal of PACs were Pounds 2892 and Pounds 21,164, respectively. CONCLUSION: The results of this study indicate that withdrawal of PACs from routine clinical use in ICUs within the NHS would be considered cost effective in the current decision-making climate.
Aged↗
Winter excess mortality in intensive care in the UK: an analysis of outcome adjusted for patient case mix and unit workload.
OBJECTIVE: To investigate whether mortality in UK intensive care units is higher in winter than in non-winter and to explore the importance of variations in case mix and increased pressure on ICUs. DESIGN AND SETTING: Cohort study in 115 adult, general ICUs in England, Wales and Northern Ireland. PATIENTS AND PARTICIPANTS: 113,389 admissions from 1995 to 2000. MEASUREMENTS AND RESULTS: Hospital mortality following admission to ICU was compared between winter (December-February) and non-winter (March-November). The causes of any observed differences were explored by adjusting for the case mix of admissions and the workload of the ICUs. Crude hospital mortality was higher in winter. After adjusting for case mix using the APACHE II mortality probability this effect was reduced but still significant. When additional factors reflecting case mix and workload were introduced into the model, the overall effect of winter admission was no longer significant. Factors reflecting both the case mix of the individual patient and of the patients in surrounding beds were found to be significantly associated with outcome. After adjustment for other factors, the occupancy of the unit (proportion of beds occupied) was not significantly associated with mortality. CONCLUSIONS: The excess winter mortality observed in UK ICUs can be explained by variation in the case mix of admissions. Unit occupancy was not associated with mortality.
APACHE↗
Characteristics and outcome for admissions to adult, general critical care units with acute severe asthma: a secondary analysis of the ICNARC Case Mix Programme Database.
INTRODUCTION: This report describes the case mix, outcome and activity (duration of intensive care unit [ICU] and hospital stay, inter-hospital transfer, and readmissions to the ICU) for admissions to ICUs for acute severe asthma, and investigates the effect of case mix factors on outcome. METHODS: We conducted a secondary analysis of data from a high-quality clinical database (the Intensive Care National Audit and Research Centre [ICNARC] Case Mix Programme Database) of 129,647 admissions to 128 adult, general critical care units across England, Wales and Northern Ireland over the period 1995-2001. RESULTS: Asthma accounted for 2152 (1.7%) admissions, and in 57% mechanical ventilation was employed during the first 24 hours in the ICU. A total of 147 (7.1%) patients died in intensive care and 199 (9.8%) died before discharge from hospital. The mean age was 43.6 years, and the ratio of women to men was 2:1. Median length of stay was 1.5 days in the ICU and 8 days in hospital. Older age, female sex, having received cardiopulmonary resuscitation (CPR) within 24 hours before admission, having suffered a neurological insult during the first 24 hours in the ICU, higher heart rate, and hypercapnia were associated with greater risk for in-hospital death after adjusting for Acute Physiology and Chronic Health Evaluation II score. CPR before admission, neurological insult, hypoxaemia and hypercapnia were associated with receipt of mechanical ventilation after adjusting for Acute Physiology and Chronic Health Evaluation II score. CONCLUSION: ICU admission for asthma is relatively uncommon but remains associated with appreciable in-hospital mortality. The greatest determinant of poor hospital survival in asthma patients was receipt of CPR within 24 hours before admission to ICU. Clinical management of these patients should be directed at preventing cardiac arrest before admission.
APACHE↗
Case mix, outcome and length of stay for admissions to adult, general critical care units in England, Wales and Northern Ireland: the Intensive Care National Audit & Research Centre Case Mix Programme Database.
INTRODUCTION: The present paper describes the methods of data collection and validation employed in the Intensive Care National Audit & Research Centre Case Mix Programme (CMP), a national comparative audit of outcome for adult, critical care admissions. The paper also describes the case mix, outcome and activity of the admissions in the Case Mix Programme Database (CMPD). METHODS: The CMP collects data on consecutive admissions to adult, general critical care units in England, Wales and Northern Ireland. Explicit steps are taken to ensure the accuracy of the data, including use of a dataset specification, of initial and refresher training courses, and of local and central validation of submitted data for incomplete, illogical and inconsistent values. Criteria for evaluating clinical databases developed by the Directory of Clinical Databases were applied to the CMPD. The case mix, outcome and activity for all admissions were briefly summarised. RESULTS: The mean quality level achieved by the CMPD for the 10 Directory of Clinical Databases criteria was 3.4 (on a scale of 1 = worst to 4 = best). The CMPD contained validated data on 129,647 admissions to 128 units. The median age was 63 years, and 59% were male. The mean Acute Physiology and Chronic Health Evaluation II score was 16.5. Mortality was 20.3% in the CMP unit and was 30.8% at ultimate discharge from hospital. Nonsurvivors stayed longer in intensive care than did survivors (median 2.0 days versus 1.7 days in the CMP unit) but had a shorter total hospital length of stay (9 days versus 16 days). Results for the CMPD were comparable with results from other published reports of UK critical care admissions. CONCLUSIONS: The CMP uses rigorous methods to ensure data are complete, valid and reliable. The CMP scores well against published criteria for high-quality clinical databases.
Adult↗
Hospitals' star ratings and clinical outcomes: ecological study.
Explore the source record for details and available documents.
Hospitals, Public↗
Impact of exclusion criteria on case mix, outcome, and length of stay for the severity of disease scoring methods in common use in critical care.
OBJECTIVE: To identify the exclusion criteria for the major severity of disease scoring methods in critical care and to investigate the impact of the exclusion criteria on the case mix, outcomes and length of stay for admissions to intensive care units (ICUs) in England, Wales and Northern Ireland. DESIGN: Cohort study-analysis of prospectively collected data. SETTING: 127 adult, general (mixed medical/surgical) ICUs in England, Wales and Northern Ireland. PATIENTS: 120,503 admissions between 1995 and 2001. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: Thirteen different exclusion criteria were identified from the original methodological/validation papers and data collection manuals for APACHE II, APACHE III, SAPS II and MPM II. Application of the original exclusion criteria for the four, major severity of disease scoring methods resulted in exclusion of between 11.5% and 14.6% of admissions. Hospital mortality for the overall cohort was 29.0% but ranged from 4.7% to 50.1% among those groups excluded. After application of the exclusion criteria for each scoring method, there was little difference in overall hospital mortality or median ICU and hospital length of stay for the included admissions when compared with the original cohort. At the level of individual ICUs, there were differences in hospital mortality before and after exclusions-minimum -3.1% to maximum 9.5% (APACHE II), minimum -2.8% to maximum 9.4% (APACHE III), minimum -3.1% to maximum 16.1% (SAPS II) and minimum -3.1% to maximum 16.5% (MPM II). The mean difference across individual ICUs was -0.5 % (95% CI -0.7% to -0.2%) for APACHE II, -0.2% (95% CI -0.2% to 0.1%) for APACHE III, 2.0% (95% CI 1.7% to 2.4%) for SAPS II and 2.1% (95% CI 1.7% to 2.5%) for MPM II. SAPS II and MPM II showed systematic variation. A survey of the literature found wide variation in the exclusion criteria reported in subsequent, published research using a single severity of disease scoring method (APACHE II). CONCLUSIONS: Exclusion criteria used in critical care research are often ill-defined and poorly reported. More attention to the choice of exclusion criteria and their effect on the reported results is essential. We hope this study will raise the need for both better reporting of exclusion criteria applied in studies and promote the need for a common set of explicit exclusion criteria for these methods.
APACHE↗
Socioeconomic status and outcome from intensive care in England and Wales.
OBJECTIVE: The objective of this study was to estimate the association between socioeconomic status (SES) and outcome for admissions to intensive care. RESEARCH DESIGN: Retrospective cohort study. SUBJECTS: We studied 51,572 admissions to 99 intensive-care units in England and Wales between 1995 and 2000. MEASURES: The SES of admissions was measured using Carstairs deprivation scores. Outcome was hospital mortality after adjustment for case mix using the APACHE II method. RESULTS: Admissions of lower SES were, on average, younger and less likely to be following surgery. There was evidence of a SES gradient for hospital mortality in admissions after elective surgery after adjusting for case mix (test for trend P <0.001), with higher SES associated with lower mortality. In the least-deprived quintile of SES, the odds ratio for hospital mortality was 0.70 (95% confidence interval, 0.58-0.84) compared with the most deprived quintile. There was no evidence of a SES gradient for hospital mortality in nonsurgical or emergency surgical admissions, and the decision to withdraw active treatment did not differ by SES. CONCLUSIONS: There is a SES gradient for hospital mortality in elective surgical admissions that is not explained by differences in case mix or the withdrawal of active treatment. Further research is required to establish if this finding can be explained by unmeasured differences in health status at admission to an intensive-care unit or differences in care and to establish the potential impact these results may have on interpreting comparative surgical performance data.
APACHE↗
Epidemiology of severe sepsis occurring in the first 24 hrs in intensive care units in England, Wales, and Northern Ireland.
OBJECTIVE: To investigate the numbers, clinical characteristics, resource use, and outcomes of admissions who met precise clinical and physiologic criteria for severe sepsis (as defined in the PROWESS trial) in the first 24 hrs in the intensive care unit. DESIGN: Observational cohort study, with retrospective analysis of prospectively collected data. SETTING: Ninety-one adult general intensive care units in England, Wales, and Northern Ireland between 1995 and 2000. PATIENTS: Patients were 56,673 adult admissions. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: We found that 27.1% of adult intensive care unit admissions met severe sepsis criteria in the first 24 hrs in the intensive care unit. Most were nonsurgical (67%), and the most common organ system dysfunctions were seen in the cardiovascular (88%) and respiratory (81%) systems. Modeling the data for England and Wales for 1997 suggested that 51 (95% confidence interval, 46-58) per 100,000 population per year were admitted to intensive care units and met severe sepsis criteria in the first 24 hrs.Of the intensive care unit admissions who met severe sepsis criteria in the first 24 hrs, 35% died before intensive care unit discharge and 47% died during their hospital stay. Hospital mortality rate ranged from 17% in the 16-19 age group to 64% in those >85 yrs. In England and Wales in 1997, an estimated 24 (95% confidence interval, 21-28) per 100,000 population per year died after intensive care unit admissions with severe sepsis in the first 24 hrs. For intensive care unit admissions who met severe sepsis criteria in the first 24 hrs, median intensive care unit length of stay was 3.56 days (interquartile range, 1.50-9.32) and median hospital length of stay was 18 days (interquartile range, 8-36 days). These admissions used 45% of the intensive care unit and 33% of the hospital bed days used by all intensive care unit admissions. CONCLUSIONS: Severe sepsis is common and presents a major challenge for clinicians, managers, and healthcare policymakers. Intensive care unit admissions meeting severe sepsis criteria have a high mortality rate and high resource use.
Adolescent↗