Safety culture: is the "unit" the right "unit of analysis"?
Explore the source record for details and available documents.
Biomedical subjects
Publications and source records attributed to Amy K Rosen.
Explore the source record for details and available documents.
BACKGROUND: Providers use risk-adjustment systems to help manage healthcare costs. Typically, ordinary least squares (OLS) models on either untransformed or log-transformed cost are used. We examine the predictive ability of several statistical models, demonstrate how model choice depends on the goal for the predictive model, and examine whether building models on samples of the data affects model choice. METHODS: Our sample consisted of 525,620 Veterans Health Administration patients with mental health (MH) or substance abuse (SA) diagnoses who incurred costs during fiscal year 1999. We tested two models on a transformation of cost: a Log Normal model and a Square-root Normal model, and three generalized linear models on untransformed cost, defined by distributional assumption and link function: Normal with identity link (OLS); Gamma with log link; and Gamma with square-root link. Risk-adjusters included age, sex, and 12 MH/SA categories. To determine the best model among the entire dataset, predictive ability was evaluated using root mean square error (RMSE), mean absolute prediction error (MAPE), and predictive ratios of predicted to observed cost (PR) among deciles of predicted cost, by comparing point estimates and 95% bias-corrected bootstrap confidence intervals. To study the effect of analyzing a random sample of the population on model choice, we re-computed these statistics using random samples beginning with 5,000 patients and ending with the entire sample. RESULTS: The Square-root Normal model had the lowest estimates of the RMSE and MAPE, with bootstrap confidence intervals that were always lower than those for the other models. The Gamma with square-root link was best as measured by the PRs. The choice of best model could vary if smaller samples were used and the Gamma with square-root link model had convergence problems with small samples. CONCLUSION: Models with square-root transformation or link fit the data best. This function (whether used as transformation or as a link) seems to help deal with the high comorbidity of this population by introducing a form of interaction. The Gamma distribution helps with the long tail of the distribution. However, the Normal distribution is suitable if the correct transformation of the outcome is used.
BACKGROUND: Although difficulties in applying risk-adjustment measures to mental health populations are increasingly evident, a model designed specifically for patients with psychiatric disorders has never been developed. OBJECTIVE: Our objective was to develop and validate a case-mix classification system, the "PsyCMS," for predicting concurrent and future mental health (MH) and substance abuse (SA) healthcare costs and utilization. SUBJECTS: Subjects included 914,225 veterans who used Veterans Administration (VA) healthcare services during fiscal year 1999 (FY99) with any MH/SA diagnosis (International Classification of Diseases, 9th Revision, Clinical Modification [ICD-9-CM] codes 290.00-312.99, 316.00-316.99). METHODS: We derived diagnostic categories from ICD-CM codes using Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition definitions, clinical input, and empiric analyses. Weighted least-squares regression models were developed for concurrent (FY99) and prospective (FY00) MH/SA costs and utilization. We compared the predictive ability of the PsyCMS with several case-mix systems, including adjusted clinical groups, diagnostic cost groups, and the chronic illness and disability payment system. Model performance was evaluated using R-squares and mean absolute prediction errors (MAPEs). RESULTS: Patients with MH/SA diagnoses comprised 29.6% of individuals seen in the VA during FY99. The PsyCMS accounted for a distinct proportion of the variance in concurrent and prospective MH/SA costs (R=0.11 and 0.06, respectively), outpatient MH/SA utilization (R=0.25 and 0.07), and inpatient MH/SA utilization (R=0.13 and 0.05). The PsyCMS performed better than other case-mix systems examined with slightly higher R-squares and lower MAPEs. CONCLUSIONS: The PsyCMS has clinically meaningful categories, demonstrates good predictive ability for modeling concurrent and prospective MH/SA costs and utilization, and thus represents a useful method for predicting mental health costs and utilization.
BACKGROUND: The Patient Safety Indicators (PSIs), developed by the Agency for Healthcare Research and Quality, are useful screening tools for highlighting areas in which quality should be further investigated and providing useful benchmarks for tracking progress. OBJECTIVES: Our objectives were to: 1) provide a descriptive analysis of the incidence of PSI events from 2001 to 2004 in the Veterans Health Administration (VA); 2) examine trends in national PSI rates at the hospital discharge level over time; and 3) assess whether hospital characteristics (eg, teaching status, number of beds, and degree of quality improvement implementation) and baseline safety-related hospital performance predict future hospital safety-related performance. METHODS: We examined changes in risk-adjusted PSI rates at the discharge level, calculated the correlation between hospitals' risk-adjusted PSI rates in 2001 with subsequent years, and developed generalized linear models to examine predictors of hospitals' 2004 risk-adjusted PSI rates. RESULTS: Risk-adjusted rates of 2 of the 15 PSIs demonstrated significant trends over time. Rates of iatrogenic pneumothorax increased over time, whereas rates of failure to rescue decreased. Most PSIs demonstrated consistent rates over time. After accounting for patient and hospital characteristics, hospitals' baseline risk-adjusted PSI rates were the most important predictors of their 2004 risk-adjusted rates for 8 PSIs. CONCLUSIONS: The PSIs are useful tools for tracking and monitoring patient safety events in the VA. Future research should investigate whether trends reflect better or worse care or increased attention to documenting patient safety events.
OBJECTIVE: To assess the potential contribution of the Agency for Healthcare Research and Quality Patient Safety Indicators (PSIs) to organizational learning for patient safety improvement. PRINCIPAL FINDINGS: Patient safety improvement requires organizational learning at the system level, which entails changes in organizational routines that cut across divisions, professions, and levels of hierarchy. This learning depends on data that are varied along a number of dimensions, including structure-process-outcome and from granular to high-level; and it depends on integration of those varied data. PSIs are inexpensive, easy to use, less subject to bias than some other sources of patient safety data, and they provide reliable estimates of rates of preventable adverse events. CONCLUSIONS: From an organizational learning perspective, PSIs have both limitations and potential contributions as sources of patient safety data. While they are not detailed or timely enough when used alone, their simplicity and reliability make them valuable as a higher-level safety performance measure. They offer one means for coordination and integration of patient safety data and activity within and across organizations.
OBJECTIVE: Health outcome assessments have become an expectation of regulatory and accreditation agencies. We examined whether a clinically credible risk adjustment methodology for the outcome of change in health status can be developed for performance assessment of integrated service networks. STUDY DESIGN: Longitudinal study. SETTING: Outpatient. STUDY PARTICIPANTS: Thirty-one thousand eight hundred and twenty-three patients from 22 Veterans Health Administration (VHA) integrated service networks were followed for 18 months. MAIN MEASURE: The physical (PCS) and mental (MCS) component scales from the Veterans Rand 36-items Health Survey (VR-36) and mortality. The outcomes were decline in PCS (decline in PCS scores greater than -6.5 points or death) and MCS (decline in MCS scores greater than -7.9 points). RESULTS: Four thousand three hundred and twenty-eight (13.6%) patients showed a decline in PCS scores greater than -6.5 points, 4322 (13.5%) had a decline in MCS scores by more than -7.9 points, and 1737 died (5.5%). Multivariate logistic regression models were used to adjust for case-mix. The models performed reasonably well in cross-validated tests of discrimination (c-statistics = 0.72 and 0.68 for decline in PCS and MCS, respectively) and calibration. The resulting risk-adjusted rates of decline in PCS and MCS and ranks of the networks differed considerably from unadjusted ratings. CONCLUSION: It is feasible to develop clinically credible risk adjustment models for the outcomes of decline in PCS and MCS. Without adequate controls for case-mix, we could not determine whether poor patient outcomes reflect poor performance, sicker patients, or other factors. This methodology can help to measure and report the performance of health care systems.
BACKGROUND: The Patient Safety Indicators (PSIs), an administrative data-based tool developed by the Agency for Healthcare Research and Quality, are increasingly being used to screen for potential in-hospital patient safety problems. Although the Veterans Health Administration (VA) is a national leader in patient safety, accurate information on the epidemiology of patient safety events in the VA is still unavailable. OBJECTIVES: Our objectives were to: (1) apply the AHRQ PSI software to VA administrative data to identify potential instances of compromised patient safety; (2) determine occurrence rates of PSI events in the VA; and (3) examine the construct validity of the PSIs. METHODS: We examined differences between observed and risk-adjusted PSI rates in the VA, compared VA and non-VA PSI rates, and investigated the construct validity of the PSIs by examining correlations of the PSIs with other outcomes of VA hospitalizations. RESULTS: We identified 11,411 PSI events in the VA nationwide in FY'01. Observed PSI rates per 1000 discharges ranged from 0.007 for "transfusion reaction" to 155.5 for "failure to rescue." There were significant, although small, differences between VA and non-VA risk-adjusted PSI rates. Hospitalizations with PSI events had longer lengths of stay, higher mortality, and higher costs than those without PSI events. CONCLUSIONS: Our results suggest that the PSIs may be useful as a patient safety screening tool in the VA. Our PSI rates were consistent with the national incidence of low rates; however, differences between VA and non-VA rates suggest that inadequate case-mix adjustment may be contributing to these findings.
OBJECTIVES: To examine whether quality of care differed for veterans in Department of Veterans Affairs (VA) nursing homes and those on contract in community nursing homes, and whether the VA was contracting with nursing homes providing better quality of care than other nursing homes. DESIGN: Observational study using administrative databases from 1997 to 1999. SETTING: Ten VA and 650 community nursing homes in New York state. PARTICIPANTS: Four thousand seven hundred sixty-three veteran and 195,438 nonveteran residents of these nursing homes. MEASUREMENTS: Risk-adjusted rates of pressure ulcer development, functional decline, behavioral decline, and mortality. RESULTS: Veterans in VA nursing homes were significantly (P< .05) less likely to develop a pressure ulcer (odds ratio (OR)=0.63) but more likely to experience functional decline (OR=1.6) than veterans in community nursing homes. Residents of community nursing homes with VA contracts were significantly (P< .05) less likely to develop a pressure ulcer (OR=0.91) but more likely to die than residents in noncontract homes. Few nursing homes were consistently among the best or worst performers on all measures; only seven of 650 nursing homes were in the top or bottom decile and 34 in the top or bottom quartile for each measure. CONCLUSION: Large purchasers and providers of nursing home care such as the VA are unlikely to find information on quality of care useful in making decisions on whether they should "make" or "buy" care. Nursing homes performing well on one quality measure may perform poorly on another, and it is difficult to identify nursing homes that are consistently among the best or worst. Other consumers may encounter similar difficulties when using data on nursing home quality.
OBJECTIVES: To identify the extent of inappropriate prescribing using criteria for proper use developed by the Agency for Healthcare Research and Quality (AHRQ) and dose-limitation criteria defined by Beers, as well as to describe duration of use and patient characteristics associated with inappropriate prescribing for older people. DESIGN: Retrospective national Veterans Health Administration (VA) administrative database analysis. SETTING: VA outpatient facilities during fiscal year 2000 (FY00). PARTICIPANTS: Veterans aged 65 and older having at least one VA outpatient visit in FY00 (N=1,265,434). MEASUREMENTS: Operational definitions of appropriate use were developed based on recommendations of an expert panel convened by the AHRQ (Zhan criteria). Inappropriate use was identified based on these criteria and inappropriate use of drugs per Beers criteria for dose-limitations in older people. Furthermore, duration of use and patient characteristics associated with inappropriate use were described. RESULTS: After adjusting for diagnoses, dose, and duration, inappropriate prescribing decreased from 33% to 23%. Exposure to inappropriate drugs was prolonged. Pain relievers, benzodiazepines, antidepressants, and musculoskeletal agents constituted 61% of inappropriate prescribing. Whites, patients with psychiatric comorbidities, and patients receiving more medications were most likely to receive inappropriate drugs. Women were more likely to receive Zhan criteria drugs; men were more likely to receive dose-limited drugs CONCLUSION: For the most part, the Zhan criteria did not explain inappropriate prescribing, which includes problems related to dose and duration of prescriptions. Interventions targeted at prescriptions for pain relievers, benzodiazepines, antidepressants, and musculoskeletal agents may dramatically decrease inappropriate prescribing and improve patient outcomes.
BACKGROUND: Linking process and outcomes is critical to accurately estimating healthcare quality and quantifying its benefits. OBJECTIVES: The objective of this study was to explore the relationship of guideline-based depression process measures with subsequent overall and psychiatric hospitalizations. RESEARCH DESIGN: This is a retrospective cohort study during which we used administrative and centralized pharmacy records for sample identification, derivation of guideline-based process measures (antidepressant dosage and duration adequacy), and subsequent hospitalization ascertainment. Depression care was measured from June 1, 1999, through August 31, 1999. We used multivariable regression to evaluate the link between depression care and subsequent overall and psychiatric hospitalization, adjusting for patient age, race, sex, socioeconomic status, comorbid illness, and hospitalization in the prior 12 months. SUBJECTS: We studied a total of 12,678 patients from 14 Northeastern VHA hospitals. RESULTS: We identified adequate antidepressant dosage in 90% and adequate duration in 45%. Those with adequate duration of antidepressants were less likely to be hospitalized in the subsequent 12 months than those with inadequate duration (odds ratio [OR],.90; 95% confidence interval [CI], .81-1.00). Those with adequate duration of antidepressants were less likely to have a psychiatric hospitalization in the subsequent 12 months than those with inadequate duration (OR, .82; 95% CI, .69-.96). We did not demonstrate a significant link between dosage adequacy and subsequent overall or psychiatric hospitalization. CONCLUSIONS: Guideline-based depression process measures derived from centralized data sources offer an important method of depression care surveillance. Their accuracy in capturing depression care quality is supported by their link to healthcare utilization. Further work is needed to assess the effect of implementing these quality indicators on depression care.
OBJECTIVES: We examined racial differences in cardiac catheterization rates and reviewed whether patients' beliefs or other variables were associated with observed disparities. METHODS: We did a prospective observational cohort study of 1045 White and African American patients at 5 Veterans Affairs (VA) medical centers whose nuclear imaging studies indicated reversible cardiac ischemia. RESULTS: There were few demographic differences between White and African American patients in our sample. African Americans were less likely than Whites to undergo cardiac catheterization. African Americans were more likely than Whites to indicate a strong reliance on religion and to report racial and social class discrimination and were less likely to indicate a generalized trust in people but did not differ from White patients on numerous other attitudes about health and health care. Neither sociodemographic or clinical characteristics nor patients' beliefs explained the observed disparities, but physicians' assessments of the procedure's importance and patients' likelihood of coronary disease seemed to account for differences not otherwise explained. CONCLUSIONS: Patients' preferences are not the likely source of racial disparities in the use of cardiac catheterization among veterans using VA care, but physicians' assessments warrant further attention.
OBJECTIVE: To explore the relationship of systemwide continuous quality improvement (CQI) with depression care quality in the Veterans Health Administration (VHA). STUDY DESIGN: Observational study using data from 2 VHA studies. PATIENTS AND METHODS: The Depression Care Quality Study (DCQS) was a retrospective cohort study of depression care quality in the northeastern United States involving 12 678 patients cared for at 14 VHA facilities; it used guideline-based process measures (ie, dosage and duration adequacy). The VHA CQI survey was a cross-sectional survey of systemwide CQI among a representative sample of VHA hospitals; it assessed CQI and organizational culture (OC) at 116 VHA hospitals nationwide and provided data on the 14 study facilities. We used analysis of variance to identify differences in the adequacy of depression care among these facilities. Pearson's correlation was used to identify the relationship of CQI and OC with facility-level depression care adequacy. RESULTS: Mean depression care adequacy differed among the 14 DCQS facilities (P < .0001). Overall dosage adequacy was 90% (range: 87%-92%). Overall duration adequacy was 45% (range: 39%-64%). There was no correlation between CQI and either dosage adequacy (r= .004, P= .98) or duration adequacy (r= -.17, P= .55). Similarly, there was no correlation between OC and either dosage adequacy (r= -.35, P= .22) or duration adequacy (r= -.12, P= .68). CONCLUSION: Although CQI may help bridge the healthcare quality gap, it may not be associated with higher disease-specific quality of care.
Patient safety has become a major public health concern following the publication of the landmark report, To Err Is Human, by the Institute of Medicine in 1999. This report, along with a subsequent report, Crossing the Quality Chasm, recommended the design of a safer health care system by integrating well-established safety methods to avert medical errors. However, neither patient safety report specifically addressed the implications of safety for elderly patients. This article examines those implications by describing the association between aging and medical errors, identifying geriatric syndromes as medical errors, and focusing on six recommendations that will improve the safety of geriatric care. These six recommendations include the detection and reporting of geriatric syndromes, identifying system failures when geriatric syndromes occur, establishing dedicated geriatric units, improving the continuity of care, reducing adverse drug events, and improving geriatric training programs.
BACKGROUND: Guideline-based depression process measures provide a powerful way to monitor depression care and target areas needing improvement. OBJECTIVES: To assess the adequacy of depression care in the Veterans Health Administration (VHA) using guideline-based process measures derived from administrative and centralized pharmacy records, and to identify patient and provider characteristics associated with adequate depression care. RESEARCH DESIGN: This is a cohort study of patients from 14 VHA hospitals in the Northeastern United States which relied on existing databases. Subject eligibility criteria: at least one depression diagnosis during 1999, neither schizophrenia nor bipolar disease, and at least one antidepressant prescribed in the VHA during the period of depression care profiling (June 1, 1999 through August 31, 1999). Depression care was evaluated with process measures defined from the 1997 VHA depression guidelines: antidepressant dosage and duration adequacy. We used multivariable regression to identify patient and provider characteristics predicting adequate care. SUBJECTS: There were 12,678 patients eligible for depression care profiling. RESULTS: Adequate dosage was identified in 90%; 45% of patients had adequate duration of antidepressants. Significant patient and provider characteristics predicting inadequate depression care were younger age (<65), black race, and treatment exclusively in primary care. CONCLUSIONS: Under-treatment of depression exists in the VHA, despite considerable mental health access and generous pharmacy benefits. Certain patient populations may be at higher risk for inadequate depression care. More work is needed to align current practice with best-practice guidelines and to identify optimal ways of using available data sources to monitor depression care quality.
BACKGROUND: Although most widely used risk adjustment systems use diagnosis data to classify patients, there is growing interest in risk adjustment based on computerized pharmacy data. The Veterans Health Administration (VHA) is an ideal environment in which to test the efficacy of a pharmacy-based approach. OBJECTIVE: To examine the ability of RxRisk-V to predict concurrent and prospective costs of care in VHA and compare the performance of RxRisk-V to a simple age/gender model, the original RxRisk, and two leading diagnosis-based risk adjustment approaches: Adjusted Clinical Groups and Diagnostic Cost Groups/Hierarchical Condition Categories. METHODS: The study population consisted of 161,202 users of VHA services in Washington, Oregon, Idaho, and Alaska during fiscal years (FY) 1996 to 1998. We examined both concurrent and predictive model fit for two sequential 12-month periods (FY 98 and FY 99) with the patient-year as the unit of analysis, using split-half validation. RESULTS: Our results show that the Diagnostic Cost Group /Hierarchical Condition Categories model performs best (R2 = 0.45) among concurrent cost models, followed by ADG (0.31), RxRisk-V (0.20), and age/sex model (0.01). However, prospective cost models other than age/sex showed comparable R2: Diagnostic Cost Group /Hierarchical Condition Categories R2 = 0.15, followed by ADG (0.12), RxRisk-V (0.12), and age/sex (0.01). CONCLUSIONS: RxRisk-V is a clinically relevant, open source risk adjustment system that is easily tailored to fit specific questions, populations, or needs. Although it does not perform better than diagnosis-based measures available on the market, it may provide a reasonable alternative to proprietary systems where accurate computerized pharmacy data are available.
OBJECTIVES: To compare the disease burden of Veterans Health Administration (VA) patients at six study sites with all other VA patients and the Medicare population. DESIGN: A 60% random sample of all VA veteran patients during federal fiscal year 1997 was obtained from administrative databases. A split-sample technique provided a 40% sample (n = 1,046,803) for development and a 20% sample (n = 524,461) for validation. We selected the six study sites from the 40% sample, yielding a total of 50,080 patients in those sites. METHODS: We used Diagnostic Cost Groups to classify patients into clinical groupings based on age, gender, and International Classification of Diseases, Ninth Revision, Clinical Modification diagnoses. The Diagnostic Cost Group model produces relative risk scores that describe patients' expected resource use normalized to the Medicare population. We compared the severity of the six sites with each other and with all other VA facilities and the severity of VA patients with that of Medicare beneficiaries. RESULTS: There were minor statistically significant differences between the study sites and all other VA facilities. Compared with the Medicare population, VA's population was younger and had lower expected resource use (relative risk scores were 1.0 and 0.76, respectively). CONCLUSIONS: Disease burden of the six study sites is representative of all other VA facilities. Although lower relative risk scores suggest that VA patients are healthier than Medicare beneficiaries, when age is taken into account, scores are more comparable. Interpreting the expected resource utilization of the VA population against other benchmarks should be performed carefully.
OBJECTIVE: To compare the rankings for health care utilization performance measures at the facility level in a Veterans Health Administration (VHA) health care delivery network using pharmacy- and diagnosis-based case-mix adjustment measures. DATA SOURCES/STUDY SETTING: The study included veterans who used inpatient or outpatient services in Veterans Integrated Service Network (VISN) 20 during fiscal year 1998 (October 1997 to September 1998; N = 126,076). Utilization and pharmacy data were extracted from VHA national databases and the VISN 20 data warehouse. STUDY DESIGN: We estimated concurrent regression models using pharmacy or diagnosis information in the base year (FY1998) to predict health service utilization in the same year. Utilization measures included bed days of care for inpatient care and provider visits for outpatient care. PRINCIPAL FINDINGS: Rankings of predicted utilization measures across facilities vary by case-mix adjustment measure. There is greater consistency within the diagnosis-based models than between the diagnosis- and pharmacy-based models. The eight facilities were ranked differently by the diagnosis- and pharmacy-based models. CONCLUSIONS: Choice of case-mix adjustment measure affects rankings of facilities on performance measures, raising concerns about the validity of profiling practices. Differences in rankings may reflect differences in comparability of data capture across facilities between pharmacy and diagnosis data sources, and unstable estimates due to small numbers of patients in a facility.
OBJECTIVE: To assess the performance of Diagnostic Cost Groups (DCGs) in explaining variation in concurrent utilization for a defined subgroup, patients with substance abuse (SA) disorders, within the Department of Veterans Affairs (VA). DATA SOURCES: A 60 percent random sample of veterans who used health care services during Fiscal Year (FY) 1997 was obtained from VA administrative databases. Patients with SA disorders (13.3 percent) were identified from primary and secondary ICD-9-CM diagnosis codes. STUDY DESIGN: Concurrent risk adjustment models were fitted and tested using the DCG/HCC model. Three outcome measures were defined: (1) "service days" (the sum of a patient's inpatient and outpatient visit days), (2) mental health/substance abuse (MH/SA) service days, and (3) ambulatory provider encounters. To improve model performance, we ran three DCG/HCC models with additional indicators for patients with SA disorders. DATA COLLECTION: To create a single file of veterans who used health care services in FY 1997, we merged records from all VA inpatient and outpatient files. PRINCIPAL FINDINGS: Adding indicators for patients with mild/moderate SA disorders did not appreciably improve the R-squares for any of the outcome measures. When indicators were added for patients with severe SA who were in the most costly category, the explanatory ability of the models was modestly improved for all three outcomes. CONCLUSIONS: Modifying the DCG/HCC model with additional markers for SA modestly improved homogeneity and model prediction. Because considerable variation still remained after modeling, we conclude that health care systems should evaluate "off-the-shelf" risk adjustment systems before applying them to their own populations.