Biomedical subjects

S S Ehrmeyer

Publications and source records attributed to S S Ehrmeyer.

17 recordsLinked to original sources

PubMed1992

Health Care Financing Administration's new proficiency testing rules. Use of a statistical approach to predict long-term performance using the categories 'successful,' 'probation,' and 'suspended'.

Under the Clinical Laboratory Improvement Act of 1967 (CLIA-67) and the Medicare Act, the Health Care Financing Administration's proficiency testing rules apply uniformly to all hospital and reference laboratories. We examined the relationship between internal laboratory performance as characterized by bias and coefficient of variation and proficiency testing performance, categorized as "successful," "probation," and "suspended." Under the March 14, 1990, final rule, a laboratory with suspended testing for even one analyte may be required to cease testing in the entire subspecialty, eg, routine chemistry, unless it ceases testing for that analyte. Analyzing this regimen as a Markov process, we obtained the steady-state solution for performance for one to 27 analytes. While 1.1% of laboratories testing for five analytes with internal or day-to-day coefficients of variation at 50% of the CLIA-67 proficiency testing limit would be suspended, 19.5% of laboratories having biases of 50% and coefficients of variation of 33% would be suspended. We conclude that after eight events, there will be an unacceptably high rate of suspensions.

Centers for Medicare and Medicaid Services, U.S.↗

PubMed1992

Intralaboratory performance requirements necessary to pass proficiency testing: CAP-1990 vs CLIA-1967 (March 14, 1990) formats compared.

The pre-1990 College of American Pathologists' (CAP) Proficiency Testing (PT) program used a two samples per analyte/four challenges per year format with performance or pass-fail grading criteria determined by the program. On Jan. 1, 1991, the Clinical Laboratory Improvement Act of 1967 (CLIA-67) final rules (March 14, 1990) mandated a revised PT format of five samples per analyte/four challenges per year, with the regulations specifying minimum performance criteria. Extending our previous analysis, we compare the maximum permissible intralaboratory imprecision at low bias compatible with passing external PT in the former CAP and current CLIA-67 formats. If a laboratory is able to reduce its internal coefficient of variation (CV) to less than 44% of the PT criterion for each analyte, its overall chance of adverse action for any of the 27 routine chemistry analytes specified in CLIA-67 will be less than 1% in a two-year (eight PT challenges or events) period. Consideration of actual interlaboratory CVs from CAP surveys suggest that a reduction of this magnitude may be difficult for the analytes total cholesterol and blood urea nitrogen, where intralaboratory imprecision comparable with the group standard deviation (SD) from 1990 CAP surveys would yield individual adverse action (PT failure) rates of 5% and 1%, respectively. Five other analytes have CLIA-67 performance limits dangerously close to CAP interlaboratory CVs.

Chemistry, Clinical↗

PubMed1992

Limitations of proficiency testing under CLIA '67.

Proficiency testing (PT), recognized as a quality-assurance (QA) and quality-improvement tool, also has become the cornerstone of the Health Care Financing Administration's (HCFA) regulatory strategy under the revised Clinical Laboratory Improvement Act of 1967 (CLIA '67) and the proposed Clinical Laboratory Improvement Amendments of 1988 (CLIA '88). Use of PT as a regulatory tool corrupts it for things it can do better. PT as a primary regulatory strategy has severe limitations. We explore the nature of these limitations and their implications for clinical laboratories as they impact on the long-term success of HCFA's approved regulatory PT programs in 1991 and beyond, and CLIA '88 PT, which is to be implemented in 1994.

Chemistry, Clinical↗

PubMed1992

Review of actual proficiency-testing performance under CLIA '67 (March 14, 1990) rules: perspective from the first year's data.

Under the Clinical Laboratory Improvement Act of 1967 the Health Care Financing Administration's proficiency-testing requirement applies to approximately 12,000 hospital, reference, and large-clinic laboratories in the United States. The Wisconsin State Laboratory of Hygiene is approved by the Health Care Financing Administration to provide proficiency testing in all specialties and subspecialties. The focus of the program is to provide highly specialized service and support to a limited number of participants in order to assess intralaboratory performance correctly. We report the findings over the four proficiency-testing events in 1991 for the subspecialty of routine chemistry, which serves approximately 470 participants. Failure rates for individual analytes on single proficiency testing events ranged from 0% to 13%. After four events or one year, if the mandated evaluation criteria and failure rules were strictly applied, as many as 11% of the laboratories could have found themselves involuntarily suspended from offering all routine chemistry testing.

Chemistry, Clinical↗

PubMed1990

1990 Medicare/CLIA final rules for proficiency testing: minimum intralaboratory performance characteristics (CV and bias) needed to pass.

On March 14, 1990, the Centers for Disease Control and the Health Care Financing Administration published criteria for defining minimum performance in proficiency testing (PT). Using our previously described computer modeling technique, we determined the likelihood of passing PT under the new rules. The model relates combinations of intralaboratory CV and bias to PT performance criteria. For example, a laboratory with a bias of zero and an internal CV of 5% will pass a 10% fixed-limit PT criterion (i.e., the criterion for glucose analyses) 98% of the time when five samples are used. The model provides similar analyses for all PT criteria and all relevant combinations of CV and bias. The probability of passing PT decreases as the number of analytes tested increases, i.e., from 98% to 37% as the number of analytes increases from 1 to 20. A laboratory's internal CV has a greater effect on the outcome of PT than do the corresponding bias values. We conclude that a laboratory that operates with methods that have internal CVs less than or equal to 33% and biases less than or equal to 20% of the PT criteria will have a greater than 99% chance of passing PT.

Centers for Disease Control and Prevention, U.S.↗

PubMed1988

The relationship of intralaboratory bias and imprecision on laboratories' ability to meet medical usefulness limits.

The previously described computer modeling technic empirically develops quantitative relationships between intralaboratory performance, as characterized by individual laboratories' coefficients of variation (CVs) and biases, and clinical "medical usefulness limits." These limits determine the magnitude of total analytic error, the combined effects of CV and bias that can be tolerated by the clinician. The computer model delineates all combinations of CV and bias compatible with specified medical usefulness limits. Both CV and bias are critical in determining a laboratory's ability to meet medical usefulness limits. For example, a laboratory with a 6% CV and zero bias will meet the +/- 10% or less total analytic error (medical usefulness limit) 90% of the time. If the medical usefulness limit is expanded to +/- 15%, a laboratory with a 6% CV can tolerate coexisting relative biases of up to 4% and still meet this limit 95% of the time. Plots of the limiting values for combinations of intralaboratory CV and bias are given that allow the laboratory's results to fall within medical usefulness limits of 2, 5, 10, 15, and 20%.

Clinical Laboratory Techniques↗

PubMed1988

Use of computer modeling to predict the magnitude of intralaboratory error tolerated by proposed CDC interlaboratory proficiency testing performance criteria.

In December 1987, the Centers for Disease Control (CDC) proposed to the Health Care Financing Administration revised criteria for evaluating participating laboratories' performance in proficiency-testing programs. If these criteria are accepted, they will become the minimum standard for all regulatory proficiency-testing programs. To evaluate a laboratory's performance in a clinical chemistry proficiency-testing program, the CDC proposed a combination of the use of fixed limits, multiples of the interlaboratory group standard deviations, and absolute values. In addition, laboratories would be required to meet 70% of the most recent proficiency-testing challenges. Because the purpose of regulatory proficiency testing is to identify poorly performing laboratories, it is essential that regulators be aware of the relationship between the regulatory criteria and the actual magnitude of intralaboratory error they tolerate. Through computer simulation, we determined for 21 chemistry analytes the amount of intralaboratory error tolerated by the CDC-proposed criteria. We evaluated the effectiveness of the proposed criteria by comparing the levels of total intralaboratory error permitted by a proficiency-testing program by using the CDC criteria with actual currently achievable levels of performance and defined medical usefulness needs. The proposed CDC criteria were too lenient for six analytes, about correct for seven, and too stringent on two; no medical usefulness limits were available for six.

Blood Chemical Analysis↗

PubMed1988

Use of alternative rules (other than the 1(2)s) for evaluating interlaboratory performance data.

Previous studies have documented the ineffectiveness of using either the group mean +/- 2 group standard deviations (SD) or the 1(2)s rule as the standard of acceptable performance in evaluating interlaboratory proficiency testing (PT) data. Using computer simulation of PT data, we evaluated the efficiency of 244 alternatives to the 1(2)s rule, all based on the PT population's mean and SD. Using the traditional interlaboratory PT format, we determined the ability of each rule to correctly identify both good and deficient intralaboratory performance. The rules are based on results from one to five PT samples "analyzed" at the same time. Because the effectiveness of the criteria set for acceptable performance in a PT program is influenced by the population SD, each rule's capabilities were examined for PT populations with interlaboratory SDs ranging from 1% through 10% of the population mean value. All rules achieve their maximum efficiency over a narrow range of interlaboratory SDs. For PT evaluations of intralaboratory performance to be optimally effective, selection of the rule must be based on the SD of the PT population.

Computers↗

PubMed1988

Proficiency testing programs--promises, progress, and problems. A 40-year prospective.

Proficiency testing as a means of interlaboratory improvement is over 40 years old. Proficiency testing as a tool of the regulatory process is nearly 20. While successful, ie, the quality of laboratory work in the United States has improved significantly during the past 40 years, this union of regulation and education continues to be a marriage of unequals. Clearly, those who started the professional laboratory community on the road to interlaboratory proficiency testing envisioned a promise of improved quality. The promise is fulfilled; even on an interlaboratory basis, the quality of work (precision) approaches or even exceeds the clinicians' requirements for medically useful data. Major problems continue to exist: the prospect of regulating a vast population of "physicians' office laboratories" presents new challenges for program providers, and as methodologies in laboratories have improved, some of the long-held systems of grading and performance evaluation are beginning to fail. The failure is not one of intent, logistics, or statistics. Proficiency testing, as a fine sharp tool, is corrupted when applied in a situation in which it is ill suited to perform. New knowledge will renew the promise and allow the professional, clinical laboratory community to progress.

Clinical Laboratory Techniques↗

PubMed1988

An evaluation of the ability of proficiency testing programs to determine intralaboratory performance. Peer group statistics vs clinical usefulness limits.

Proficiency testing programs use two basic approaches for evaluating participant performance: (1) state-of-the-art data based on the interlaboratory group mean and group SD, or the 1(2)s rule; and (2) fixed limits or assigned intervals around the true value. Our previously described computer model emulates the state-of-the-art approach by simulating the performance of 401 laboratories in interlaboratory proficiency testing programs using the 1(2)s rule. The use of fixed limits to evaluate laboratory performance can be simulated by simple probability calculations. We compare the 1(2)s rule and fixed limits on the basis of their ability to identify actual intralaboratory performance correctly using efficiency. In both cases, the maximum efficiency never exceeds 91% and depends on population SD or fixed limit, specific requirements for medical usefulness, and the prevalence (10%) of poorly performing laboratories in the proficiency testing population. This calculation gives some insight into why neither of these approaches adequately differentiates between good and bad intralaboratory performance.

Computer Simulation↗

PubMed1987

Interlaboratory proficiency-testing programs: a computer model to assess their capability to correctly characterize intralaboratory performance.

We developed a computer model of an interlaboratory survey program to study the ability of proficiency testing (PT) programs to detect intralaboratory errors (total, random, and systematic). It uses a base interlaboratory PT population of 400 laboratories and one test laboratory each with uniquely defined intralaboratory characteristics, i.e., mean, standard deviation (SD), and bias. A gaussian random-number generator uses these parameters to simulate 401 test results analogous to the analysis of one PT sample by each laboratory. The test laboratory's intralaboratory error is expressed as various combinations of bias and coefficient of variation (CV); its simulated survey result is evaluated by a performance criterion derived from the group statistics. To eliminate statistical artifacts, the computer model repeats the complete simulation process 400 times and determines the percentage of the test laboratory's results that fail to meet a specified performance criterion. The computer model can use assigned values or actual intralaboratory data.

Computer Simulation↗

PubMed1987

Ability of the 1(2)s rule to detect substandard performance in interlaboratory proficiency testing.

The most common evaluation criterion used by regulatory and voluntary interlaboratory proficiency testing (PT) programs to judge the quality of chemistry results is based on the group mean +/- 2.0 standard deviations (SD), the 1(2)s rule. The mean and SD are those of the selected PT population. Through computer simulations we have quantified, for the first time, the ability of the 1(2)s criterion to detect "acceptable" as well as substandard ("unacceptable") intralaboratory performance by PT. We found that the 1(2)s rule correctly identifies "acceptable" performance--i.e., low intralaboratory coefficient of variation (CV) values and small bias values--greater than 90% of the time. However, this criterion fails to detect laboratories with "unacceptable" bias and (or) CV at least 23% of the time. The high failure rate of the 1(2)s rule raises serious questions about its appropriateness for use by PT programs.

Computer Simulation↗

PubMed1987

An assessment of the use of fixed limits to characterize intralaboratory performance by proficiency testing.

Currently, proficiency-testing (PT) programs are expanding the use of fixed limits to evaluate interlaboratory performance. These limits are an attempt to relate total allowable intralaboratory analytical error and performance in a PT program. Fixed limits are a means of counteracting the effects of overly stringent performance requirements derived from the interlaboratory group mean and standard deviations (SD) achieved by today's very precise analytical systems. Our previously described computer model of a PT program is used to delineate the quantitative relationship between the magnitude of intralaboratory coefficient of variation (CV) and bias that is compatible with fixed interlaboratory limits of 5, 10, 15, and 20% relative error. However, fixed limits alone do not fully characterize intralaboratory performance. The efficiency, or the percentage of times a laboratory's PT results are required to fall within the fixed limit, also must be considered when criteria for PT performance are being set. For example, a 10% fixed limit and a 80% efficiency will allow laboratories having CV-bias combinations from 8%-0%, 7%-4%, etc., to pass PT. The use of a 95% efficiency will allow CV-bias combinations from 4.8%-0%, 3.5%-4%, etc., to pass PT. We give a figure that depicts all possible combinations for four fixed limits and six efficiencies.

Computer Simulation↗

PubMed1986

Quality control and quality assurance.

The office laboratory's need for quality is no different from that of any other clinical laboratory. If patients are to receive the benefit of physician's office testing, reliable, high-quality laboratory results are essential. To achieve this, the physician's office laboratory must have an adequate quality assurance program. Several fundamental components of such a program have been addressed in this article: procedure manuals, record-keeping, maintenance logs, quality control charts, participation in proficiency testing, and laboratory inspection. If your state's regulations do not yet require these activities in the physician's office laboratory, they soon will! A successful laboratory's quality assurance program will provide the following assurances. (1) Quality practices are established and followed by all personnel involved with the testing in the laboratory. (2) The technologist performing the test will know when systems and instruments are working properly and the patients' results are reliable. (3) High-quality information needed by the physician interpreting or evaluating patient laboratory results will be generated. (4) A set of written records is available demonstrating to the inspector that uniform and acceptable protocols have been established and are practiced in the laboratory. One short article cannot provide all the specifics for a laboratory's quality assurance program. The manufacturers and suppliers of instruments and reagents should be able to provide support in the following areas: calibration, type of controls to be used, development of a control chart, required maintenance procedures, establishment of accuracy and precision, and troubleshooting. If they cannot or will not, your laboratory should, perhaps, consider an alternative vendor to supply instrumentation and/or reagents. Additionally, resources such as the professional organizations, consultants, other clinical laboratories, and the inspectors or certifying agencies should also be considered in developing a comprehensive quality assurance program.

Ambulatory Care↗

PubMed1985

Adequacy of interlaboratory precision criteria in measuring intralaboratory performance.

We compared the predictive value of the various criteria used for grading pH and blood-gas measurements in interlaboratory proficiency testing programs with performance as determined from actual intralaboratory quality-assurance data. The evaluation criteria were the two-standard-deviation interval (2 SDI) proposed by the College of American Pathologists (CAP); CAP's proposed fixed criteria; and the fixed criteria of the American Thoracic Society (ATS). These were compared with 95% confidence limits derived from the individual laboratories' actual intralaboratory data. We found that the CAP's most-stringent criterion (2 SDI) overestimated the number of outliers (unacceptable results) for PCO2 and PO2, whereas the proposed fixed limits underestimated them. For pH and PCO2 the ATS's limits, which are more stringent, more closely match the individual laboratory's actual performance as measured by conventional (mean +/- 2 SD) intralaboratory quality-assurance practices.

Blood Gas Analysis↗

PubMed1985

Alternative statistical approach to evaluating interlaboratory performance.

We report a new technique for realistic assessment of laboratory performance as measured by proficiency testing. Interlaboratory results accumulated from 129 participants during 18 months provide the baseline data from which we established "state-of-the-art" performance criteria for three ranges of pH, pCO2, and pO2. By concurrent use of two statistical measurement techniques, the cumulative percentile rank and the algebraic and absolute mean error, laboratories can accurately evaluate their performance in terms of acceptable state-of-the-art criteria, total error, or medical usefulness. The approach facilitates assessment of the nature of the errors that have led to inferior performance and identification of probable areas where improvement is possible. If criteria based on regulatory standards or medical usefulness goals are included, the system can provide a basis for licensure or professional quality improvement.

Blood Gas Analysis↗

PubMed1979

Model course for retraining inactive medical technologists.

In the belief that there is an ongoing need for the retraining of inactive medical technologists, the University of Wisconsin-Madison initiated such a program in 1978. A model course in routine chemistry and instrumentation was developed, presented, and evaluated. This model is described in detail. Results indicate that most inactive medical technologists can regain competence. The program and individual courses are designed so they can be easily replicated or adapted for use by others interested in retraining inactive medical tehcnologists.

Clinical Competence↗