SEARCH · PubMed Health

Results for “Software Validation”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2Linked to original sources

PubMedDate not supplied

Evaluation of clinical decision aids--more to think about.

Validation and clinical testing of medical decision aids is a hot topic at present. We state a few main points which, in our opinion, have not received the attention they deserve. Our main concerns have to do with the statistical design and analysis of field evaluation studies; with long-term effects; and with the interplay between medicine and the software industry. The points represent our reaction to a recent symposium on the topic, published in this special issue. We take the opportunity to supplement the papers at hand in various ways and, where we see a risk of misconceptions or poor practice becoming cemented, try to point out the proper course.

Decision Making, Computer-Assisted↗

PubMedDate not supplied

The evaluation of clinical decision support systems: what is necessary versus what is interesting.

The evaluation of clinical decision support systems has long been recognized as an important part of the overall field of medical computing, as well as a complex and varied field in its own right. There are a number of purposes for which such an evaluation might be performed. This paper focuses on one of those purposes, the assurance of safety when preparing to release the system for general use. The assurance of safety involves two major stages: (1) a comprehensive validation of the system's knowledge and advice, and (2) a period of use which exposes the system's advice and interface to the clinical environment.

Decision Making, Computer-Assisted↗

PubMed1992

Development of a computer application for headache diagnosis: the Headache Diagnostic System.

A computer-assisted decision support system for headache classification and diagnosis is presented. The system provides: structured headache data collection, automated data quality control and management, and automated headache classification. The system is based upon headache classification and diagnostic criteria recently developed by the International Headache Society. Development, logical structure and validation of the system are discussed. The system is shown to provide a general improvement in headache classification reliability and has utility for validation and improvement of present headache classification criteria.

Adult↗

PubMed1992

Automated recognition of corrupted arterial waveforms using neural network techniques.

A data acquisition system that automatically discards corrupted or undesirable signals would save untold hours of drudgery for researchers. Continuous recording of variables to provide detailed behavior patterns generates huge amounts of raw data. Unfortunately waveforms usually require visual inspection for isolating desired behavior or validating signal integrity. This tedious and time-consuming step can potentially be eliminated using a novel computer science technique. We have trained a simulated neural network to recognize corrupted arterial pressure waveforms. Our system can now evaluate the validity of the arterial waveform without human intervention with an average false positive error rate of 2.2% and an average false negative error rate of 12.6%.

Artifacts↗

PubMed1989

Cellular viability in human tumor micro-organ cultures: in situ quantitation by image processing.

At present, cytotoxicity measurements using the fluorescent cytoprint assay are based on achieving complete cell death in cultures of drug-sensitive tumors. Thus, the usefulness of the assay would be extended if partial effects of chemotherapeutic drugs could be quantified. In this study, we addressed the issue by developing and validating a thresholding algorithm for automatic image processing that can be used to quantify the areas occupied by viable (i.e., fluorescent) micro-organs in the culture.

Drug Screening Assays, Antitumor↗

PubMed1992

Validation of the medical expert system PNEUMON-IA.

The present study validates the expert system PNEUMON-IA. The aim of PNEUMON-IA is assessing the etiology of community-acquired pneumonias from clinical, radiological, and laboratory data obtained at the onset of the disease. Validation was performed using data from medical records of 76 patients with proven clinical diagnosis of pneumonia. The etiological diagnoses provided by PNEUMON-IA were compared to those established by five specialists unrelated to the development of the expert system. For each etiological possibility, both PNEUMON-IA and the experts provided a causal possibility, expressed as a linguistic label (i.e., "almost impossible"). Linguistic labels were then converted to numeric values. In the majority of cases, an etiological diagnosis was unavailable to be used as a gold standard. To overcome this limitation, distances between arrays of etiological possibilities given by specialists and by PNEUMON-IA were considered as an agreement measure between diagnoses. Cluster analysis based on those distances was used to classify PNEUMON-IA among experts. Results showed the same differences between specialists and PNEUMON-IA as among the specialists themselves. The method used to validate PNEUMON-IA could prove useful to assess the performance of expert systems in fields where no gold standard is available.

Adolescent↗

PubMed1991

[Bacterio-expert: an integrated system for assisting in the validation of antibiotic sensitivity tests. Retrospective application in 4053 Staphylococcus].

Bacterio-expert is a simple expert system for assisting in the validation of antibiotic sensitivity testing. This system is incorporated in a data acquisition and editing program for bacteriologic test (Bacterio program written in Turbo-Pascal for personal computer users by the same authors). The principles of this system are explained and results with 4,053 antibiotic sensitivity tests on Staphylococcus aureus isolates are reported. Approximately 10% of tests required corrections.

Anti-Bacterial Agents↗

PubMed1988

A performance evaluation of the expert system ANEMIA.

This paper reports the results of an evaluation study of the current level of performance given by ANEMIA, a knowledge-based consultation system addressing the clinical problem of managing anemic patients. ANEMIA was developed on a mainframe using the AI programming scheme EXPERT and then translated into a version running on a personal computer. At present the system is able to provide assistance in the diagnosis and management of 65 disease entities. After extensive local testing of accuracy, completeness, and consistency of the knowledge base included into ANEMIA, we designed a study to evaluate whether the system is able to appropriately mirror also the reasoning of well-known hematologists other than those who provided the knowledge. We were also interested in testing whether there were conflicting opinions among hematologists. Thus, we designed a validation study in which ANEMIA's performance could be compared with that of six hematologists and the interexpert consensus evaluated. ANEMIA's overall performance was judged acceptable in 87% (26/30) of the cases, while expert evaluators agreed with their colleagues in 90% (27/30) of them. A low interexpert consensus was found: considering the ratings given by different hematologists to the same ANEMIA performance, complete agreement occurred only 47% of the time.

Adult↗

PubMed1989

A computer-based interview system for patients with back pain. A validation study.

A microcomputer-based system has been designed to interview patients with a view to investigating and establishing common syndromes of back and leg pain. In a randomized crossover validation study, 50 consecutive outpatients were interviewed by the computer and had a conventional clerking by a doctor. The conventional clerking made minor errors in 3.75% of questions answered and major errors in 0.90%. The computer made minor errors in 6.75% of questions and major errors in 5.45%. The majority of the computer errors were due to inadequate question design. These have been corrected, and it is anticipated that the computer will now have an overall rate of 94% correct answers and be sufficiently accurate to pursue the aim of clinical syndrome identification.

Back Pain↗

PubMed1991

Validation, clinical trial, and evaluation of a radiology expert system.

The PHOENIX Radiology Consultant is a rule-based expert system which assists physicians in planning radiological work-up strategies. This article describes the methods used to create and validate the system's knowledge base. The feasibility and acceptability of PHOENIX were tested for two years in a clinical trial. During this period, the system was used 1,421 times, an average of 13.7 times per week, primarily by medical students and nonradiologist physicians. Much of the system's use occurred at night and on weekends, when the radiology department was not fully staffed. Several physicians were enlisted to further evaluate the utility of the system. The results of their evaluation indicate that an expert system that helps physicians select diagnostic-imaging studies can serve as a useful and informative component of a radiology information system, and is particularly useful for medical students and physicians in training.

Algorithms↗

PubMed1991

Analysis of brain and cerebrospinal fluid volumes with MR imaging. Part I. Methods, reliability, and validation.

A computerized system was developed to process standard spin-echo magnetic resonance (MR) imaging data for estimation of brain parenchyma and cerebrospinal fluid (CSF) volumes. In phantom experiments, the estimated volumes corresponded closely to the true volumes (r = .998), with a mean error less than 1.0 cm3 (for phantom volumes ranging from 5 to 35 cm3), with excellent intra- and interobserver reliability. In a clinical validation study with actual brain images of 10 human subjects, the average coefficient of variation between observers for the measurement of absolute brain and CSF volumes was 1.2% and 6.4%, respectively. The intraclass correlations for three expert operators is greater than .99 in the measurement of brain and ventricular volumes and greater than .94 for total CSF volume. Therefore, the authors believe that their technique to analyze MR images of the brain performed with acceptable levels of accuracy and reliability and that it can be used to measure brain and CSF volumes for clinical research. This technique could be helpful in the correlation of neuroanatomic measurements to behavioral and physiologic parameters in neuropsychiatric disorders.

Algorithms↗

PubMedDate not supplied

A strategy for development of computerized critical care decision support systems.

It is not enough to merely manage medical information. It is difficult to justify the cost of hospital information systems (HIS) or intensive care unit (ICU) patient data management systems (PDMS) on this basis alone. The real benefit of an integrated HIS or PDMS is in decision support. Although there are a variety of HIS and ICU PDMS systems available there are few that provide ICU decision support. The HELP system at the LDS Hospital is an example of a HIS which provides decision support on many different levels. In the ICU there are decision support tools for antibiotic therapy, nutritional management, and management of mechanical ventilation. Computer protocols for the management of mechanical ventilation (respiratory evaluation, ventilation, oxygenation, weaning and extubation) in patients with adult respiratory distress syndrome ((ARDS) have already been developed and clinically validated at the LDS Hospital. These protocols utilize the bedside intensive care unit (ICU) computer terminal to prompt the clinical care team with therapeutic and diagnostic suggestions. The protocols (in paper flow diagram and computerized form) have been used for over 40,000 hours in more than 125 adult respiratory distress syndrome (ARDS) patients. The protocols controlled care for 94% of the time. The remainder of the time patient care was not protocol controlled was a result of the patient being in states not covered by current protocol logic (e.g. hemodynamic instability, or transport for X-Ray studies). 52 of these ARDS patients met extra corporal membrane oxygenation (ECMO) criteria. The survival of the ECMO criteria ARDS patients was 41%, four times that expected (9%) from historical data (p less than 0.0002).(ABSTRACT TRUNCATED AT 250 WORDS)

Attitude of Health Personnel↗

PubMedDate not supplied

The development of a comprehensive, institution-based patient risk evaluation program: II. Validity and reliability of questionnaire data.

The accuracy of historical information derived from self-administered questionnaires must be confirmed. We report the results of studies conducted to assess the reliability and validity of data collected from a comprehensive cancer risk factor questionnaire developed at The University of Texas M.D. Anderson Cancer Center. A comparison of the basic demographic data of a randomly selected sample of 80 respondents and 70 nonrespondents revealed no fundamental ethnic or socioeconomic differences. We verified self-reported past illnesses, surgical procedures, and cancers by reviewing 72 patient charts, using stringent diagnostic criteria for verification. We noted substantial agreement between self-reported and documented illnesses and operations. With the exception of nine patients who misclassified metastatic disease, the verification of primary cancers was excellent. We determined reliability by interviewing 50 of these patients by telephone. Questions with a dichotomous outcome (e.g., smoking status) were reliably answered; however, those requiring quantification (e.g., amount of alcohol consumed) were less accurately reported on interview. While we recognize the limitations of self-administered questionnaires, we believe this program will develop into a comprehensive, standardized, easily accessible patient risk factor data base.

Cancer Care Facilities↗

PubMed1992

Linear models for the prediction of stature from foot and boot dimensions.

Estimation of stature from the dimensions of foot or shoeprints has considerable forensic value in developing descriptions of suspects from evidence at the crime scene and in corroborating height estimates from witnesses. This study extends the findings of previous researchers by exploring linear models with and without gender and race indicators, and by validating the most promising models on a large, recently collected military database. Boot size and outsole dimensions are also examined as predictors of stature. The results of this study indicate that models containing both foot length and foot breadth are significantly better than those containing only foot length. Models with race/gender indicators also perform significantly better than do models without race/gender indicators. However, the difference in performance is slight, and the availability of reliable gender and race information in most forensic situations is uncertain. Analogous results were obtained for models utilizing boot size/width and outsole length/width, and in this study these variables performed nearly as well as the foot dimensions themselves. Although the adjusted R2 values for these models clearly reflect a strong relationship between foot/boot length and stature, individual 95% prediction limits for even the best models are +/- 86 mm (3.4 in.). This suggests that models estimating stature from foot/shoe-prints may be useful in the development of subject descriptions early in a case but, because of their imprecision, may not always be helpful in excluding individual suspects from consideration.

Anthropometry↗

PubMed1986

An automated technique for analysis of current transitions in multilevel single-channel recordings.

Detailed kinetic studies of ion channel gating are best carried out using the patch-clamp technique which permits the measurement of the ionic current through individual channels. Typical patch-clamp recordings show the current signal, in the form of a sequence of rectangular pulses (analogous to a random telegraph signal), riding on slow baseline drift, partially obscured by high-frequency noise and distorted by filtering. In order to analyze such recordings, we have developed a set of interactive Pascal programs based on a feature-detection algorithm capable of identifying current transitions in multiple-channel recordings in the presence of substantial levels of noise and drift. Software operation is largely automated but includes provisions for examination and correction of the output. The software was optimized and systematically evaluated using simulated data with variable amounts of noise and drift. Results indicate that satisfactory performance is obtained for signal-to-noise ratio as low as four even with uncommonly large baseline drift. Steady-state processing speeds varied from 1,000 to 4,000 samples per second depending on data complexity.

Automation↗

PubMed1990

Experience with gated cardiac software phantoms for quality control of applications programmes.

Ten gated cardiac software phantoms, representing normal and abnormal clinical conditions, were transferred to 9 different computer systems and tested with 11 cardiac programmes. Problems of inappropriate data format were encountered when analysing the phantoms on other systems. The global left ventricular ejection fraction (LVEF) values resulting from the different programmes were compared. Significant programme differences were found, the programmes falling generally into two groups. Few LVEF outlier values were identified. Full functional assessment of cardiac phantoms requires a set of different views which together form the phantom. Application of software phantoms for programme assessment and training requires prudence and attention to the acquired data format, frame time, and gating method for the appropriate use of phantoms.

Computer Simulation↗

PubMed1991

Designing an outcome-oriented computer decision-support system for cardiovascular ICU--a preliminary report.

This paper describes the conceptual framework and preliminary results of an outcome-oriented decision-support system prototype for the cardiovascular intensive care unit (CVICU). The major characteristics of this design include: (1) its problem-based approach to solving clinical problems; (2) an integrated structure with the hospital information system in terms of its data, model and knowledge bases; (3) proposed alternative modes of interaction that include monitoring and critiquing; (4) and research modules that design, manage, and analyze outcome-based clinical studies. At present, an initial prototype has been implemented on a PC as a set of modules accessible from a main menu. The structural framework of the overall system is fairly well defined but only limited quantitative, statistical and expert knowledge has been captured. The second phase of the project involves porting the prototype to a Unix workstation environment, refining and adding models to the model base, expanding its knowledge bases, reasoning capability, and testing the prototype with actual clinical cases in a real-time fashion.

Alberta↗

PubMed1991

A new microcomputer software system evaluation paradigm: the medical perspective.

The new fourth-generation software has enormously eased the burden of computing for users, but they have also created a confusing, difficult problem in software evaluation and selection. Therefore it is argued that a sound and complete evaluation paradigm is a key element in an efficient and effective software system design and use process. While much has been written about software evaluation in general, in the medical field, the guidances and recommendations previously provided are too general to be of practical use. Considering also the other weaknesses of conventional evaluation paradigms we have decided to develop a more adequate one which will have a solid theoretical framework, specific guidances, strict and well-defined taxonomic space, and a fair ranking approach. In the present paper we will therefore introduce our new evaluation paradigm and show its applicability in evaluation and selection of medical software systems according to their usability.

Classification↗