SEARCH · PubMed Health

Results for “Software Validation”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15Linked to original sources

PubMed1994

The role of quality assurance in computer inspections.

Changing technology affords the Quality Assurance auditor with the challenge of applying computer validation concepts to a variety of computer system types. In addition, these technology changes have caused the developers role to change as well. In an innovative research facility, the developer may include an in-house professional group, a vendor, or an end-user. With these issues in mind, the QA auditor needs a tool to accomplish the task of inspecting systems as they are being created. The prospective inspection process is the tool for accomplishing this task. This inspection involves the QA auditor's involvement in the development of a new system, as a member of the development team, from the initial creation through the implementation of the computer system. This presentation will focus on illustrating the steps in conducting the prospective inspection process, from expected deliverables and document reviews to final report and management notification. The benefits of QA involvement in the development process will also be discussed.

Clinical Laboratory Information Systems↗

PubMed2004

Application of GEANT4 radiation transport toolkit to dose calculations in anthropomorphic phantoms.

In this paper, we present a novel implementation of a dose calculation application, based on the GEANT4 Monte Carlo toolkit. Validation studies were performed with an homogeneous water phantom and an Alderson-Rando anthropomorphic phantom both irradiated with high-energy photon beams produced by a clinical linear accelerator. As input, this tool requires computer tomography images for automatic codification of voxel-based geometries and phase-space distributions to characterize the incident radiation field. Simulation results were compared with ionization chamber, thermoluminescent dosimetry data and commercial treatment planning system calculations. In homogeneous water phantom, overall agreement with measurements were within 1-2%. For anthropomorphic simulated setups (thorax and head irradiation) mean differences between GEANT4 and TLD measurements were less than 2%. Significant differences between GEANT4 and a semi-analytical algorithm implemented in the treatment planning system, were found in low-density regions, such as air cavities with strong electronic disequilibrium.

Algorithms↗

PubMed1996

The performance of the knowledge-based system VALAB revisited: an evaluation after five years.

In 1988, inundated by the tedious work of validation of laboratory reports in a large hospital biochemistry laboratory, we designed VALAB, a knowledge-based system specially dedicated to this iterative function. Coping at first with a few biochemical tests, the program has been progressively expanded to forty-five common chemical tests. Simultaneously some new rules have been introduced to "weight" the conclusion in different circumstances and rules taking into consideration some clinical data have also been written. Moreover the program moved to other disciplines, pH and blood gases, haematology and coagulation. Accordingly the evaluation protocol has been modified, incorporating a new step, the consensus decision of the pathologists, operating within the initial protocol and based upon the various criteria of epidemiology. These major changes and improvements have led us to check and describe again the performance of this updated VALAB knowledge-based system.

Artificial Intelligence↗

PubMedDate not supplied

Pattern recognition in health insurance claims databases.

Information in claims databases resides in data patterns rather than in data elements. Finding this information requires new terminology, a willingness to pose questions of form rather than specific hypotheses, and a quality control system that elevates the correctness of data relations above the validity of single facts. The language of claims data is a newspeak of CPT (Current Procedural Terminology), HCPCS (Health Care Financing Agency Common Procedure Coding System), ICD (International Classification of Disease), and NDC (National Drug Codes) for pharmaceutical codes. The techniques of pattern discovery are really ways of asking the data for classes of relations, and they vary in their reliance on external information. Sometimes, the question is entirely constrained by preceding factors. Other times we may recast the natural history of disease into a claims context and ask the data to give us the shape of disease evolution. We can use highly automated systems to evaluate the relations between prespecified factors, or empirical techniques to search out common relations that we have not specified in advance. Using massive data sets requires that quality control corresponds to the nature of the high-level information that we derive from large databases.

Databases as Topic↗

PubMedDate not supplied

Computerized decision support for concurrent utilization review using the HELP system.

OBJECTIVE: Development and evaluation of computerized concurrent utilization review (UR) support taking advantage of a clinically rich computerized patient database. DESIGN: The Automated Support System for Utilization Review (ASSURE) applies the Appropriateness Evaluation Protocol (AEP) Day of Care criteria to computerized patient data in the HELP hospital information system. This paper reports the development, verification, and validation of ASSURE. MEASUREMENTS: Implementation correctness was verified by measuring agreement with a nurse reviewer, using separate sample sets for all 20 criteria for a total of 560 current inpatients. Usefulness in detecting inappropriate days of care was validated by two nurse reviewers who were crossed with manual and computer-assisted review methods in a blocked design for 168 current inpatients. Agreement with reviewers, sensitivity, specificity, positive predictive value, and negative predictive value were measured. RESULTS: Agreement was very good for satisfaction of criteria, and good for appropriateness of day of care. A patient day identified by ASSURE as potentially inappropriate would be twice as likely to be judged inappropriate by a reviewer as a randomly selected patient day. Review of the 10% of patient days identified as potentially inappropriate by ASSURE would identify approximately 21% of the inappropriate days of care. CONCLUSION: ASSURE is a clinically useful tool for screening adult acute care patients for inappropriate days of care, and promises to make a major contribution to reducing health care costs. The prognosis for successful routine clinical use is good.

Artificial Intelligence↗

PubMed1996

A program for the user-independent computation of the correlation dimension and the largest Lyapunov exponent of heart rate dynamics from small data sets.

We propose a specially optimized computer program for the user-independent calculation of the correlation dimension D and the largest Lyapunov exponent L of heart rate dynamics on the basis of only 1024 electrocardiographically recorded RR intervals (heartbeat intervals). The validity of our program was established by analyzing a set of artificial standard signals. Our norm values of the correlation dimension (D = 5.37 +/- 0.62) and the largest Lyapunov exponent (L = 0.561 +/- 0.037 bits/beat) of RR dynamics, obtained from 79 healthy adults aged 26.3 +/- 4.8 years, were independent of gender and age; D and L correlated slightly with each other (Pearson correlation coefficient r = 0.26). Short-term reliability, tested for 25 of our subjects by two successive recordings, was fair: the intraclass correlation coefficients (ICCs) were 0.45 and 0.41 for D and L of RR dynamics, respectively. However, long-term reliability, tested for eight of our subjects by ten weekly recordings, was acceptable for L (ICC = 0.40) but not for D (ICC = 0.01). These results permit group comparisons on the basis of single measurements of L of RR dynamics. A reliable differentiation between young healthy individuals requires four measurements of L.

Adult↗

PubMedDate not supplied

Use of an artificial neural network (ANN) for classifying nursing care needed, using incomplete input data.

BACKGROUND: In German nursing insurance, the act of classifying the client into four categories of disability is based on legally defined distinct criteria. When classifying deceased persons it is often impossible to collect all the required information. PRIMARY OBJECTIVE: We aimed to determine the ability of an artificial neural network (ANN) to calculate the category of disability, to investigate the response of the ANN to input items of different nature, quantity and data quality, and to estimate the minimum number of training data required. RESEARCH DESIGN: The investigation was conducted as a retrospective observational study. METHODS AND PROCEDURES: The analysis was based on routine records of 14000 adult clients of the nursing insurance. Several ANNs were trained, varying nature, number and quality of the input items as well as the size of the training data set. Each ANN's classification competence was tested on independent validation data, judging the ANN's conformance to the result of the individual expert assessment, using kappa statistics. MAIN RESULTS: Fed with all 30 input items available, the net classified 80% of cases correctly (weighted kappa = 0.78). Using three input items, weighted kappa was 0.63. Severe misclassification (deviation by more than one category in either direction) ranged between 0.2% (all 30 input items) and 3.7% (3/30 items). The less complete the individual input items were, the less accurate was the net's estimate. A 20% rate of missing values was well tolerated. A training set comprising 500 cases was adequate. CONCLUSIONS: The input item set inherits redundancy. The ANN's ability to correctly respond to subsets of input items makes it a powerful tool in quality control. In the categorization of deceased persons when only an incomplete input item set is available, the ANN can achieve satisfactory results.

Activities of Daily Living↗

PubMedDate not supplied

Monitoring expert system performance using continuous user feedback.

OBJECTIVE: To evaluate the applicability of metrics collected during routine use to monitor the performance of a deployed expert system. METHODS: Two extensive formal evaluations of the GermWatcher (Washington University School of Medicine) expert system were performed approximately six months apart. Deficiencies noted during the first evaluation were corrected via a series of interim changes to the expert system rules, even though the expert system was in routine use. As part of their daily work routine, infection control nurses reviewed expert system output and changed the output results with which they disagreed. The rate of nurse disagreement with expert system output was used as an indirect or surrogate metric of expert system performance between formal evaluations. The results of the second evaluation were used to validate the disagreement rate as an indirect performance measure. Based on continued monitoring of user feedback, expert system changes incorporated after the second formal evaluation have resulted in additional improvements in performance. RESULTS: The rate of nurse disagreement with GermWatcher output decreased consistently after each change to the program. The second formal evaluation confirmed a marked improvement in the program's performance, justifying the use of the nurses' disagreement rate as an indirect performance metric. CONCLUSIONS: Metrics collected during the routine use of the GermWatcher expert system can be used to monitor the performance of the expert system. The impact of improvements to the program can be followed using continuous user feedback without requiring extensive formal evaluations after each modification. When possible, the design of an expert system should incorporate measures of system performance that can be collected and monitored during the routine use of the system.

Expert Systems↗

PubMed1999

SuperStar: a knowledge-based approach for identifying interaction sites in proteins.

An empirical method for identifying interaction sites in proteins is described and validated. The method is based entirely on experimental information about non-bonded interactions occurring in small-molecule crystal structures. These data are used in the form of scatterplots that show the experimentally observed distribution of one functional group (the "contact group" or "probe") around another. A template molecule (e.g. a protein binding site) is broken down into structure fragments and the scatterplots, showing the distribution of a chosen probe around these structure fragments, are superimposed on the corresponding parts of the template. The scatterplots are then translated into a three-dimensional map that shows the propensity of the probe at different positions around the template molecule. The method is illustrated for l -arabinose-binding protein, complexed with l -arabinose and with d -fucose, and for dihydrofolate reductase complexed with methotrexate. The method is validated on 122 X-ray structures of protein-ligand complexes. For all the binding sites of these proteins, propensity maps are generated for four different probes: a charged NH+3nitrogen, a carbonyl oxygen, a hydroxyl oxygen and a methyl carbon atom. Next, the maps are compared with the experimentally observed positions of ligand atoms of these types. For 74% of these ligand atoms (84% of the solvent-inaccessible ones) the calculated propensity of the matching probe at the experimental positions is higher than expected by chance. For 68% of the atoms (82% of the solvent-inaccessible ones) the propensity of the matching probe is higher than that of the other three probes. These results indicate that the approach generally gives good predictions for protein-ligand interactions. The potential applications of the propensity maps range from an aid in manual docking and structure-based drug design to their use in pharmacophore development.

Artificial Intelligence↗

PubMed1998

Mean regional cerebral blood flow images of normal subjects using technetium-99m-HMPAO by automated image registration.

UNLABELLED: The purpose of this study was twofold: to calculate relative uptake values for 99mTc-HMPAO in various regions of the normal brain after alignment and registration to a standard shape and size, and to validate the automated image registration (AIR) program for SPECT-to-SPECT transformation. METHODS: Thirty subjects took part in this study. Technetium-99m-HMPAO brain SPECT and x-ray-CT scans were acquired. SPECT images were normalized to an average activity of 100 counts/pixel. Intersubject accuracy was evaluated on brain images of 17 normal subjects (mean age = 64.9 +/- 8.7 yr). These images were aligned and registered to a standard size and shape with the help of AIR. Realigned images were overlaid on reference images to determine the overlap areas. Intrasubject accuracy was evaluated by realigning 20 degree rotated brain images with an index calculated as: overlap area/(overlap area + nonoverlap area). Anatomical variability between realigned target and reference images was evaluated by measurements on corresponding x-ray-CT scans, realigned using transformations that were established by the SPECT images. Realigned brain SPECT images of 30 normal subjects (mean age = 50.7 +/- 18.7 yr), including those subjects examined in the accuracy validation study, were used to generate mean and s.d. images. Images based on the mean value of each voxel (n = 30) were compared with other mean images prepared by the human brain atlas (HBA) standardization technique on a voxel-by-voxel basis to generate T maps. RESULTS: Accuracy indices were 0.98 +/- 0.006 and 0.99 +/- 0.002 for the intersubject and intrasubject evaluations, respectively. The maximum anatomical variability was 4.7 mm after realignment. Paired Student's t-test comparisons of mean HBA and AIR images revealed statistically significant differences for the deep white matter, pons and occipito-temporal regions. These differences could be explained by variation in the population being studied and the protocol for data handling by AIR and HBA. CONCLUSION: AIR aligns and registers brain SPECT images with acceptable accuracy, without the necessity of MRI or x-ray-CT scans.

Brain↗

PubMedDate not supplied

A strategy for development of computerized critical care decision support systems.

It is not enough to merely manage medical information. It is difficult to justify the cost of hospital information systems (HIS) or intensive care unit (ICU) patient data management systems (PDMS) on this basis alone. The real benefit of an integrated HIS or PDMS is in decision support. Although there are a variety of HIS and ICU PDMS systems available there are few that provide ICU decision support. The HELP system at the LDS Hospital is an example of a HIS which provides decision support on many different levels. In the ICU there are decision support tools for antibiotic therapy, nutritional management, and management of mechanical ventilation. Computer protocols for the management of mechanical ventilation (respiratory evaluation, ventilation, oxygenation, weaning and extubation) in patients with adult respiratory distress syndrome ((ARDS) have already been developed and clinically validated at the LDS Hospital. These protocols utilize the bedside intensive care unit (ICU) computer terminal to prompt the clinical care team with therapeutic and diagnostic suggestions. The protocols (in paper flow diagram and computerized form) have been used for over 40,000 hours in more than 125 adult respiratory distress syndrome (ARDS) patients. The protocols controlled care for 94% of the time. The remainder of the time patient care was not protocol controlled was a result of the patient being in states not covered by current protocol logic (e.g. hemodynamic instability, or transport for X-Ray studies). 52 of these ARDS patients met extra corporal membrane oxygenation (ECMO) criteria. The survival of the ECMO criteria ARDS patients was 41%, four times that expected (9%) from historical data (p less than 0.0002).(ABSTRACT TRUNCATED AT 250 WORDS)

Attitude of Health Personnel↗

PubMed2002

Quantitative evaluation of proteins in one- and two-dimensional polyacrylamide gels using a fluorescent stain.

The characteristics of protein detection and quantitation with SYPRO Ruby protein gel stain in one- and two-dimensional polyacrylamide gels were evaluated. Sodium dodecyl sulfate-polyacrylamide gel electrophoresis (SDS-PAGE) analyses of three different purified recombinant proteins showed that the limits of detection were comparable to the limits of detection with ammoniacal silver staining and were protein-specific, ranging from 0.5 to 5 ng. The linearity of the relationship between protein level and SYPRO Ruby staining intensity also depended on the individual protein, with observed linear dynamic ranges of 200-, 500-, and, 1000-fold for proteins analyzed by SDS-PAGE. SYPRO Ruby protein gel stain was also evaluated in two-dimensional electrophoretic (2-DE) analysis of Escherichia coli proteins. The experiment involved analysis of replicates of the same sample as well as dilution of the sample from 0.5 to 50 nug total protein across gels. In addition to validating the 2-DE system itself, the experiment was used to evaluate three different image analysis programs: Z3 (Compugen), Progenesis (Nonlinear Dynamics), and PDQuest (Bio-Rad). In each program, we analyzed the 2-DE images with respect to sensitivity and reproducibility of overall protein spot detection, as well as linearity of response for 20 representative proteins of different molecular weights and pI. Across all three programs, coefficients of variation (CV) in total number of spots detected among replicate gels ranged from 4 to 11%. For the 20 representative proteins, spot quantitation was also comparable with CVs for gel-to-gel reproducibility ranging from 3 to 33%. Using Progenesis and PDQuest, a 1000-fold linear dynamic range of SYPRO Ruby was demonstrated with a single known protein. These two programs were more suitable than Z3 for examining individual protein spot quantity across a series of gels and gave comparable results.

Bacterial Proteins↗

PubMedDate not supplied

The development of a comprehensive, institution-based patient risk evaluation program: II. Validity and reliability of questionnaire data.

The accuracy of historical information derived from self-administered questionnaires must be confirmed. We report the results of studies conducted to assess the reliability and validity of data collected from a comprehensive cancer risk factor questionnaire developed at The University of Texas M.D. Anderson Cancer Center. A comparison of the basic demographic data of a randomly selected sample of 80 respondents and 70 nonrespondents revealed no fundamental ethnic or socioeconomic differences. We verified self-reported past illnesses, surgical procedures, and cancers by reviewing 72 patient charts, using stringent diagnostic criteria for verification. We noted substantial agreement between self-reported and documented illnesses and operations. With the exception of nine patients who misclassified metastatic disease, the verification of primary cancers was excellent. We determined reliability by interviewing 50 of these patients by telephone. Questions with a dichotomous outcome (e.g., smoking status) were reliably answered; however, those requiring quantification (e.g., amount of alcohol consumed) were less accurately reported on interview. While we recognize the limitations of self-administered questionnaires, we believe this program will develop into a comprehensive, standardized, easily accessible patient risk factor data base.

Cancer Care Facilities↗

PubMed1996

Validity of computerized predictions of dentoskeletal and soft tissue profile changes after mandibular setback and maxillary impaction osteotomies.

The aim of the present investigation was to evaluate the validity of the predictions of a computerized cephalometric system (Dentofacial Planner) regarding dentoskeletal and soft tissue profile changes after mandibular setback and maxillary impaction osteotomies. Tracings of lateral cephalograms taken at the end of preoperative orthodontics (within 1 week before surgery) and approximately 1 year after the operation were digitized and entered into the Dentofacial Planner. For the mandibular setback group, the computerized predictions tended to place the mandible less posteriorly than the actual situation and to significantly underestimate the mandibular plane angle, the total anterior skeletal and soft tissue facial heights, the lower anterior skeletal facial height, and the upper lip height. In the maxillary impaction group, the prediction printouts significantly overestimated the total anterior soft tissue facial height, the upper lip height and the inclination and curvature of the lower lip and underestimated the soft tissue thickness in the regions of pogonion and point B.

Adolescent↗

PubMed2004

The challenges of imaging based computational fluid dynamics.

Image based Computational Fluid Dynamics (CFD) simulation of the cardiovascular system is increasingly becoming important and its application in everyday medical practice can already be envisaged. The goal of this workshop is to address all the factors involved in the development of a computational framework/software for modelling and analyses of the cardiovascular system and provide examples. The development of such framework, requires integration, management and interpretation of data from several technology areas such as a) feature detection and extraction of arterial geometry from imaging data, b) adaptive grid generation techniques for 3-D asymmetric geometries c) hemodynamic modelling, disparate length-scale model, and fluid-tissue interaction with high-performance computing, d) CFD data validation, e) feature extraction/detection and visualisation algorithms, f) graphical user interface to allow remote visualisation of post processed data. These computational tools are employed to study flow in specific problem sites in the vascular tree such as the carotid, femoral, coronary and abdominal arteries. Such studies provide understanding of the factors involved in the initialisation and evolution of arterial disease due to altered flow conditions (as a result of plaque formation) such as flow separation and reversal, and low and oscillatory wall shear stress. It is also used to study the effect of various clinical procedures such as the implantation of stents, vascular grafts, vascular prostheses and artificial valve implants on local and global hemodynamics. This workshop will address a new emerging paradigm in clinical practice known as predictive medicine for effective surgical planning and post surgical rehabilitation. The workshop will also address the difficulties in the implementation of some of the technology areas in this application with examples of carotid, femoral, and abdominal artery simulations.

Algorithms↗

PubMedDate not supplied

Evaluation of Sleep Expert--a computer-aided decision support system for sleep disorders.

Sleep Expert--a medical decision support system--is a prototype program, with knowledge based on the International Classification of Sleep Disorders (1990). The goal of this project was to evaluate Sleep Expert. In the evaluation project the knowledge of the program was first validated. Three physicians, experts in sleep disorders, were asked to choose 10 typical patient cases with sleep disorders, and to write a description. They also made a diagnosis for each case. Next, each expert made a diagnosis of the cases supplied by the other experts. They were not given the original diagnosis. The 'right diagnosis' (so-called majority agreement) was determined from the three diagnoses. Then the diagnosis of each expert was compared with the 'right diagnosis'. Two physicians, not experts in sleep disorders, were asked to make a diagnosis by using Sleep Expert. Compared to the 'right diagnosis' the diagnoses of each user (non-expert physician) were correct to 63 and 70% of cases, which is quite a good result, although it does not reach the level of the expert physicians (> or = 87%). The functionality of Sleep Expert was studied by using a limited inquiry. On the basis of the user inquiry Sleep Expert provided a useful clinical tool for non-experts.

Diagnosis, Computer-Assisted↗

PubMed2003

Development and validation of a BEAMnrc component module for accurate Monte Carlo modelling of the Varian dynamic Millennium multileaf collimator.

A new component module (CM), designated DYNVMLC, was developed to fully model the geometry of the Varian Millennium 120 leaf collimator using the BEAMnrc Monte Carlo code. The model includes details such as the leaf driving screw hole, support railing groove and leaf tips. Further modifications also allow sampling of leaf sequence files to simulate the movement of the multileaf collimator (MLC) leaves during an intensity modulated radiation therapy (IMRT) delivery. As an initial validation of the code, the individual leaf geometries were visualized by tracing particles through the component module and recording their position each time a leaf boundary was crossed. A model of the Varian CL21EX linear accelerator 6 MV photon beam incorporating the new CM was built with the BEAMnrc user code. The leaf material density and abutting leaf air gap were chosen to match simulated leaf leakage profiles with film measurements in a solid water phantom. Simulated depth dose and off-axis profiles for a variety of MLC defined static fields agreed to within 2% with ion chamber and diode measurements in a water phantom. Simulated dose distributions for IMRT intensity patterns delivered using both static and dynamic techniques were found to agree with film measurements to within 4%. A comparison of interleaf leakage profiles for the new CM and an equivalent leaf model using the existing VARMLC CM demonstrated that the simplified geometry of VARMLC is not able to accurately predict the details of the MLC leakage for the 120 leaf collimator.

Computer Simulation↗

PubMed2006

The BCI competition. III: Validating alternative approaches to actual BCI problems.

A brain-computer interface (BCI) is a system that allows its users to control external devices with brain activity. Although the proof-of-concept was given decades ago, the reliable translation of user intent into device control commands is still a major challenge. Success requires the effective interaction of two adaptive controllers: the user's brain, which produces brain activity that encodes intent, and the BCI system, which translates that activity into device control commands. In order to facilitate this interaction, many laboratories are exploring a variety of signal analysis techniques to improve the adaptation of the BCI system to the user. In the literature, many machine learning and pattern classification algorithms have been reported to give impressive results when applied to BCI data in offline analyses. However, it is more difficult to evaluate their relative value for actual online use. BCI data competitions have been organized to provide objective formal evaluations of alternative methods. Prompted by the great interest in the first two BCI Competitions, we organized the third BCI Competition to address several of the most difficult and important analysis problems in BCI research. The paper describes the data sets that were provided to the competitors and gives an overview of the results.

Algorithms↗