SEARCH · PubMed Health

Results for “Software Validation”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11Linked to original sources

PubMed2004

Comparative analysis of methods for representing and searching for transcription factor binding sites.

MOTIVATION: An important step in unravelling the transcriptional regulatory network of an organism is to identify, for each transcription factor, all of its DNA binding sites. Several approaches are commonly used in searching for a transcription factor's binding sites, including consensus sequences and position-specific scoring matrices. In addition, methods that compute the average number of nucleotide matches between a putative site and all known sites can be employed. Such basic approaches can all be naturally extended by incorporating pairwise nucleotide dependencies and per-position information content. In this paper, we evaluate the effectiveness of these basic approaches and their extensions in finding binding sites for a transcription factor of interest without erroneously identifying other genomic sequences. RESULTS: In cross-validation testing on a dataset of Escherichia coli transcription factors and their binding sites, we show that there are statistically significant differences in how well various methods identify transcription factor binding sites. The use of per-position information content improves the performance of all basic approaches. Furthermore, including local pairwise nucleotide dependencies within binding site models results in statistically significant performance improvements for approaches based on nucleotide matches. Based on our analysis, the best results when searching for DNA binding sites of a particular transcription factor are obtained by methods that incorporate both information content and local pairwise correlations. AVAILABILITY: The software is available at http://compbio.cs.princeton.edu/bindsites.

Algorithms↗

PubMed2003

Mistake proofing and redundancy in machine validation.

Mistake proofing and redundancy in design are becoming essential features of the validated environment. Properly applied and embraced, they will further increase the reliability of machines and the quality of the products emerging from the medical device industry. More importantly they will help focus validation on machine enhancement and reliability and reduce the dependence on operator intervention and vigilance in ensuring desired quality levels are achieved. They are the only way of eliminating the transient fault and aberration in performance that has arisen from the complexity of modern machine design.

Computer-Aided Design↗

PubMed1997

Evaluation of the VALAB expert system.

The validation of a clinical laboratory report is a process that guarantees the results contained in the report have been obtained under satisfactory metrological conditions and that they are compatible with the information available on the patient. This validation is generally carried out manually by a clinical laboratory professional, but also may be done by an expert system properly programmed, such as the VALAB system. The evaluation presented in this article consists of comparing human and system decisions of validation for 500 randomly selected clinical laboratory reports from hospitalized patients. In this evaluation, 84.8% of the reports examined by the VALAB are accepted directly without any human aid, and only 15.2% require examination by clinical biochemists.

Clinical Chemistry Tests↗

PubMed2006

Benchmarking a memetic algorithm for ordering microarray data.

This work introduces a new algorithm for "gene ordering". Given a matrix of gene expression data values, the task is to find a permutation of the gene names list such that genes with similar expression patterns should be relatively close in the permutation. The algorithm is based on a combined approach that integrates a constructive heuristic with evolutionary and Tabu Search techniques in a single methodology. To evaluate the benefits of this method, we compared our results with the current outputs provided by several widely used algorithms in functional genomics. We also compared the results with our own hierarchical clustering method when used in isolation. We show that the use of images, corrupted with known levels of noise, helps to illustrate some aspects of the performance of the algorithms and provide a complementary benchmark for the analysis. The use of these images, with known high-quality solutions, facilitates in some cases the assessment of the methods and helps the software development, validation and reproducibility of results. We also propose two quantitative measures of performance for gene ordering. Using these measures, we make a comparison with probably the most used algorithm (due to Eisen and collaborators, PNAS 1998) using a microarray dataset available on the public domain (the complete yeast cell cycle dataset).

Algorithms↗

PubMed2005

Virtual commissioning of a treatment planning system for proton therapy of ocular cancers.

The virtual commissioning of a treatment planning system (TPS) for ocular proton beam therapy was performed using Monte Carlo (MC) simulations and a model of a double-scattering ocular treatment nozzle. The simulations produced both the input data required by the TPS and the dose distributions to validate the analytical predictions from the TPS. An MC simulation of a typical ocular melanoma treatment was compared with the TPS predictions, revealing generally good agreement in the absorbed dose distribution. However, in the depth-dose profiles, differences >5% existed in the proximal region of all validation cases considered. Comparison of the radiation coverage at or above the 90% dose level, showed that MC calculated coverage was 82% and 68% of the coverage calculated by the TPS in two planes intersecting the tumour.

Body Burden↗

PubMed1999

Validation of bioassays for quality control.

For a biological assay to be useful for quality control it should fail bad lots, pass good lots, and estimate relative potency with high accuracy and precision. To fail a lot we rely most heavily on the test for parallelism. For the parallelism test and other preliminary tests as well as for inference, appropriate estimates of assay variation are crucial. Location effects on 96 well plates and serial dilution of samples using multichannel pipettes make it difficult to obtain good estimates of assay variation. This paper develops the use of a split-block design and analysis where blocks are reasonably consistent regions of a plate; this approach removes some location effects, allows other location effects to be treated as assay variation and provides appropriate measures of assay variation. Randomization, even within the split-block design, is difficult without robots to reduce the likelihood of procedural errors. There are hardware, software, and validation obstacles to implementation of robots in the bioassay laboratory. More generally, validation of a bioassay should be reported on log relative potency and must address between- and within-assay variation. When between assay variation is not small, the usual weighted approach to combining relative potency estimates (which ignores between-assay variation) is inappropriate; a simple sampling average and standard deviation is a better solution.

Biological Assay↗

PubMed2005

A protocol for building and evaluating predictors of disease state based on microarray data.

MOTIVATION: Microarray gene expression data are increasingly employed to identify sets of marker genes that accurately predict disease development and outcome in cancer. Many computational approaches have been proposed to construct such predictors. However, there is, as yet, no objective way to evaluate whether a new approach truly improves on the current state of the art. In addition no 'standard' computational approach has emerged which enables robust outcome prediction. RESULTS: An important contribution of this work is the description of a principled training and validation protocol, which allows objective evaluation of the complete methodology for constructing a predictor. We review the possible choices of computational approaches, with specific emphasis on predictor choice and reporter selection strategies. Employing this training-validation protocol, we evaluated different reporter selection strategies and predictors on six gene expression datasets of varying degrees of difficulty. We demonstrate that simple reporter selection strategies (forward filtering and shrunken centroids) work surprisingly well and outperform partial least squares in four of the six datasets. Similarly, simple predictors, such as the nearest mean classifier, outperform more complex classifiers. Our training-validation protocol provides a robust methodology to evaluate the performance of new computational approaches and to objectively compare outcome predictions on different datasets.

Algorithms↗

PubMed1998

Multiagency outcome evaluation of children's services: a case study.

Outcome monitoring has become a focus of accountability for public and nonprofit human service agencies. Besides providing answers to funders' questions about the services' impact, outcome monitoring helps administrators improve program effectiveness. After a three-year development period and a one-year implementation experience, SumOne for Kids represents a technically advanced outcome-monitoring system for children's mental health and/or child welfare services. Initiated, designed, and tested by 31 children's service agencies throughout Pennsylvania, and with state bureaucrats' and policy makers' encouragement, SumOne for Kids represents an effort to create a bottom-up/top-down process for implementing a statewide outcome-monitoring system. This article describes the genesis of this outcome-monitoring system, primary design principles, use of social validation for outcome selection, resolution of methodological difficulties, and reasons for selecting functional over clinical outcomes. The article reviews lessons learned through the development experience instructive to children's service managers, program evaluators, and industry leaders interested in establishing outcome-monitoring systems.

Child↗

PubMed1995

Validation procedures for the Hamilton Thorne Integrated Visual Optical System sperm and cell analyzer.

The Hamilton Thorne Integrated Visual Optical System (IVOS) analyzer is a combined internal optical and computer system widely applied to laboratory animals, such as rat, rabbit, and canine, sperm in reproductive toxicology. It measures sperm motility, progressive motility, velocity, motion parameters, and concentration. The system is specially designed to facilitate on-site validation. Digital encoding allows image storage with absolute replay fidelity for later reexamination, enabling on-site validation complying with Good Laboratory Practices. A NIST-certified scale provides the length standard on which validation is based. Sperm typically move in wavy tracks, which must be characterized and validated. Playback of exact sperm tracks allows visual validation of sperm head position, given as Cartesian coordinates, and visual determination of motility from the playback screen. A cursor checks coordinate values, and manual confirmation of sperm motion parameters may be performed directly from the validated coordinates. Agreement to within 0.2% is obtained between manual and IVOS computation. Concentration is determined using a specific DNA stain that enables discrimination between sperm and somatic cell nuclei. The IVOS concentration of sperm nuclei in homogenized rat testis and cauda epididymis has been determined to be within 5% of manual hemacytometer counts of the same homogenate.

Humans↗

PubMed2005

A neural network based classification scheme for cytotoxicity predictions:Validation on 30,000 compounds.

Elimination of cytotoxic compounds in the early phases of drug discovery can save substantial amounts of research and development costs. An artificial neural network based approach using atomic fragmental descriptors has been developed to categorize compounds according to their in vitro human cytotoxicity. Fragmental descriptors were obtained from the Atomic7 linear logP calculation method implemented in Pallas PrologP program. We used cytotoxicity values obtained from an in-house screening campaign of a diverse set of 30,000 drug-like molecules. The training set included only the most and least toxic 12,998 compounds, however, cytotoxicity data for all compounds were used for validation. The proposed approach can be safely used for filtering out potentially cytotoxic candidates from the development pipeline before synthesis or assays during lead development or lead optimisation. The trained neural network misclassified less than 5% percent of the non-toxic and 9% of the toxic compounds.

Cell Line↗

PubMed1994

Methodological issues in validating decision-support systems for insulin dosage adjustment.

Safety and reliability of advice from new computer systems should be confirmed before embarking on prospective hospital trials. This process of preliminary testing is termed 'validation'. Though it forms a fundamental stage in system development, few standards exist for choosing and implementing tests. In the present paper, a validation methodology is developed in the domain of diabetes and intended for general use in chronic health management. It is based on a peer review protocol and incorporates empirical measures indicating: applicability of results to the real environment; variation among doctors; comparisons between doctors' and computer advice; and relative merits of different computer algorithms.

Algorithms↗

PubMed2006

SynTReN: a generator of synthetic gene expression data for design and analysis of structure learning algorithms.

BACKGROUND: The development of algorithms to infer the structure of gene regulatory networks based on expression data is an important subject in bioinformatics research. Validation of these algorithms requires benchmark data sets for which the underlying network is known. Since experimental data sets of the appropriate size and design are usually not available, there is a clear need to generate well-characterized synthetic data sets that allow thorough testing of learning algorithms in a fast and reproducible manner. RESULTS: In this paper we describe a network generator that creates synthetic transcriptional regulatory networks and produces simulated gene expression data that approximates experimental data. Network topologies are generated by selecting subnetworks from previously described regulatory networks. Interaction kinetics are modeled by equations based on Michaelis-Menten and Hill kinetics. Our results show that the statistical properties of these topologies more closely approximate those of genuine biological networks than do those of different types of random graph models. Several user-definable parameters adjust the complexity of the resulting data set with respect to the structure learning algorithms. CONCLUSION: This network generation technique offers a valid alternative to existing methods. The topological characteristics of the generated networks more closely resemble the characteristics of real transcriptional networks. Simulation of the network scales well to large networks. The generator models different types of biological interactions and produces biologically plausible synthetic gene expression data.

Algorithms↗

PubMed1991

[Bacterio-expert: an integrated system for assisting in the validation of antibiotic sensitivity tests. Retrospective application in 4053 Staphylococcus].

Bacterio-expert is a simple expert system for assisting in the validation of antibiotic sensitivity testing. This system is incorporated in a data acquisition and editing program for bacteriologic test (Bacterio program written in Turbo-Pascal for personal computer users by the same authors). The principles of this system are explained and results with 4,053 antibiotic sensitivity tests on Staphylococcus aureus isolates are reported. Approximately 10% of tests required corrections.

Anti-Bacterial Agents↗

PubMed2005

MPrime: efficient large scale multiple primer and oligonucleotide design for customized gene microarrays.

BACKGROUND: Enhancements in sequencing technology have recently yielded assemblies of large genomes including rat, mouse, human, fruit fly, and zebrafish. The availability of large-scale genomic and genic sequence data coupled with advances in microarray technology have made it possible to study the expression of large numbers of sequence products under several different conditions in days where traditional molecular biology techniques might have taken months, or even years. Therefore, to efficiently study a number of gene products associated with a disease, pathway, or other biological process, it is necessary to be able to design primer pairs or oligonucleotides en masse rather than using a time consuming and laborious gene-by-gene method. RESULTS: We have developed an integrated system, MPrime, in order to efficiently calculate primer pairs or specific oligonucleotides for multiple genic regions based on a keyword, gene name, accession number, or sequence fasta format within the rat, mouse, human, fruit fly, and zebrafish genomes. A set of products created for mouse housekeeping genes from MPrime-designed primer pairs has been validated using both PCR-amplification and DNA sequencing. CONCLUSION: These results indicate MPrime accurately incorporates standard PCR primer design characteristics to produce high scoring primer pairs for genes of interest. In addition, sequence similarity for a set of oligonucleotides constructed for the same set of genes indicates high specificity in oligo design.

Animals↗

PubMed2002

Validity and reliability of a voice-recognition game analysis system for field sports.

The purpose of this study was to assess the ability of observers to use voice-recognition analysis to accurately classify gait transitions and quantify gait durations typical of team games. Inter-rater and intra-rater reliability was also determined. Four males were filmed performing pre-determined gait protocols. each comprising different sequences of walking, jogging, running and sprinting. Two operators independently classified gait transitions and the time spent in each gait was determined by the voice recognition system. All gait modes as measured by trained observers demonstrated statistically significant correlations (p < 0.01) to pre-determined measurement criteria. The mean absolute error for all gait transitions was less than half a second (0.32-0.36 s) with the maximum percentage error being approximately 4% for the walk, jog and run gaits and 10% for sprinting. Gait classification error was low at 1.9%. The intra-rater and inter-rater reliability was consistently high ranging from r = 0.87 to 0.99. In conclusion, observers using voicerecognition software provided valid measures of time spent in each of the four gait categories with 90% or better accuracy achieved.

Adult↗

PubMed2004

Performance measures for video object segmentation and tracking.

We propose measures to evaluate quantitatively the performance of video object segmentation and tracking methods without ground-truth (GT) segmentation maps. The proposed measures are based on spatial differences of color and motion along the boundary of the estimated video object plane and temporal differences between the color histogram of the current object plane and its predecessors. They can be used to localize (spatially and/or temporally) regions where segmentation results are good or bad; and/or they can be combined to yield a single numerical measure to indicate the goodness of the boundary segmentation and tracking results over a sequence. The validity of the proposed performance measures without GT have been demonstrated by canonical correlation analysis with another set of measures with GT on a set of sequences (where GT information is available). Experimental results are presented to evaluate the segmentation maps obtained from various sequences using different segmentation approaches.

Algorithms↗

PubMedDate not supplied

Validation of a detailed computer model for the electric fields in the brain.

A computer model has been designed for the calculation of the electrical fields in the head, based on the finite difference method. This method has not previously been applied for head modelling. The model was validated by using three concentric spheres and comparing it with an analytic model. Three levels of accuracy were tested. The forward solutions show that the finite difference algorithm works correctly and, by selecting the size of the volume elements properly, accurate results are obtained. The model will be applied to accurate and realistic geometries of the human head obtained from magnetic resonance images.

Brain↗

PubMed2003

Validation of the EGS usercode DOSE3D for internal beta dose calculation at the cellular and tissue levels.

Internal radiotherapy is currently focusing on beta emitters such as 90Y or 131I because of their high-energy emissions. However, conventional dosimetric methods (MIRD) are known to be limited for such applications. They are unable to take into account microscopic radionuclide distribution because standardized anthropomorphic phantoms are used, and absorbed dose is calculated at the organ level. New tools are therefore required for dose assessment at cellular and tissue level (10-100 microm). The purpose of this study was to validate, at this scale, a Monte Carlo usercode (DOSE3D), based on the MORSE combinatorial geometry package and the EGS code system. Dose point-kernel calculations in water were compared to those published by Cross et al and Simpkin and Mackie. They confirm that DOSE3D is a reliable tool for cellular dosimetry in various geometric configurations.

Beta Particles↗