SEARCH · PubMed Health

Results for “Performance benchmarking”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19Linked to original sources

PubMed2026

Five tenets for advancing evidence-based precision medicine.

Precision medicine for complex diseases uses individual-level characteristics to improve prediction of risk, therapeutic response and prognosis. Many precision medicine studies leverage existing data types and analytic methods to reveal new insights; however, beyond oncology, there has been limited success in translating precision medicine research for complex diseases into clinical practice. Thus, there is a need to identify areas for improvement, particularly in translation-oriented analytical methods and study designs. In this perspective article, we outline five fundamental tenets to enhance the efficient clinical translation of precision medicine research. These tenets focus on addressing (1) heterogeneity in risk, response and prognosis; (2) signal robustness; (3) structured statistical benchmarking against key performance indicators; (4) precision trial designs; and (5) risks and benefits to individuals and society. Our intention is to promote clinically meaningful, reproducible, scalable and equitable health outcomes through precision medicine, beyond those possible through contemporary approaches.

Precision Medicine↗

PubMed2006

Cytogenetics, molecular and ultrastructural characteristics of biphenotypic acute leukemia identified by the EGIL scoring system.

Biphenotypic acute leukemia (BAL) is a rare, difficult to diagnose entity. Its identification is important for risk stratification in acute leukemia (AL). The scoring proposal of the European Group for the Classification of Acute Leukemia (EGIL) is useful for this purpose, but its performance against objective benchmarks is unclear. Using the EGIL system, we identified 23 (3.4%) BAL from among 676 newly diagnosed AL patients. Mixed, small and large blast cells predominated, with FAB M2 and L1 constituting the majority. All patients were positive for myeloid (M) markers and either B cell (B) (17 or 74%) or T cell (T) (8 or 34%) markers with two exceptional patients demonstrating trilineage phenotype. Six (50%) of studied M-B cases were positive for both IGH and TCR. In six (26%) patients myeloid lineage commitment was also demonstrable by electron cytochemistry. Abnormal findings were present in 19 (83%) patients by cytogenetics/FISH/molecular analysis as follows: t(9;22) (17%); MLL gene rearrangement (26%); deletion(6q) (13%); 12p11.2 (9%); numerical abnormalities (13%), and three (13%) new, previously unreported translocations t(X;6)(p22.3;q21); t(2;6)(q37;p21.3); and t(8;14)(p21;q32). In conclusion, the EGIL criteria for BAL appear robust when compared against molecular techniques that, if applied routinely, could aid in detecting BAL and help in risk stratification.

Acute Disease↗

PubMed1998

A decision tree system for finding genes in DNA.

MORGAN is an integrated system for finding genes in vertebrate DNA sequences. MORGAN uses a variety of techniques to accomplish this task, the most distinctive of which is a decision tree classifier. The decision tree system is combined with new methods for identifying start codons, donor sites, and acceptor sites, and these are brought together in a frame-sensitive dynamic programming algorithm that finds the optimal segmentation of a DNA sequence into coding and noncoding regions (exons and introns). The optimal segmentation is dependent on a separate scoring function that takes a subsequence and assigns to it a score reflecting the probability that the sequence is an exon. The scoring functions in MORGAN are sets of decision trees that are combined to give a probability estimate. Experimental results on a database of 570 vertebrate DNA sequences show that MORGAN has excellent performance by many different measures. On a separate test set, it achieves an overall accuracy of 95 %, with a correlation coefficient of 0.78, and a sensitivity and specificity for coding bases of 83 % and 79%. In addition, MORGAN identifies 58% of coding exons exactly; i.e., both the beginning and end of the coding regions are predicted correctly. This paper describes the MORGAN system, including its decision tree routines and the algorithms for site recognition, and its performance on a benchmark database of vertebrate DNA.

Algorithms↗

PubMed2026

PGS-GS: a framework integrating polygenic scores and genomic selection in animal breeding.

Genomic prediction has become a central paradigm in biology, enabling quantitative inference of genetic contributions to complex traits across humans, animals, and plants. Although genomic research in human genetics and animal breeding shares a highly homologous methodological foundation, significant barriers persist in their analytical paradigms and application scenarios. This study aims to promote cross-disciplinary integration by introducing human-derived polygenic scores (PGS) algorithms into animal genomic selection (GS) and proposing a PGS-GS framework with a preliminary weighting-based implementation. We systematically benchmarked the predictive performance and computational efficiency of 20 algorithms, including classical linear models, machine learning, PGS, and PGS-GS using both array and whole-genome sequencing (WGS) data across four major agricultural species: beef cattle, sheep, pigs, and chickens. Our results demonstrate that PGS and PGS-GS algorithms achieve predictive accuracy competitive with genomic best linear unbiased prediction (GBLUP) while offering markedly higher computational efficiency. Moreover, incorporating PGS-derived prior information into weighted linear and non-linear models outperformed conventional weighted GBLUP. The results provide empirical evidence to inform algorithm selection and highlight the potential of integrating human-derived PGS methodologies into animal genomic prediction frameworks.

Animals↗

PubMed2026

CBIcall: a configuration-driven framework for variant calling in large sequencing cohorts.

MOTIVATION: Variant calling for next-generation sequencing (NGS) data relies on a diverse ecosystem of tools and workflows. Large-scale collaborative studies increasingly adopt federated analysis, where each institution processes sensitive data locally using standardized pipelines. Deploying identical pipelines across multiple centers remains challenging because heterogeneous software environments and computing policies can cause workflow divergence and inconsistent results. RESULTS: We developed CBIcall, a workflow backend-flexible, configuration-driven framework that runs standardized variant-calling pipelines from raw FASTQ files to analysis-ready VCFs. Users define each analysis in a single YAML parameters file, which CBIcall resolves against a controlled workflow registry and resource catalog. The execution driver validates parameters and checks compatibility among pipelines, analysis modes, workflow backends, genome builds, tool versions, and resource bundles. CBIcall supports reproducibility auditing by comparing executions using recorded provenance and output fingerprints. CBIcall dispatches validated workflows natively through Bash, Cromwell, Nextflow and Snakemake backends and provides production-ready pipelines for germline WES, WGS (single-sample or cohort joint genotyping following GATK Best Practices), and mitochondrial DNA analysis. We evaluated analytical performance using public benchmark datasets and validated reproducibility across four computing environments. We further deployed CBIcall in the EU HEREDITARY project, where it processed 1102 samples with both WES and mtDNA pipelines on an institutional HPC system, supporting its suitability for reproducible cohort-scale genomic analyses. AVAILABILITY AND IMPLEMENTATION: CBIcall is open source (GPLv3) and distributed with ready-to-run pipelines; full dependency and installation documentation is available at https://github.com/CNAG-Biomedical-Informatics/cbicall.

Journal Article↗

PubMed2026

ALPAR: automated learning pipeline for antimicrobial resistance.

SUMMARY: The field of machine learning in antimicrobial resistance (AMR) research has experienced rapid growth, fueled by advancements in high-throughput genome sequencing and the growing capacity of computational resources. However, the complexity and lack of standardized data preparation and bioinformatic analyses present significant challenges, especially for newcomers to the domain. In response to these challenges, we introduce ALPAR (Automated Learning Pipeline for Antimicrobial Resistance), a comprehensive AMR data analysis tool covering the entire process from processing of raw genomic data to training machine learning models to interpretation of results. Our method relies on a reproducible pipeline that integrates widely used bioinformatics tools, presenting a simplified, automatic workflow specifically tailored for single-reference AMR analysis. Accepting genomic data in the form of FASTA files as input, ALPAR facilitates the generation of machine learning-ready data tables and both the training of machine learning and the execution of genome-wide association studies (GWAS) experiments. Additionally, our tool offers supplementary functionalities such as phylogeny-based analysis of the distribution of mutations, enhancing its utility for researchers. The tool has also proven its performance in competitive benchmarks, winning the 2024 CAMDA Anti-Microbial Resistance Prediction Challenge and placing third in the 2025 edition. AVAILABILITY AND IMPLEMENTATION: ALPAR is open-source and freely accessible via GitHub (https://github.com/kalininalab/ALPAR). The pipeline is fully reproducible and can be easily installed as a Conda package (https://anaconda.org/kalininalab/ALPAR).

Machine Learning↗

PubMed2025

Augmented kurtosis-based projection pursuit: a novel, advanced machine learning approach for multi-omics data analysis and integration.

Due to the heterogeneity of multi-omics data, exacting their maximum information potential remains a challenge. Whereas some solutions have been offered, most cannot overcome the large linear dynamic range associated with such data, while others require large biological effect sizes to produce meaningful models. Here, we (i) perform a comprehensive benchmarking of multi-omics data analysis tools, and (ii) introduce kurtosis-based projection pursuit analysis, augmented with classification and regression trees (kPPA-CART) as a robust, easy-to-implement alternative. Using ground truth data, we demonstrate that kPPA-CART exhibits superiority in inferring biological significance from low-intensity (low-count) features and studies with small biological effect sizes. Applying it to experimental breast cancer data from The Cancer Genome Atlas, we identify novel genes that cluster the samples into subtypes that mimic the canonical PAM50 classes with notable improvements. Validating with external metastatic breast cancer data from the AURORA US consortium, kPPA-CART identifies genes that are associated with poor event-free survival and additional clustering associated with increased tumor mutational burden. Finally, we provide an R package and an online implementation of kPPA-CART.

Humans↗

PubMed2005

Limitations and potentials of current motif discovery algorithms.

Computational methods for de novo identification of gene regulation elements, such as transcription factor binding sites, have proved to be useful for deciphering genetic regulatory networks. However, despite the availability of a large number of algorithms, their strengths and weaknesses are not sufficiently understood. Here, we designed a comprehensive set of performance measures and benchmarked five modern sequence-based motif discovery algorithms using large datasets generated from Escherichia coli RegulonDB. Factors that affect the prediction accuracy, scalability and reliability are characterized. It is revealed that the nucleotide and the binding site level accuracy are very low, while the motif level accuracy is relatively high, which indicates that the algorithms can usually capture at least one correct motif in an input sequence. To exploit diverse predictions from multiple runs of one or more algorithms, a consensus ensemble algorithm has been developed, which achieved 6-45% improvement over the base algorithms by increasing both the sensitivity and specificity. Our study illustrates limitations and potentials of existing sequence-based motif discovery algorithms. Taking advantage of the revealed potentials, several promising directions for further improvements are discussed. Since the sequence-based algorithms are the baseline of most of the modern motif discovery algorithms, this paper suggests substantial improvements would be possible for them.

Algorithms↗

PubMed2003

Applying diagnostic cost groups to examine the disease burden of VA facilities: comparing the six "Evaluating VA Costs" study sites with other VA sites and Medicare.

OBJECTIVES: To compare the disease burden of Veterans Health Administration (VA) patients at six study sites with all other VA patients and the Medicare population. DESIGN: A 60% random sample of all VA veteran patients during federal fiscal year 1997 was obtained from administrative databases. A split-sample technique provided a 40% sample (n = 1,046,803) for development and a 20% sample (n = 524,461) for validation. We selected the six study sites from the 40% sample, yielding a total of 50,080 patients in those sites. METHODS: We used Diagnostic Cost Groups to classify patients into clinical groupings based on age, gender, and International Classification of Diseases, Ninth Revision, Clinical Modification diagnoses. The Diagnostic Cost Group model produces relative risk scores that describe patients' expected resource use normalized to the Medicare population. We compared the severity of the six sites with each other and with all other VA facilities and the severity of VA patients with that of Medicare beneficiaries. RESULTS: There were minor statistically significant differences between the study sites and all other VA facilities. Compared with the Medicare population, VA's population was younger and had lower expected resource use (relative risk scores were 1.0 and 0.76, respectively). CONCLUSIONS: Disease burden of the six study sites is representative of all other VA facilities. Although lower relative risk scores suggest that VA patients are healthier than Medicare beneficiaries, when age is taken into account, scores are more comparable. Interpreting the expected resource utilization of the VA population against other benchmarks should be performed carefully.

Adolescent↗

PubMed2024

Proteome-scale tissue mapping using mass spectrometry based on label-free and multiplexed workflows.

Multiplexed bimolecular profiling of tissue microenvironment, or spatial omics, can provide deep insight into cellular compositions and interactions in healthy and diseased tissues. Proteome-scale tissue mapping, which aims to unbiasedly visualize all the proteins in a whole tissue section or region of interest, has attracted significant interest because it holds great potential to directly reveal diagnostic biomarkers and therapeutic targets. While many approaches are available, however, proteome mapping still exhibits significant technical challenges in both protein coverage and analytical throughput. Since many of these existing challenges are associated with mass spectrometry-based protein identification and quantification, we performed a detailed benchmarking study of three protein quantification methods for spatial proteome mapping, including label-free, TMT-MS2, and TMT-MS3. Our study indicates label-free method provided the deepest coverages of ~3500 proteins at a spatial resolution of 50 µm and the highest quantification dynamic range, while TMT-MS2 method holds great benefit in mapping throughput at >125 pixels per day. The evaluation also indicates both label-free and TMT-MS2 provide robust protein quantifications in identifying differentially abundant proteins and spatially co-variable clusters. In the study of pancreatic islet microenvironment, we demonstrated deep proteome mapping not only enables the identification of protein markers specific to different cell types, but more importantly, it also reveals unknown or hidden protein patterns by spatial co-expression analysis.

Journal Article↗

PubMed2006

Statistical mechanics of community detection.

Starting from a general ansatz, we show how community detection can be interpreted as finding the ground state of an infinite range spin glass. Our approach applies to weighted and directed networks alike. It contains the ad hoc introduced quality function from [J. Reichardt and S. Bornholdt, Phys. Rev. Lett. 93, 218701 (2004)] and the modularity Q as defined by Newman and Girvan [Phys. Rev. E 69, 026113 (2004)] as special cases. The community structure of the network is interpreted as the spin configuration that minimizes the energy of the spin glass with the spin states being the community indices. We elucidate the properties of the ground state configuration to give a concise definition of communities as cohesive subgroups in networks that is adaptive to the specific class of network under study. Further, we show how hierarchies and overlap in the community structure can be detected. Computationally efficient local update rules for optimization procedures to find the ground state are given. We show how the ansatz may be used to discover the community around a given node without detecting all communities in the full network and we give benchmarks for the performance of this extension. Finally, we give expectation values for the modularity of random graphs, which can be used in the assessment of statistical significance of community structure.

Journal Article↗

PubMed2004

An efficient method for computing leave-one-out error in support vector machines with Gaussian kernels.

In this paper, we give an efficient method for computing the leave-one-out (LOO) error for support vector machines (SVMs) with Gaussian kernels quite accurately. It is particularly suitable for iterative decomposition methods of solving SVMs. The importance of various steps of the method is illustrated in detail by showing the performance on six benchmark datasets. The new method often leads to speedups of 10-50 times compared to standard LOO error computation. It has good promise for use in hyperparameter tuning and model comparison

Computing Methodologies↗

PubMed2001

Evolutionary optimization of RBF networks.

One of the main obstacles to the widespread use of artificial neural networks is the difficulty of adequately defining values for their free parameters. This article discusses how Radial Basis Function (RBF) networks can have their parameters defined by genetic algorithms. For such, it presents an overall view of the problems involved and the different approaches used to genetically optimize RBF networks. A new strategy to optimize RBF networks using genetic algorithms is proposed, which includes new representation, crossover operator and the use of a multiobjective optimization criterion. Experiments using a benchmark problem are performed and the results achieved using this model are compared to those achieved by other approaches.

Algorithms↗

PubMed1998

Can academic radiology departments become more efficient and cost less?

PURPOSE: To determine how successful two large academic radiology departments have been in responding to market-driven pressures to reduce costs and improve productivity by downsizing their technical and support staffs while maintaining or increasing volume. MATERIALS AND METHODS: A longitudinal study was performed in which benchmarking techniques were used to assess the changes in cost and productivity of the two departments for 5 years (fiscal years 1992-1996). Cost per relative value unit and relative value units per full-time equivalent employee were tracked. RESULTS: Substantial cost reduction and productivity enhancement were realized as linear improvements in two key metrics, namely, cost per relative value unit (decline of 19.0% [decline of $7.60 on a base year cost of $40.00] to 28.8% [$12.18 of $42.21]; P < or = .001) and relative value unit per full-time equivalent employee (increase of 46.0% [increase of 759.55 units over a base year productivity of 1,651.45 units] to 55.8% [968.28 of 1,733.97 units]; P < .001), during the 5 years of study. CONCLUSION: Academic radiology departments have proved that they can "do more with less" over a sustained period.

Academic Medical Centers↗

PubMed2006

Dynamic tables: an architecture for managing evolving, heterogeneous biomedical data in relational database management systems.

Data sparsity and schema evolution issues affecting clinical informatics and bioinformatics communities have led to the adoption of vertical or object-attribute-value-based database schemas to overcome limitations posed when using conventional relational database technology. This paper explores these issues and discusses why biomedical data are difficult to model using conventional relational techniques. The authors propose a solution to these obstacles based on a relational database engine using a sparse, column-store architecture. The authors provide benchmarks comparing the performance of queries and schema-modification operations using three different strategies: (1) the standard conventional relational design; (2) past approaches used by biomedical informatics researchers; and (3) their sparse, column-store architecture. The performance results show that their architecture is a promising technique for storing and processing many types of data that are not handled well by the other two semantic data models.

Computational Biology↗

PubMedDate not supplied

Epidemiology of surgically treated abdominal aortic aneurysms in the United States, 1988 to 2000.

Abdominal aortic aneurysm (AAA) repair is a complex procedure about which little information exists regarding trends in surgical practice in the United States. This study was undertaken to define benchmark data regarding performance and outcomes of conventional AAA repair that might be used in comparisons with endovascular AAA repair data. Patients undergoing repair of intact (n = 87,728) or ruptured (n = 16,295) AAAs in the Nationwide Inpatient Sample (NIS) for 1988 to 2000 were studied. The NIS represents a 20% stratified random sample of all discharges from US hospitals. Unadjusted and case mix-adjusted analyses of in-hospital mortality and length of stay were performed. The overall frequency of intact AAA repair remained relatively stable during the study period, ranging from 18.1 to 16.3 operations/100,000 adults between 1988 and 2000, respectively. The operative mortality rate for intact AAA repair decreased significantly (p < .001) from 6.5% in 1988 to 4.3% in 2000. Length of stay following intact AAA repair also declined significantly (p < .001) from a median of 11 days in 1988 (interquartile range [IQR] 9-15 days) to 7 days in 2000 (IQR 5-10 days). The incidence of ruptured AAA repair decreased significantly (p < .001) from 4.2 to 2.6 operations/100,000 adults between 1988 and 2000, respectively. Mortality for ruptured AAA repair, averaging 45.6%, did not decrease significantly during the study period. Intact AAA repair by conventional means has become increasingly safe, with decreased operative mortality and shorter hospital stays. Ruptured AAA repair by conventional means has not become safer but has decreased in incidence, suggesting possible reductions in risk factors contributing to rupture, coupled with more timely intact AAA repairs.

Aged↗

PubMed2006

Predicting phase steps in phase-shifting interferometry in the presence of noise and harmonics.

A novel method for estimating pixelwise phase step values in phase-shifting interferometry is presented. The method is based on the linear prediction property of the intensity fringes recorded temporally at a pixel on the charged-coupled device. The salient features of the method lie in their ability to handle linear miscalibration errors, to compensate for the presence of harmonics in an optical configuration and detector nonlinearity, and to allow for the use of arbitrary phase steps. The robustness of the proposed method is studied in the presence of noise and a comparison with several benchmarking algorithms is performed. The simulation results show the efficiency of the algorithm in retrieving the wrapped phase.

Journal Article↗

PubMed2002

Quality of health care surveillance systems: review and implementation in the Swiss setting.

Quality of health care has been a subject of attention for many years in the USA and in Europe. Since the introduction of the new federal law on insurance in 1996 it has evolved to a progressively more important issue within the Swiss health care system. In this review, some theoretical concepts of quality of health care, variations, and surveillance systems are explored. Examples of quality of health care surveillance systems that have been developed successfully in the USA, in Canada, in Australia, and in Europe are discussed. They all demonstrate the interest in creating a large range of quality indicators in the surveillance system and in evaluating hospital performance using a benchmark approach. Currently, the measurement of quality with appropriate indicators is a subject of intense debate between the Swiss Hospitals Association (H+) and the Swiss Health Insurance Consortium (Santésuisse). Examples of existing surveillance systems in Switzerland are the Outcome Verein in Zurich and the quality of care program of the Canton of Valais. The FoQual association has also contributed to the debate by reviewing six indicators, which could be used nationally for a healthcare surveillance system. In this debate it is important to stress that ideal quality indicators intended for use as measures of quality in Swiss hospitals need to be both appropriate and valid. Only indicators that fulfil these conditions should be integrated in a Swiss health care surveillance system. Priority needs to be given to quality indicators and methods with the highest level of evidence and with a solid scientific basis.

Australia↗