PubMed Health⌕ Search

SEARCH · PubMed Health

Results for “Performance benchmarking”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 937 records · Page 52Linked to original sources
PubMed2024

NovoBoard: A Comprehensive Framework for Evaluating the False Discovery Rate and Accuracy of De Novo Peptide Sequencing.

De novo peptide sequencing is one of the most fundamental research areas in mass spectrometry-based proteomics. Many methods have often been evaluated using a couple of simple metrics that do not fully reflect their overall performance. Moreover, there has not been an established method to estimate the false discovery rate (FDR) of de novo peptide-spectrum matches. Here we propose NovoBoard, a comprehensive framework to evaluate the performance of de novo peptide-sequencing methods. The framework consists of diverse benchmark datasets (including tryptic, nontryptic, immunopeptidomics, and different species) and a standard set of accuracy metrics to evaluate the fragment ions, amino acids, and peptides of the de novo results. More importantly, a new approach is designed to evaluate de novo peptide-sequencing methods on target-decoy spectra and to estimate and validate their FDRs. Our FDR estimation provides valuable information to assess the reliability of new peptides identified by de novo sequencing tools, especially when no ground-truth information is available to evaluate their accuracy. The FDR estimation can also be used to evaluate the capability of de novo peptide sequencing tools to distinguish between de novo peptide-spectrum matches and random matches. Our results thoroughly reveal the strengths and weaknesses of different de novo peptide-sequencing methods and how their performances depend on specific applications and the types of data.

Peptides↗
PubMed2026

Genomic selection in timothy (Phleum pratense L.): a comprehensive evaluation of prediction models, multi-trait strategies, and forward validation across Norwegian environments.

This study presents a comprehensive evaluation of genomic selection (GS) in timothy (Phleum pratense L.), comparing nine prediction models across yield and quality traits at two Norwegian locations. Forward validation with independent full-sib (FS2) families revealed a substantial generalization gap, highlighting the need for realistic accuracy assessment in polyploid forage breeding. Timothy (Phleum pratense L.) is the most important forage grass in Northern Europe, yet genomic selection has not been systematically evaluated in this hexaploid species. We assessed 889 FS2-families originating from biparental crosses among 49 cultivars/populations. The FS2-families were genotyped with 30,698 SNP markers derived from genotyping-by-sequencing (GBS) and field tested for three harvest years at a highland and a lowland continental location in Southern Norway. Nine genomic prediction models were compared for six yield traits (dry matter yield per cut and total) and six quality traits (protein, digestibility, and fiber fractions) across three cuts/year. Within-training cross-validation accuracies were moderate to high (mean r = 0.62), with Random Forest and SVR consistently outperforming GBLUP. However, forward validation using 213 independent FS2-families revealed dramatically lower accuracies (mean r = 0.16), with only 16 of 30 trait-dataset combinations reaching statistical significance (p < 0.05). Genomic heritabilities (GREML), estimated across environments, ranged from near zero for the quality traits to 0.55 for the yield traits. Multi-trait models improved accuracy by 3-5% over single-trait approaches, while FS2 families-by-environment interaction models with Random Forest achieved the highest within-training accuracy (mean r = 0.71). Marker density analysis showed accuracy plateauing at approximately 15000 SNPs. Genetic correlations among the yield component traits were estimated by multi-trait REML; correlations among the quality traits could not be estimated reliably because their genomic heritabilities were low. A multi-trait selection index identified top-performing FS2-families for further crossing recommendations. These results provide a benchmark for GS implementation in hexaploid timothy and emphasize that cross-validation substantially overestimates prediction accuracy for truly independent material.

Norway↗
PubMed2006

[Conceiving and implementing an ISO 9001:2000 quality management system: quality improvement and efficiency increase over 3 years at the Dept. of Ophthalmology, Sultan Qaboos University in Oman].

BACKGROUND: Oman is a developing country of the Middle East with a University Hospital in a national health system. Problems related to documentation, structure, and procedure prevailing in a multicultural environment result in a lack of productivity. The aim of this study was to test the feasibility of introducing a quality management system according to ISO 9001:2000 into a university department of ophthalmology with special reference to quality and efficiency data over 3 years. MATERIALS AND METHODS: Standardized types of documentation (35 documents, 183 forms and formats, and 453 registers) under a formal roof (booklets for work instructions, quality management, procedures) were conceived in 2001 and implemented, and were evaluated by descriptive statistics over the years 2002-2005 with respect to quality and efficiency. In 2005, the departmental QMS was merged into the QMS of the whole hospital. RESULTS: Establishment of "responsibility clusters" with structured surveillance tasks, procedural improvements, and continuous audits resulted in a significant quality improvement already during the implementation until ISO 9001:2000 certification in 2002, and over the following 3 years. The data were assessed by non-conformities documented monthly (total inpatients: -50%, outpatients: -20%, operating theater: -100%, teaching: -10%; p <0.05) and increased efficiency assessed by the most important performance indicators (before QMS vs a median of 3 years with QMS): major surgery: +459%, inpatient stay: -49.7%, occupancy rate: +63.9%, technical diagnostics: +292.9%, postgraduate teaching total: +740%, research output total: +330% (p <0.05). CONCLUSIONS: It is feasible to introduce the quality management system according to ISO 9001:2000 into a developing country in the Middle East. It has an immediate effect on the increase of efficiency (measured by performance indicators) and quality in all systems providing the possibility for regional benchmarking.

Academic Medical Centers↗
PubMed2001

Improving performance in public hospitals: a role for comparative costs?

In order to encourage performance improvements, the English government has set targets for acute hospitals to reduce their unit costs. Targets are based on analysis of costs across all acute hospitals. This policy has parallels with that of 'yardstick competition', advocated as a means to encourage efficiency in industries that lack competitive pressures. However, the prospect of cost improvements may not be realised in England. Firstly, there are insufficient incentives to respond appropriately to the provision of comparative cost information. Secondly, there is more than one index purporting to measure relative hospital costs. As comparison of unit costs is highly dependent on the measurement technique adopted, caution should be exercised when setting performance targets.

Benchmarking↗
PubMed1999

Postpartum perineal management and best practice.

Sutured damage to the perineum occurs in at least 39% of women being delivered in Australia and perineal wound breakdown is associated with perineal oedema. The available literature was reviewed to ascertain what might constitute best practice for postpartum perineal wound management. Classification of levels of evidence was used to prioritise the available evidence. Little evidence that might be considered scientifically robust was found. Recommendations based on the available levels of evidence include a careful emphasis on perineal hygiene, cryotherapy, elevation of the foot of the bed in the presence of perineal oedema and regularly performed pelvic floor exercises.

Benchmarking↗
PubMed2003

Nucleic acid technology screening of Australian blood donors for hepatitis C and human immunodeficiency virus-1 RNA: comparison of two high-throughput testing strategies.

BACKGROUND AND OBJECTIVES: The aim of this study was to compare the performance of two high-throughput strategies for hepatitis C virus (HCV) and human immunodeficiency virus-1 (HIV-1) RNA nucleic acid technology (NAT) screening in a volunteer blood-donor population. MATERIALS AND METHODS: The semiautomated Chiron Procleix HIV-1/HCV transcription mediated amplification (TMA) assay was used to screen 1 439 765 donations in two different testing configurations. Three sites (termed PDT sites) performed a mixture of individual donation (ID) and minipool (MP) testing, where 1 113 288 donations were screened as pools of 24 and an additional 32 003 donations were screened in ID format. A further two sites (termed SDT sites) screened 294 474 donations exclusively in ID format. RESULTS: A significantly higher proportion of initial NAT reactives that failed to react on follow-up testing [termed non-repeatably reactive (NRR)] was observed for ID testing at SDT sites than at PDT sites (0.082 vs. 0.047%: P < 0.01). Within the PDT sites, however, there was no significant difference between the NRR rate for MP or ID samples (0.037 vs. 0.047%; not significant). There was a significant difference in failed run rates between PDT and SDT sites (P < 0.01), with PDT sites having a higher run failure rate owing to non-amplification of the internal control. The PDT sites also had a significantly higher overall invalid sample rate. However, the invalid sample rate, specifically caused by known equipment failure, was significantly higher in the SDT sites, possibly attributable to greater usage (P < 0.0001). Four HCV NAT-positive/antibody-negative samples were identified in the course of the study. CONCLUSIONS: The comparison of the performance of PDT with SDT sites identified only minor differences that did not adversely impact on the timely release of blood products. Although both ID and MP strategies showed excellent specificity, irrespective of site configuration, the significantly increased NRR rate, observed exclusively for ID testing performed at SDT sites, indicates a potential for contamination that may limit the number of samples that can optimally be processed using ID testing. The performance data for ID testing in particular should serve as a useful benchmark for evaluating candidate NAT systems that are fully automated.

Australia↗
PubMed2000

Measuring compliance with preventive care guidelines: standardized patients, clinical vignettes, and the medical record.

OBJECTIVE: To determine how accurately preventive care reported in the medical record reflects actual physician practice or competence. DESIGN: Scoring criteria based on national guidelines were developed for 7 separate items of preventive care. The preventive care provided by randomly selected physicians was measured prospectively for each of the 7 items. Three measurement methods were used for comparison: (1) the abstracted medical record from a standardized patient (SP) visit; (2) explicit reports of physician practice during those visits from the SPs, who were actors trained to present undetected as patients; and (3) physician responses to written case scenarios (vignettes) identical to the SP presentations. SETTING: The general medicine primary care clinics of two university-afflliated VA medical centers. PARTICIPANTS: Twenty randomly selected physicians (10 at each site) from among eligible second- and third-year general internal medicine residents and attending physicians. MEASUREMENTS AND MAIN RESULTS: Physicians saw 160 SPs (8 cases x 20 physicians). We calculated the percentage of visits in which each prevention item was recorded in the chart, determined the marginal percentage improvement of SP checklists and vignettes over chart abstraction alone, and compared the three methods using an analysis-of-variance model. We found that chart abstraction underestimated overall prevention compliance by 16% (P < .01) compared with SP checklists. Chart abstraction scores were lower than SP checklists for all seven items and lower than vignettes for four items. The marginal percentage improvement of SP checklists and vignettes to performance as measured by chart abstraction was significant for all seven prevention items and raised the overall prevention scores from 46% to 72% (P < .0001). CONCLUSIONS: These data indicate that physicians perform more preventive care than they report in the medical record. Thus, benchmarks of preventive care by individual physicians and institutions that rely solely on the medical record may be misleading, at best.

Adult↗
PubMed2004

Quantum chemical ab initio calculations of correlation effects in complex polymers: poly(para-phenylene).

Different quantum chemical approaches to the ground state correlation energy per unit cell of infinite poly(para-phenylene) (PPP) chains are presented. PPP is an organic polymer with interesting optical properties, due to its conjugated, aromatic pi system. The inclusion of correlation effects is crucial for a sound quantum chemical description of such a system. The correlation calculations were performed on the coupled cluster with single and double excitations (CCSD) level of theory using Dunning's spd correlation consistent polarized valence double-zeta basis sets. The correlation energy per unit cell is determined by means of the incremental method, which comprises series of CCSD calculations with partial excitation spaces. The resulting correlation energy per unit cell of PPP is -21.797 eV and compares well with that obtained by a simple but much more demanding cluster convergence approach (-21.775 eV). In addition, the accuracy and performance of the incremental scheme is discussed with respect to full CCSD benchmark calculations on PPP oligomers. Two variants are considered, the conventional one based on bond-type local units, and an extended one based on natural chemical subunits. Whereas it is difficult to reach "chemical" accuracy with the first variant, the second variant allows an accurate and efficient treatment with only a few individual CCSD calculations for a polymer with an aromatic pi system such as PPP.

Journal Article↗
PubMed2025

Vcfexpress: flexible, rapid user-expressions to filter and format VCFs.

MOTIVATION: Variant call format (VCF) files are the standard output format for various software tools that identify genetic variation from DNA sequencing experiments. Downstream analyses require the ability to query, filter, and modify them simply and efficiently. Several tools are available to perform these operations from the command line, including BCFTools, vembrane, slivar, and others. RESULTS: Here, we introduce vcfexpress, a new, high-performance toolset for the analysis of VCF files, written in the Rust programming language. It is nearly as fast as BCFTools, but adds functionality to execute user expressions in the lua programming language for precise filtering and reporting of variants from a VCF or BCF file. We demonstrate performance and flexibility by comparing vcfexpress to other tools using the vembrane benchmark. AVAILABILITY AND IMPLEMENTATION: vcfexpress is available under the MIT license at https://github.com/brentp/vcfexpress with code used for the manuscript deposited in https://doi.org/10.5281/zenodo.14756838.

Software↗
PubMed2026

scATAnno: Automated Cell Type Annotation for Single-cell ATAC-seq Data.

Recent advances in single-cell epigenomic techniques have increased the demand for single-cell assay for transposase-accessible chromatin using sequencing (scATAC-seq) analysis. One key analytical task is to determine cell type identity based on epigenetic data. Here, we introduce scATAnno, a Python package designed to automatically annotate scATAC-seq data using large-scale scATAC-seq reference atlases. This workflow generates reference atlases from publicly available datasets, enabling accurate cell type annotation by integrating query data with reference atlases without the use of single-cell RNA sequencing (scRNA-seq) data. To enhance annotation accuracy, we incorporated k-nearest neighbors (KNN)-based and weighted distance-based uncertainty scores to effectively detect cell populations within the query data that are distinct from all cell types in the reference data. We compared and benchmarked scATAnno against five other published cell annotation approaches, demonstrating its superior performance across multiple datasets and metrics. We further showcased the utility of scATAnno across multiple datasets, including peripheral blood mononuclear cells (PBMCs), triple-negative breast cancer (TNBC), and basal cell carcinoma (BCC), and demonstrated that scATAnno accurately annotates cell types across diverse biological conditions. Overall, scATAnno is a useful tool for scATAC-seq reference atlas construction and cell type annotation and can facilitate the interpretation of new scATAC-seq datasets in complex biological systems. scATAnno is publicly available at https://scatanno-main.readthedocs.io/.

Single-Cell Analysis↗
PubMed2026

UFold-X: an enhanced Dual & Dynamic U-Mamba model for long-range RNA secondary structure prediction.

RNA secondary structure is essential for understanding the functions of non-coding RNAs, ribosomal RNAs, and viral genomes. However, accurate prediction of long RNA structures remains challenging due to complex long-range interactions and the limited availability of long-RNA training data. We present UFold-X, a dual-branch deep learning framework that combines a convolutional encoder for local structure modeling with a Mamba-based Visual State Space Module for capturing long-range dependencies. A dynamic gating mechanism adaptively integrates the two branches according to sequence length. UFold-X was evaluated on multiple benchmark datasets containing RNAs up to 5000 nucleotides. To rigorously assess generalization, we introduced a cross-clan benchmark for long RNAs. Under this stringent setting, UFold-X achieved performance comparable to state-of-the-art classical approaches while achieving the best performance among deep learning-based methods. Additional cross-family and within-family evaluations further demonstrated robust transferability and competitive predictive performance. UFold-X also maintained excellent computational efficiency, requiring only 0.08 s per sequence on average. To assess biological consistency, we developed a SHAPE-based reactivity prediction variant (UFold-X-R) and an integrated metric, the Hybrid Reactivity-Pairing Score (HRPS). UFold-X-R showed strong agreement with experimental icSHAPE data and achieved the highest HRPS among all evaluated methods. A user-friendly web server is available at https://ufold-x.ai4bread.com.

Nucleic Acid Conformation↗
PubMed2006

Multiple alignment of protein sequences with repeats and rearrangements.

Multiple sequence alignments are the usual starting point for analyses of protein structure and evolution. For proteins with repeated, shuffled and missing domains, however, traditional multiple sequence alignment algorithms fail to provide an accurate view of homology between related proteins, because they either assume that the input sequences are globally alignable or require locally alignable regions to appear in the same order in all sequences. In this paper, we present ProDA, a novel system for automated detection and alignment of homologous regions in collections of proteins with arbitrary domain architectures. Given an input set of unaligned sequences, ProDA identifies all homologous regions appearing in one or more sequences, and returns a collection of local multiple alignments for these regions. On a subset of the BAliBASE benchmarking suite containing curated alignments of proteins with complicated domain architectures, ProDA performs well in detecting conserved domain boundaries and clustering domain segments, achieving the highest accuracy to date for this task. We conclude that ProDA is a practical tool for automated alignment of protein sequences with repeats and rearrangements in their domain architecture.

Algorithms↗
PubMed2006

Recurrent neural network as a linear attractor for pattern association.

We propose a linear attractor network based on the observation that similar patterns form a pipeline in the state space, which can be used for pattern association. To model the pipeline in the state space, we present a learning algorithm using a recurrent neural network. A least-squares estimation approach utilizing the interdependency between neurons defines the dynamics of the network. The region of convergence around the line of attraction is defined based on the statistical characteristics of the input patterns. Performance of the learning algorithm is evaluated by conducting several experiments in benchmark problems, and it is observed that the new technique is suitable for multiple-valued pattern association.

Letter↗
PubMedDate not supplied

Healthcare system performance indicators: a new beginning for a reformed Canadian healthcare system.

Canada's healthcare system is the envy of many other countries and their citizens around the world. And like other members of the Organization for Economic Cooperation and Development, Canada is under pressure to reduce healthcare expenditures. One strategy being explored by Canadian provincial governments is to regionalize or decentralize the delivery of healthcare services. To ensure achievement of its goals for each region, the Ministry of Health requires each regional health authority to submit a business plan and annual report, including performance information and underlying performance indicators. Performance indicators are used to measure process and outcome as well; they are useful for generating questions about the behavior or the performance of a provider or organization. A system for defining systemic performance measures for a publicly funded healthcare system does not exist. However, the performance indicators described in this article provide a sampling of provider-based healthcare performance.

Annual Reports as Topic↗
PubMed2002

Prevalence of parvovirus B19 and parvovirus V9 DNA and antibodies in paired bone marrow and serum samples from healthy individuals.

Parvovirus B19 (hereafter referred to as B19) exhibits a marked tropism to human bone marrow (BM), and infection may lead to erythema infectiosum, arthropathy, hydrops fetalis, and various hematologic disorders. Recently, a distinct parvovirus isolate termed V9 with an unknown clinical spectrum was discovered. In contrast to the many studies of B19 serology and viremia, valid information on the frequency of B19 or V9 DNA in the BM of healthy individuals is limited. To develop a reference value, paired BM and serum samples from healthy subjects were tested for the presence of B19 and V9 DNA and specific antibodies. Immunoglobulin M (IgM) was not found in any of the serum samples. The prevalence of IgG showed a gradual and steady increase from 37% in children aged 1 to 5 years to 87% in people aged >50 years. When 190 well-characterized subjects were examined, B19 DNA was detected in the BM of 4 individuals (2.1%; 95% confidence interval, 0.58 to 5.3%) while none of the paired serum samples showed evidence of circulating viral DNA. V9 DNA was not found in any of the BM or serum samples. The finding of B19 DNA probably indicated a primary infection in one 7-year-old individual and reinfection or reactivation of persistent infection in the remaining three persons, aged 47 to 58 years. Serving as a benchmark for future studies, these findings are useful when interpreting epidemiologic data, performing BM transplantation, or considering clinical implications of parvovirus infection.

Adolescent↗