PubMed Health⌕ Search

SEARCH · PubMed Health

Results for “Performance benchmarking”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12Linked to original sources

metaExpertPro: A Computational Workflow for Metaproteomics Spectral Library Construction and Data-Independent Acquisition Mass Spectrometry Data Analysis.

Analysis of large-scale data-independent acquisition mass spectrometry metaproteomics data remains a computational challenge. Here, we present a computational pipeline called metaExpertPro for metaproteomics data analysis. This pipeline encompasses spectral library generation using data-dependent acquisition MS, protein identification and quantification using data-independent acquisition mass spectrometry, functional and taxonomic annotation, as well as quantitative matrix generation for both microbiota and hosts. By integrating FragPipe and DIA-NN, metaExpertPro offers compatibility with both Orbitrap and timsTOF MS instruments. To evaluate the depth and accuracy of identification and quantification, we conducted extensive assessments using human fecal samples and benchmark tests. Performance tests conducted on human fecal samples indicated that metaExpertPro quantified an average of 45,000 peptides in a 60-min diaPASEF injection. Notably, metaExpertPro outperformed three existing software tools by characterizing a higher number of peptides and proteins. Importantly, metaExpertPro maintained a low factual false discovery rate of approximately 5% for protein groups across four benchmark tests. Applying a filter of five peptides per genus, metaExpertPro achieved relatively high accuracy (F-score = 0.67-0.90) in genus diversity and showed a high correlation (rSpearman = 0.73-0.82) between the measured and true genus relative abundance in benchmark tests. Additionally, the quantitative results at the protein, taxonomy, and function levels exhibited high reproducibility and consistency across the commonly adopted public human gut microbial protein databases IGC and UHGP. In a metaproteomic analysis of dyslipidemia patients, metaExpertPro revealed characteristic alterations in microbial functions and potential interactions between the microbiota and the host.

Proteomics↗

PyEvolve: a toolkit for statistical modelling of molecular evolution.

BACKGROUND: Examining the distribution of variation has proven an extremely profitable technique in the effort to identify sequences of biological significance. Most approaches in the field, however, evaluate only the conserved portions of sequences - ignoring the biological significance of sequence differences. A suite of sophisticated likelihood based statistical models from the field of molecular evolution provides the basis for extracting the information from the full distribution of sequence variation. The number of different problems to which phylogeny-based maximum likelihood calculations can be applied is extensive. Available software packages that can perform likelihood calculations suffer from a lack of flexibility and scalability, or employ error-prone approaches to model parameterisation. RESULTS: Here we describe the implementation of PyEvolve, a toolkit for the application of existing, and development of new, statistical methods for molecular evolution. We present the object architecture and design schema of PyEvolve, which includes an adaptable multi-level parallelisation schema. The approach for defining new methods is illustrated by implementing a novel dinucleotide model of substitution that includes a parameter for mutation of methylated CpG's, which required 8 lines of standard Python code to define. Benchmarking was performed using either a dinucleotide or codon substitution model applied to an alignment of BRCA1 sequences from 20 mammals, or a 10 species subset. Up to five-fold parallel performance gains over serial were recorded. Compared to leading alternative software, PyEvolve exhibited significantly better real world performance for parameter rich models with a large data set, reducing the time required for optimisation from approximately 10 days to approximately 6 hours. CONCLUSION: PyEvolve provides flexible functionality that can be used either for statistical modelling of molecular evolution, or the development of new methods in the field. The toolkit can be used interactively or by writing and executing scripts. The toolkit uses efficient processes for specifying the parameterisation of statistical models, and implements numerous optimisations that make highly parameter rich likelihood functions solvable within hours on multi-cpu hardware. PyEvolve can be readily adapted in response to changing computational demands and hardware configurations to maximise performance. PyEvolve is released under the GPL and can be downloaded from http://cbis.anu.edu.au/software.

Animals↗

The validity of predicted T-cell epitopes.

High-performing MHC class I binding predictions have been available for more than a decade; however, their value in terms of actual epitope finding has only now been estimated in a large-scale investigation undertaken by the group of Sette. This work underlines the importance of bioinformatics as a resource-saving tool in the field of epitope discovery. In addition, the data can be used to benchmark the performance of other new or existing CTL epitope-prediction tools.

Algorithms↗

Value decision-making: staff benchmarking.

Benchmarking is becoming a more important management tool--especially for setting staff levels. MGMA data, from Cost Surveys and Physician Compensation and Productivity Surveys, can help group managers set realistic goals. However, if taken simply at face value, the data may not provide adequate specificity; it may not convey the quality and value staff provide a particular organization. This paper show how to use MGMA data to perform staff benchmarking.

Benchmarking↗

On single and multiple models of protein families for the detection of remote sequence relationships.

BACKGROUND: The detection of relationships between a protein sequence of unknown function and a sequence whose function has been characterised enables the transfer of functional annotation. However in many cases these relationships can not be identified easily from direct comparison of the two sequences. Methods which compare sequence profiles have been shown to improve the detection of these remote sequence relationships. However, the best method for building a profile of a known set of sequences has not been established. Here we examine how the type of profile built affects its performance, both in detecting remote homologs and in the resulting alignment accuracy. In particular, we consider whether it is better to model a protein superfamily using a single structure-based alignment that is representative of all known cases of the superfamily, or to use multiple sequence-based profiles each representing an individual member of the superfamily. RESULTS: Using profile-profile methods for remote homolog detection we benchmark the performance of single structure-based superfamily models and multiple domain models. On average, over all superfamilies, using a truncated receiver operator characteristic (ROC5) we find that multiple domain models outperform single superfamily models, except at low error rates where the two models behave in a similar way. However there is a wide range of performance depending on the superfamily. For 12% of all superfamilies the ROC5 value for superfamily models is greater than 0.2 above the domain models and for 10% of superfamilies the domain models show a similar improvement in performance over the superfamily models. CONCLUSION: Using a sensitive profile-profile method we have investigated the performance of single structure-based models and multiple sequence models (domain models) in detecting remote superfamily members. We find that overall, multiple models perform better in recognition although single structure-based models display better alignment accuracy.

Amino Acid Sequence↗

Content-based interpretation aids for health-related quality of life measures in clinical practice. An example for the visual function index (VF-14).

BACKGROUND: In spite of a well-established development of instruments, difficulty in interpreting health related quality of life scores may limit its use in clinical practice. OBJECTIVE: To develop generalizable interpretation aids for a measure of perceived functional visual status, the VF-14 index. DESIGN: Item Response Theory (Rasch analysis) was used to analyze the performance of VF-14 items. The 'ruler' aid was derived from the most difficult activity (item) a patient is able to do without difficulty; the 'clinical scenarios' aid, first identified all significantly different clusters of items within the index and then estimated the mean expected difficulty (responses) to perform a benchmark item in each cluster. SETTING: The study was conducted in four hospitals and six ambulatory cataract surgery centers in Barcelona, Spain. PATIENTS: One hundred and ninety-eight patients scheduled for first eye cataracts surgery. MEASUREMENTS: The self-reported VF-14 index and clinical measures were used. RESULTS: All VF-14 items were found unidimensional with three items showing only partial misfit. For a patient with a VF-14 Rasch score of 71, the 'ruler' aid indicated that 'doing fine handwork' would be the most requiring activity he/she would perform without difficulty. The 'clinical scenarios' aid estimated that such a patient would be unable to 'drive at night', would have some difficulty 'reading small print' and no difficulty 'doing fine handwork', 'watching TV' or 'recognizing people'. Concordance between modeled and observed responses was fair to substantial. CONCLUSIONS: Simple content-based interpretation aids for the VF-14 scores were developed that should facilitate its use in clinical practice. These aids should be easily generalizable to other quality of life instruments.

Activities of Daily Living↗

Nonparametric regression applied to quantitative structure-activity relationships

Several nonparametric regressors have been applied to modeling quantitative structure-activity relationship (QSAR) data. The simplest regressor, the Nadaraya-Watson, was assessed in a genuine multivariate setting. Other regressors, the local linear and the shifted Nadaraya-Watson, were implemented within additive models--a computationally more expedient approach, better suited for low-density designs. Performances were benchmarked against the nonlinear method of smoothing splines. A linear reference point was provided by multilinear regression (MLR). Variable selection was explored using systematic combinations of different variables and combinations of principal components. For the data set examined, 47 inhibitors of dopamine beta-hydroxylase, the additive nonparametric regressors have greater predictive accuracy (as measured by the mean absolute error of the predictions or the Pearson correlation in cross-validation trails) than MLR. The use of principal components did not improve the performance of the nonparametric regressors over use of the original descriptors, since the original descriptors are not strongly correlated. It remains to be seen if the nonparametric regressors can be successfully coupled with better variable selection and dimensionality reduction in the context of high-dimensional QSARs.

Journal Article↗

An audit of error rates in a UK district hospital transfusion laboratory.

We have audited the error rates of our transfusion laboratory and compared these with error rates reported in the transfusion literature. Error rates were calculated using workload data from the department. The majority of errors that were detected were preanalytical and related to inadequate or incomplete data provided on the sample or request form. These errors were all corrected prior to any further action being taken on that request. The main analytical errors were transcription errors in entering patient identification information into the laboratory computer by transfusion staff together with the incorrect performance of blood group testing. For postanalytical errors the main errors were failure of nursing staff to follow procedures for the collection of blood components prior to transfusion. There were no serious consequences identified of the errors detected in this study. It was difficult to compare these results with those published in the literature in view of the different methodologies that have been reported when error rates have been determined. A standard method should be developed in the UK for calculating error rates so that laboratories can benchmark their performance against comparable organizations.

Blood Grouping and Crossmatching↗

Application of non-parametric regression to quantitative structure-activity relationships.

Several non-parametric regressors have been applied to modelling quantitative structure-activity relationship (QSAR) data. Performances were benchmarked against multilinear regression and the nonlinear method of smoothing splines. Variable selection was explored through systematic combinations of different variables and combinations of principal components. For the training set examined--539 inhibitors of the tyrosine kinase, Syk--the best two-descriptor model had a 5-fold cross-validated q2 of 0.43. This was generated by a multi-variate Nadaraya-Watson kernel estimator. A subsequent, independent, test set of 371 similar chemical entities showed the model had some predictive power. Other approaches did not perform as well. A modest increase in predictive ability can be achieved with three descriptors, but the resulting model is less easy to visualise. We conclude that non-parametric regression offers a potentially powerful approach to identifying predictive, low-dimensional QSARs.

Databases, Factual↗

Reducing disparity in behavioral health services: a report from the American College of Mental Health Administration.

UNLABELLED: The 2003 AMCHA Summit was an initial step. It served to provide a broad outline of the socio-political context and key issues involved in reducing disparities, and it provided some momentum for change. However, much more work remains to be done. The summit clearly demonstrated that the reduction of disparities requires a multi-level approach and multi-disciplinary leaders. As a neutral convener, AMCHA is in a unique position to help advance the debate and lead the field. The membership includes researchers, administrators, clinicians, and policy makers from all levels of the behavioral health system. As noted, a change agenda needs to include efforts at national, state, and local levels involving consumers, providers, purchasers, oversight organizations, and researchers. ACMHA is committed to advancing the field and helping the national effort to reduce disparities. Examples of potential projects include the following: Training: Much has been done to develop effective cultural-competency training modules and to guide states in its implementation. No one should reinvent the wheel at this time. Funding should be targeted to provide incentives to states for dissemination of existing training curricula and the documentation of effectiveness to all providers and administrators. DATA: Nationally, the field will benefit from data standards for the collection of and reporting on system disparities. This will facilitate interstate comparisons and provide baseline data for change efforts. Conducting surveys of providers, health plans, and public behavioral health systems on the availability and current uses of data by race and ethnicity is one example of a useful first step in this process of setting data standards. RESEARCH: Further research on the nature and causes of disparity is needed. There should be systematic research on factors influencing access, treatment, and outcomes for people of different cultures. Initially, because of the difficulties in deciding on standardized outcome measures, the encounter and claims data will provide the most useful information for analysis. Later, as standardized outcome measures are more widely utilized and the data collected, it may be possible to look for racial and ethnic differences in outcomes. The research agenda needs to be developed with a focus on services and health systems research data. Demonstrations: Demonstration efforts are urgently needed, similar to Connecticut's initiative, that integrate data on disparities with provider reporting, performance contracting, and system-wide interventions. These best practices need to be shared with the field. Coordination: The Summit showed that many are eager to learn from others in this area. As we move from further research to demonstration initiatives, AMCHA can play a role in coordinating these projects, particularly at the state and perhaps local levels. State efforts can benefit from best-practice presentations from other states and by an improved understanding of the nature and scope of the change required at a programmatic and local level. Local efforts need to clearly incorporate the views and perspectives of members of the community and consumers. The 2003 ACMHA Summit provided a foundation and a framework for work to proceed at all levels of the behavioral health delivery system. To accomplish meaningful change, we challenge SAMHSA, and the other federal agencies to provide the leadership to (1) develop common and core-performance measures focused on the reduction of disparities, (2) coordinate the research agenda, and (3) facilitate the use of new information technologies to collect and review these data. This is completely consistent with the vision of federal "leadership by example" that has been outlined by the Institute of Medicine (2003b) for the implementation of the "Crossing the Quality Chasm" report. We need to facilitate the efforts of the states and the federal government to identify and reduce disparities and provide a forum for states to share the results of their efforts, to benchmark their performance, and seek technical assistance. Over the next several years, we also expect that states will expand their efforts to implement evidence-based practices. However, we urge these states to implement existing evidence-based practices cautiously, especially with culturally diverse populations, due to the limited representation of ethnically diverse subjects in the research evidence on current practices. We strongly recommend collecting data on practice-based evidence-where effective interventions are routinely identified from existing practice and shared with the field, particularly those practices that seem effective with minority populations.

California↗

Re-evaluating genetic algorithm performance under coordinate rotation of benchmark functions. A survey of some theoretical and practical aspects of genetic algorithms.

In recent years, genetic algorithms (GAs) have become increasingly robust and easy to use. Current knowledge and many successful experiments suggest that the application of GAs is not limited to easy-to-optimize unimodal functions. Several results and GA theory give the impression that GAs easily escape from millions of local optima and reliably converge to a single global optimum. The theoretical analysis presented in this paper shows that most of the widely-used test functions have n independent parameters and that, when optimizing such functions, many GAs scale with an O(n ln n) complexity. Furthermore, it is shown that the current design of GAs and its parameter settings are optimal with respect to independent parameters. Both analysis and results show that a rotation of the coordinate system causes a severe performance loss to GAs that use a small mutation rate. In case of a rotation, the GA's complexity can increase up to O(nn) = O(exp(n ln n)). Future work should find new GA designs that solve this performance loss. As long as these problems have not been solved, the application of GAs will be limited to the optimization of easy-to-optimize functions.

Algorithms↗

TargetQC: A targeted quality control framework for clinical genomic testing.

Reliable genetic testing depends on accurate assessment of sequencing quality in clinically relevant genomic regions that directly influence variant interpretation. We developed TargetQC, a flexible quality control framework that supports user-defined gene sets, coverage thresholds, and variant sets for evaluating sequencing performance across exome sequencing (ES) and genome sequencing (GS) platforms. TargetQC assesses exon and gene coverage, identifies regions meeting predefined coverage thresholds, evaluates variant detection accuracy, and measures sequencing quality at pathogenic variant sites. We applied TargetQC to the reference sample NA12878 and 665 clinical samples across five ES platforms and one GS platform. ES-VendorB and ES-VendorE achieved the most complete coverage of OMIM coding regions in NA12878, whereas ES-VendorD and ES-VendorE showed the highest coverage compliance in clinical samples. ES-VendorB and GS demonstrated the highest variant detection accuracy. TargetQC provides a practical framework for benchmarking sequencing performance and informing platform selection in clinical genomics.

exome sequencing↗

On-site gamma-ray spectroscopic measurements of fission gas release in irradiated nuclear fuel.

An experimental, non-destructive in-pool, method for measuring fission gas release (FGR) in irradiated nuclear fuel has been developed. Using the method, a significant number of experiments have been performed in-pool at several nuclear power plants of the BWR type. The method utilises the 514 keV gamma-radiation from the gaseous fission product (85)Kr captured in the fuel rod plenum volume. A submergible measuring device (LOKET) consisting of an HPGe-detector and a collimator system was utilised allowing for single rod measurements on virtually all types of BWR fuel. A FGR database covering a wide range of burn-ups (up to average rod burn-up well above 60 MWd/kgU), irradiation history, fuel rod position in cross section and fuel designs has been compiled and used for computer code benchmarking, fuel performance analysis and feedback to reactor operators. Measurements clearly indicate the low FGR in more modern fuel designs in comparison to older fuel types.

Gases↗

The Medical Library Association Benchmarking Network: results.

OBJECTIVE: This article presents some limited results from the Medical Library Association (MLA) Benchmarking Network survey conducted in 2002. Other uses of the data are also presented. METHODS: After several years of development and testing, a Web-based survey opened for data input in December 2001. Three hundred eighty-five MLA members entered data on the size of their institutions and the activities of their libraries. The data from 344 hospital libraries were edited and selected for reporting in aggregate tables and on an interactive site in the Members-Only area of MLANET. The data represent a 16% to 23% return rate and have a 95% confidence level. RESULTS: Specific questions can be answered using the reports. The data can be used to review internal processes, perform outcomes benchmarking, retest a hypothesis, refute a previous survey findings, or develop library standards. The data can be used to compare to current surveys or look for trends by comparing the data to past surveys. CONCLUSIONS: The impact of this project on MLA will reach into areas of research and advocacy. The data will be useful in the everyday working of small health sciences libraries as well as provide concrete data on the current practices of health sciences libraries.

Benchmarking↗

Using risk assessment to evaluate adverse selection under capitated contracts.

Most provider organizations rely on health plan or market-based information about capitation rates, per member per month costs, and utilization trends to benchmark their performance. However, these statistics can be misleading because of differences in enrollee mix and contracting terms across provider organizations. This article describes the limitations of health plan contract provisions in protecting against adverse selection. It describes various actuarial and statistical data sources for evaluation of adverse selection. The article then presents various approaches to risk adjustment on population basis and their use in quantifying adverse selection for health plan contract negotiations.

Actuarial Analysis↗

Calibration of the n-electron valence state perturbation theory approach.

Extensive tests have been performed to benchmark and to compare with second-order perturbation theory based on a complete active space self-consistent field reference function (CASPT2), the recently developed n-electron valence state perturbation theory at second order (NEVPT2). Test calculations included the group fifteen diatomic molecules X(2) (X=N, P, As, and Sb) and the (4)S/(2)D and (4)S/(2)P splittings for the corresponding atoms, the (1)A(1)-(3)B(1) splittings for CH(2) and SiH(2), and the absorption spectra of pyrrole and of Cu(Imidazole)(2)(SH)(SH(2))(+), which is a model for plastocyanin. Comparisons with full configuration-interaction calculations and experimental data show that the accuracy of NEVPT2 is in most cases even better than CASPT2. Where intruder states hamper the CASPT2 calculations, NEVPT2 performs significantly better. Care is needed in the choice of active orbitals, for example in the calculation of the (4)S/(2)D and (4)S/(2)P splittings for the group fifteen atoms. This is due to the different treatment of orbitals belonging to the inactive or active spaces, making the NEVPT2 not invariant for the choice of active space, even in cases where the multiconfiguration self-consistent field energy is invariant.

Journal Article↗

Scalable medium-density genotyping platforms for cultivar identification, pedigree authentication, marker-assisted and genomic selection, and other applications in strawberry.

A broad spectrum of high-density genotyping approaches, including single-nucleotide polymorphism (SNP) arrays, genotyping-by-sequencing, and whole-genome reduced-representation sequencing, have been shown to perform well in strawberry (Fragaria × ananassa), despite the inherent complexity of the octoploid genome. While these approaches are effective, their routine deployment in breeding programs can be constrained by cost, computational requirements, and workflow complexity. In parallel, many breeding programs continue to rely on locus-specific assays for marker-assisted selection, resulting in fragmented and inefficient genotyping strategies. Here, we describe medium-density amplicon-based genotyping platforms for strawberry designed to provide cost-effective, turnkey solutions that integrate markers used for marker-assisted selection with genome-wide markers suitable for genomic prediction in a single laboratory assay. These platforms were developed by targeting 1,650 or 4,811 target SNPs via amplicon sequencing, and are interoperable with existing high-density genotyping resources, including a widely used 50K SNP array, thereby facilitating data integration across platforms. We benchmarked their performance relative to the 50K SNP array across breeding-relevant applications, including identity and purity testing, pedigree authentication, marker-assisted selection, and genomic selection, and further evaluated the feasibility of genotype imputation to enhance genome-wide information content. Across analyses, the 1,650- and 4,811-amplicon platforms produced results comparable to higher-density platforms while substantially reducing genotyping cost and analytical overhead. This work demonstrates that targeted amplicon-based genotyping can support efficient, scalable, and integrated genome-informed breeding, enabling the routine application of both marker-assisted and genomic selection within strawberry breeding workflows. Open-source R workflows are provided to support streamlined analyses in breeding contexts.

Fragaria↗

Selecting indicators for the quality of diabetes care at the health systems level in OECD countries.

PURPOSE: In the context of the Organization for Economic Cooperation and Development (OECD) Quality Indicators Project, a set of quality indicators for diabetes care was developed, to be used for benchmarking the performance of health care systems. BACKGROUND: Diabetes complications markedly reduce quality and length of life and are also responsible for enormous health care costs. A large body of evidence has shown that several effective treatments and practices may substantially reduce this burden. However, a marked variability has been documented in preventive and therapeutic approaches, thus suggesting that the level of diabetes care currently delivered may not produce the possible health-related gains. METHODS: Existing quality indicators have been reviewed, with particular attention to the work done by the National Diabetes Quality Improvement Alliance (NDQIA) in the US. All the measures identified were evaluated for their importance, scientific soundness, and feasibility. In addition, the panel members selected new distal outcome measures. These measures are currently not used in provider comparisons, but they could reveal valuable insight into the differential performance of health systems. RESULTS: /b>. Four process and two proximal outcome measures were selected among those endorsed by the NDQIA. In addition, three new long-term outcome measures have been proposed to gain insight into whether and to what degree differences in the processes and intermediate outcomes that are captured by the established measures translate into better outcomes for patients. CONCLUSIONS: The measures selected can contribute to policymakers' and researchers' understanding of differences in the quality of diabetes care between health systems. Further work is required to assess the availability of reliable and comparable data across OECD countries.

Benchmarking↗