PubMed Health⌕ Search

Biomedical subjects

D G Altman

Publications and source records attributed to D G Altman.

At least 19 recordsLinked to original sources
PubMed2006

Systematic reviews of clinical decision tools for acute abdominal pain.

OBJECTIVES: To review for acute abdominal pain (AAP), the diagnostic accuracies of combining decision tools (DTs) and doctors aided by DTs compared with those of unaided doctors. Also to evaluate the impact of providing doctors with an AAP DT on patient outcomes, clinical decisions and actions, what factors are likely to determine the usage rates and usability of a DT and the associated costs and likely cost-effectiveness of these DTs in routine use in the UK. DESIGN: Electronic databases were searched up to 1 July 2003. REVIEW METHODS: Data from each eligible study were extracted. Potential sources of heterogeneity were extracted for both questions. For the accuracy review, meta-analysis was conducted. Among studies comparing diagnostic accuracies of DTs with unaided doctors, error rate ratios provided estimates of the differences between the false-negative and false-positive rates of the DT and unaided doctors' performance. Pooled error rate ratios and 95% confidence intervals (CIs) for false-negative rates and false-positive rates were computed. Metaregression was used to explore heterogeneity. RESULTS: Thirty-two studies from 27 articles, all based in secondary care, were eligible for the review of DT accuracies, while two were eligible for the review of the accuracy of hospital doctors aided by DTs. Sensitivities and specificities for DTs ranged from 53 to 99% and from 30 to 99%, respectively. Those for unaided doctors ranged from 64 to 93% and from 39 to 91%, respectively. Thirteen studies reported false-positive and false-negative rates for both DTs and unaided doctors, enabling a direct comparison of their performance. In random effects meta-analyses, DTs had significantly lower false-positive rates (error rate ratio 0.62, 95% CI 0.46 to 0.83) than unaided doctors. DTs may have higher false-negative rates than unaided doctors (error rate ratio 1.34, 95% CI 0.93 to 1.93). Significant heterogeneity was present. Two studies compared the diagnostic accuracies of doctors aided by DTs to unaided doctors. In a multiarm cluster randomised controlled trial (n = 5193), the diagnostic accuracy of doctors not given access to DTs was not significantly worse (sensitivity 28.4% and specificity 96.0%) than that of three groups of aided doctors (sensitivities of 42.4-47.9%, and specificities of 95.5-96.5%, respectively). In an uncontrolled before-and-after study (n = 1484), the sensitivities and specificities of aided and unaided doctors were 95.5% and 91.5% (p = 0.24) and 78.1% and 86.4% (p < 0.001), respectively. The metaregression of DTs showed that prospective test-set validation at the site of the tool's development was associated with considerably higher diagnostic accuracy than prospective test-set validation at an independent centre [relative diagnostic odds ratio (RDOR) 8.2; 95% CI 3.1 to 14.7]. It also showed that the earlier in the year the study was performed the higher the performance (RDOR 0.88, 0.83 to 0.92), that when developers evaluated their own DT there was better performance than when independent evaluators carried out the study (RDOR = 3.0, 1.3 to 6.8), and that there was no evidence of association between other quality indicators and DT accuracy. The one eligible study of the impact study review, a four-arm cluster randomised trial (n = 5193), showed that hospital admission rates of patients by doctors not allocated to a DT (42.8%) were significantly higher than those by doctors allocated to three combinations of decision support (34.2-38.5%) (p < 0.001). There was no evidence of a difference between perforation rates (p = 0.19) and negative laparotomy rates in the four trial arms (p = 0.46). Usage rates of DTs by doctors in accident and emergency departments ranged from 10 to 77% in the six studies that reported them. Possible determinants of usability include the reasoning method used, the number of items used and the output format. A deterministic cost-effectiveness comparison demonstrated that a paper checklist is likely to be 100-900 times more cost-effective than a computer-based DT, under stated assumptions. CONCLUSIONS: With their significantly greater specificity and lower false-positive rates than doctors, DTs are potentially useful in confirming a diagnosis of acute appendicitis, but not in ruling it out. The clinical use of well-designed, condition-specific paper or computer-based structured checklists is promising as a way to improve impact on patient outcomes, subject to further research.

Abdominal Pain↗
PubMed2006

Reporting recommendations for tumor marker prognostic studies (remark).

Despite years of research and hundreds of reports on tumor markers in oncology, the number of markers that have emerged as clinically useful is pitifully small. Often, initially reported studies of a marker show great promise, but subsequent studies on the same or related markers yield inconsistent conclusions or stand in direct contradiction to the promising results. It is imperative that we attempt to understand the reasons that multiple studies of the same marker lead to differing conclusions. A variety of methodologic problems have been cited to explain these discrepancies. Unfortunately, many tumor marker studies have not been reported in a rigorous fashion, and published articles often lack sufficient information to allow adequate assessment of the quality of the study or the generalizability of study results. The development of guidelines for the reporting of tumor marker studies was a major recommendation of the National Cancer Institute - European Organisation for Research and Treatment of Cancer (NCI - EORTC) First International Meeting on Cancer Diagnostics in 2000. As for the successful CONSORT initiative for randomized trials and for the STARD statement for diagnostic studies, we suggest guidelines to provide relevant information about the study design, preplanned hypotheses, patient and specimen characteristics, assay methods, and statistical analysis methods. In addition, the guidelines suggest helpful presentations of data and important elements to include in discussions. The goal of these guidelines is to encourage transparent and complete reporting so that the relevant information will be available to others to help them to judge the usefulness of the data and understand the context in which the conclusions apply.

Biomarkers, Tumor↗
PubMed2005

REporting recommendations for tumour MARKer prognostic studies (REMARK).

Despite years of research and hundreds of reports on tumour markers in oncology, the number of markers that have emerged as clinically useful is pitifully small. Often initially reported studies of a marker show great promise, but subsequent studies on the same or related markers yield inconsistent conclusions or stand in direct contradiction to the promising results. It is imperative that we attempt to understand the reasons that multiple studies of the same marker lead to differing conclusions. A variety of methodological problems have been cited to explain these discrepancies. Unfortunately, many tumour marker studies have not been reported in a rigorous fashion, and published articles often lack sufficient information to allow adequate assessment of the quality of the study or the generalisability of the study results. The development of guidelines for the reporting of tumour marker studies was a major recommendation of the US National Cancer Institute and the European Organisation for Research and Treatment of Cancer (NCI-EORTC) First International Meeting on Cancer Diagnostics in 2000. Similar to the successful CONSORT initiative for randomised trials and the STARD statement for diagnostic studies, we suggest guidelines to provide relevant information about the study design, preplanned hypotheses, patient and specimen characteristics, assay methods, and statistical analysis methods. In addition, the guidelines suggest helpful presentations of data and important elements to include in discussions. The goal of these guidelines is to encourage transparent and complete reporting so that the relevant information will be available to others to help them to judge the usefulness of the data and understand the context in which the conclusions apply.

Biomarkers, Tumor↗
PubMed2005

Outcome selection bias in meta-analysis.

Publication bias has been previously identified as a threat to the validity of a meta-analysis. Recently, new evidence has documented an additional threat to validity, the selective reporting of trial outcomes within published studies. Several diseases have several possible measures of outcome. Some articles might report only a selection of those outcomes, perhaps those with statistically significant results. In this article, we review this problem while addressing the questions: what is within-study selective reporting? how common is it? why is it done? how can it mislead? how can it be detected?, and finally, what is the solution? We recommend that both publication bias and selective reporting should be routinely investigated in systematic reviews.

Bias↗
PubMed2005

Issues in data monitoring and interim analysis of trials.

OBJECTIVES: To address issues about data monitoring committees (DMCs) for randomised controlled trials (RCTs). DATA SOURCES: Electronic databases. Handsearching of selected books. Personal contacts with experts in the field. REVIEW METHODS: Systematic literature reviews of DMCs and small group processes in decision-making; sample surveys of: reports of RCTs, recently completed and ongoing RCTs and policies of major organisations involved in RCTs; case studies of four DMCs; and interviews with experienced DMC members. All focused on 23 prestated questions. RESULTS: Although still a minority, RCTs increasingly have DMCs. There is wide agreement that nearly all trials need some form of data monitoring. Central to the role of the DMC is monitoring accumulating evidence related to benefit and toxicity; variation in emphasis has been reflected in the plethora of names. DMCs for trials performed for regulatory purposes should be aware of any special requirements and regulatory consequences. Advantages were identified for both larger and smaller DMCs. There is general agreement that a DMC should be independent and multidisciplinary. Consumer and ethicist membership is controversial. The chair is recognised as being particularly influential, and likely to be most effective if he or she is experienced, understands both statistical and clinical issues, and is facilitating in style and impartial. There is no evidence available to judge suggested approaches to training. The review suggested that costs should be covered, but other rewards must be so minimal as to not affect decision-making. It is usual to have a minimum frequency of DMC meetings, with evidence that face-to-face meetings are preferable. It is common to have open sessions and a closed session. A report to a DMC should cover benefits and risks in a balanced way, summarised in an accessible style, avoiding excessive detail, and be as current as possible. Disadvantages of blinded analyses seem to outweigh advantages. Information about comparable studies should be included, although interaction with the DMCs of similar ongoing trials is controversial. A range of formal statistical approaches can be used, although this is only one of a number of considerations. DMCs usually reach decisions by consensus, but other approaches are sometimes used. The general, but not unanimous, view is that DMCs should be advisory rather than executive on the basis that it is the trial organisers who are ultimately responsible for the conduct of the trial. CONCLUSIONS: Some form of data monitoring should be considered for all RCTs, with reasons given where there is no DMC or when any member is not independent. An early DMC meeting is helpful, determining roles and responsibilities; planned operations can be agreed with investigators and sponsors/funders. A template for a DMC charter is suggested. Competing interests should be declared. DMC size (commonly three to eight people) is chosen to optimise performance. Members are usually independent and drawn from appropriate backgrounds, and some, particularly the chair, are experienced. A minimum frequency of meetings is usually agreed, with flexibility for more if needed. The DMC should understand and agree the statistical approach (and guidelines) chosen, with both the DMC statistician and analysis statistician competent to apply the method. A DMC's primary purpose is to ensure that continuing a trial according to its protocol is ethical, taking account of both individual and collective ethics. A broader remit in respect of wider ethical issues is controversial; arguably, these are primarily the responsibility of research ethics committees, trial steering committees and investigators. The DMC should know the range of recommendations or decisions open to it, in advance. A record should be kept describing the key issues discussed and the rationale for decisions taken. Errors are likely to be reduced if a DMC makes a thorough review of the evidence and has a clear understanding of how it should function, there is active participation by all members, differences are resolved through discussion and there is systematic consideration of the various decision options. DMCs should be encouraged to comment on draft final trial reports. These should include information about the data monitoring process and detail the DMC membership. It is recommended that groups responsible for data monitoring be given the standard name 'Data Monitoring Committee' (DMC). Areas for further research include: widening DMC membership beyond clinicians, trialists and statisticians; initiatives to train DMC members; methods of DMC decision-making; 'open' data monitoring; DMCs covering a portfolio of trials rather than single trials; DMC size and membership, incorporating issues of group dynamics; empirical study of the workings of DMCs and their decision-making, and which trials should or should not have a DMC.

Clinical Trials Data Monitoring Committees↗
PubMed2005

Indirect comparisons of competing interventions.

OBJECTIVES: To survey the frequency of use of indirect comparisons in systematic reviews and evaluate the methods used in their analysis and interpretation. Also to identify alternative statistical approaches for the analysis of indirect comparisons, to assess the properties of different statistical methods used for performing indirect comparisons and to compare direct and indirect estimates of the same effects within reviews. DATA SOURCES: Electronic databases. REVIEW METHODS: The Database of Abstracts of Reviews of Effects (DARE) was searched for systematic reviews involving meta-analysis of randomised controlled trials (RCTs) that reported both direct and indirect comparisons, or indirect comparisons alone. A systematic review of MEDLINE and other databases was carried out to identify published methods for analysing indirect comparisons. Study designs were created using data from the International Stroke Trial. Random samples of patients receiving aspirin, heparin or placebo in 16 centres were used to create meta-analyses, with half of the trials comparing aspirin and placebo and half heparin and placebo. Methods for indirect comparisons were used to estimate the contrast between aspirin and heparin. The whole process was repeated 1000 times and the results were compared with direct comparisons and also theoretical results. Further detailed case studies comparing the results from both direct and indirect comparisons of the same effects were undertaken. RESULTS: Of the reviews identified through DARE, 31/327 (9.5%) included indirect comparisons. A further five reviews including indirect comparisons were identified through electronic searching. Few reviews carried out a formal analysis and some based analysis on the naive addition of data from the treatment arms of interest. Few methodological papers were identified. Some valid approaches for aggregate data that could be applied using standard software were found: the adjusted indirect comparison, meta-regression and, for binary data only, multiple logistic regression (fixed effect models only). Simulation studies showed that the naive method is liable to bias and also produces over-precise answers. Several methods provide correct answers if strong but unverifiable assumptions are fulfilled. Four times as many similarly sized trials are needed for the indirect approach to have the same power as directly randomised comparisons. Detailed case studies comparing direct and indirect comparisons of the same effect show considerable statistical discrepancies, but the direction of such discrepancy is unpredictable. CONCLUSIONS: Direct evidence from good-quality RCTs should be used wherever possible. Without this evidence, it may be necessary to look for indirect comparisons from RCTs. However, the results may be susceptible to bias. When making indirect comparisons within a systematic review, an adjusted indirect comparison method should ideally be used employing the random effects model. If both direct and indirect comparisons are possible within a review, it is recommended that these be done separately before considering whether to pool data. There is a need to evaluate methods for the analysis of indirect comparisons for continuous data and for empirical research into how different methods of indirect comparison perform in cases where there is a large treatment effect. Further study is needed into when it is appropriate to look at indirect comparisons and when to combine both direct and indirect comparisons. Research into how evidence from indirect comparisons compares to that from non-randomised studies may also be warranted. Investigations using individual patient data from a meta-analysis of several RCTs using different protocols and an evaluation of the impact of choosing different binary effect measures for the inverse variance method would also be useful.

Bias↗
PubMed2004

Missing covariate data within cancer prognostic studies: a review of current reporting and proposed guidelines.

Prognostic models play a crucial role in the clinical decision-making process. Unfortunately, missing covariate data impede the construction of valid and reliable models, potentially introducing bias, if handled inappropriately. The extent of missing covariate data within reported cancer prognostic studies, the current handling and the quality of reporting this missing covariate data are unknown. Therefore, a review was conducted of 100 articles reporting multivariate survival analyses to assess potential prognostic factors, published within seven cancer journals in 2002. Missing covariate data is a common occurrence in studies performing multivariate survival analyses, being apparent in 81 of the 100 articles reviewed. The percentage of eligible cases with complete data was obtainable in 39 articles, and was <90% in 17 of these articles. The methods used to handle incomplete covariates were obtainable in 32 of the 81 articles with known missing data and the most commonly reported approaches were complete case and available case analysis. This review has highlighted deficiencies in the reporting of missing covariate data. Guidelines for presenting prognostic studies with missing covariate data are proposed, which if followed should clarify and standardise the reporting in future articles.

Decision Making↗
PubMed2003

Applying the right statistics: analyses of measurement studies.

The study of measurement error, observer variation and agreement between different methods of measurement are frequent topics in the imaging literature. We describe the problems of some applications of correlation and regression methods to these studies, using recent examples from this literature. We use a simulated example to show how these problems and misinterpretations arise. We describe the 95% limits of agreement approach and a similar, appropriate, regression technique. We discuss the difference vs. mean plot, and the pitfalls of plotting difference against one variable only. We stress that these are questions of estimation, not significance tests, and show how confidence intervals can be found for these estimates.

Confidence Intervals↗
PubMed2003

Evaluating non-randomised intervention studies.

OBJECTIVES: To consider methods and related evidence for evaluating bias in non-randomised intervention studies. DATA SOURCES: Systematic reviews and methodological papers were identified from a search of electronic databases; handsearches of key medical journals and contact with experts working in the field. New empirical studies were conducted using data from two large randomised clinical trials. METHODS: Three systematic reviews and new empirical investigations were conducted. The reviews considered, in regard to non-randomised studies, (1) the existing evidence of bias, (2) the content of quality assessment tools, (3) the ways that study quality has been assessed and addressed. (4) The empirical investigations were conducted generating non-randomised studies from two large, multicentre randomised controlled trials (RCTs) and selectively resampling trial participants according to allocated treatment, centre and period. RESULTS: In the systematic reviews, eight studies compared results of randomised and non-randomised studies across multiple interventions using meta-epidemiological techniques. A total of 194 tools were identified that could be or had been used to assess non-randomised studies. Sixty tools covered at least five of six pre-specified internal validity domains. Fourteen tools covered three of four core items of particular importance for non-randomised studies. Six tools were thought suitable for use in systematic reviews. Of 511 systematic reviews that included non-randomised studies, only 169 (33%) assessed study quality. Sixty-nine reviews investigated the impact of quality on study results in a quantitative manner. The new empirical studies estimated the bias associated with non-random allocation and found that the bias could lead to consistent over- or underestimations of treatment effects, also the bias increased variation in results for both historical and concurrent controls, owing to haphazard differences in case-mix between groups. The biases were large enough to lead studies falsely to conclude significant findings of benefit or harm. Four strategies for case-mix adjustment were evaluated: none adequately adjusted for bias in historically and concurrently controlled studies. Logistic regression on average increased bias. Propensity score methods performed better, but were not satisfactory in most situations. Detailed investigation revealed that adequate adjustment can only be achieved in the unrealistic situation when selection depends on a single factor. CONCLUSIONS: Results of non-randomised studies sometimes, but not always, differ from results of randomised studies of the same intervention. Non-randomised studies may still give seriously misleading results when treated and control groups appear similar in key prognostic factors. Standard methods of case-mix adjustment do not guarantee removal of bias. Residual confounding may be high even when good prognostic data are available, and in some situations adjusted results may appear more biased than unadjusted results. Although many quality assessment tools exist and have been used for appraising non-randomised studies, most omit key quality domains. Healthcare policies based upon non-randomised studies or systematic reviews of non-randomised studies may need re-evaluation if the uncertainty in the true evidence base was not fully appreciated when policies were made. The inability of case-mix adjustment methods to compensate for selection bias and our inability to identify non-randomised studies that are free of selection bias indicate that non-randomised studies should only be undertaken when RCTs are infeasible or unethical. Recommendations for further research include: applying the resampling methodology in other clinical areas to ascertain whether the biases described are typical; developing or refining existing quality assessment tools for non-randomised studies; investigating how quality assessments of non-randomised studies can be incorporated into reviews and the implications of individual quality features for interpretation of a review's results; examination of the reasons for the apparent failure of case-mix adjustment methods; and further evaluation of the role of the propensity score.

Bias↗