PubMed · 2797137
The source did not provide an abstract. Follow the original record for more information.
Explore connections, maps & timelines
1989-09-28. Peer review.. https://pubmed.ncbi.nlm.nih.gov/2797137/
Cite the original work for its findings. Save a collection to share your selection of sources.
KEEP EXPLORING
CONTEXT: Biomedical manuscripts undergo substantive change as a result of the peer review and editorial revision processes. OBJECTIVE: To characterize quantitatively problems in manuscripts identified during peer review and changes made to address these problems. DESIGN AND SETTING: Descriptive analysis of manuscripts submitted to and articles published by the Annals of Internal Medicine. A taxonomy of problems that occur in reporting clinical research was developed from analysis of changes made to 7 manuscripts between submission and publication (published October 15, 1996, and November 1, 1996). The taxonomy was used to characterize changes to 12 additional manuscripts (published January 15, 1997, to April 1, 1997). MAIN OUTCOME MEASURE: Types of problems necessitating changes to manuscripts during peer review and revision. RESULTS: Changes occurred because of 5 types of problems: too much information, too little information, inaccurate information, misplaced information, and structural problems. Changes most often occurred because information was missing or extraneous. The distribution of changes seemed to be influenced by the type of information involved (such as background or conclusions). CONCLUSION: The proposed framework may be useful for characterizing quantitatively the effects of peer review and for comparing those effects across editors, journals, and specialties.
CONTEXT: Quality of reviewers is crucial to journal quality, but there are usually too many for editors to know them all personally. A reliable method of rating them (for education and monitoring) is needed. OBJECTIVE: Whether editors' quality ratings of peer reviewers are reliable and how they compare with other performance measures. DESIGN: A 3.5-year prospective observational study. SETTING: Peer-reviewed journal. PARTICIPANTS: All editors and peer reviewers who reviewed at least 3 manuscripts. MAIN OUTCOME MEASURES: Reviewer quality ratings, individual reviewer rate of recommendation for acceptance, congruence between reviewer recommendation and editorial decision (decision congruence), and accuracy in reporting flaws in a masked test manuscript. INTERVENTIONS: Editors rated the quality of each review on a subjective 1 to 5 scale. RESULTS: A total of 4161 reviews of 973 manuscripts by 395 reviewers were studied. The within-reviewer intraclass correlation was 0.44 (P<.001), indicating that 20% of the variance seen in the review ratings was attributable to the reviewer. Intraclass correlations for editor and manuscript were only 0.24 and 0.12, respectively. Reviewer average quality ratings correlated poorly with the rate of recommendation for acceptance (R=-0.34) and congruence with editorial decision (R=0.26). Among 124 reviewers of the fictitious manuscript, the mean quality rating for each reviewer was modestly correlated with the number of flaws they reported (R=0.53). Highly rated reviewers reported twice as many flaws as poorly rated reviewers. CONCLUSIONS: Subjective editor ratings of individual reviewers were moderately reliable and correlated with reviewer ability to report manuscript flaws. Individual reviewer rate of recommendation for acceptance and decision congruence might be thought to be markers of a discriminating (ie, high-quality) reviewer, but these variables were poorly correlated with editors' ratings of review quality or the reviewer's ability to detect flaws in a fictitious manuscript. Therefore, they cannot be substituted for actual quality ratings by editors.