SEARCH · PubMed Health

Results for “scoring rubric”

Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 recordsLinked to original sources

PubMed2004

Using scoring rubrics to facilitate assessment and evaluation of graduate-level nursing students.

Graduate-level nursing students need to be able to express concepts both verbally and in writing. For many students this will be an accumulative ongoing process that is guided by the instructors in each successive class. It is important that instructors use a tool that can consistently assess and evaluate student work, while providing feedback. Scoring rubrics do this by directing students toward areas that need improvement, while also giving credit for those items that are done correctly. Scoring rubrics can be used for both written and oral assignments and can be individualized to fit the context of the subject matter. Although rubrics have been largely associated with undergraduate students, they can improve the ability of graduate nursing students in the areas of written and verbal communication.

Communication↗

PubMed2003

Validity and reliability of portfolio assessment of competency in a baccalaureate dental hygiene program.

This study examined the validity and reliability of portfolio assessment using Messick's unified framework of construct validity. Theoretical and empirical evidence was sought for six aspects of construct validity. Seven faculty raters evaluated twenty student portfolios using a primary trait analysis scoring rubric. A significant relationship (r = .81-.95; p < .01) between the seven subscales in the scoring rubric demonstrates measurement of a common construct. There was a significant relationship between portfolios and GPA (r = .70; p < .01) and the NBDHE (r = .60; p < .01). The relationship between portfolios and the Central Regional Dental Testing Service (CRDTS) examination was both weak and nonsignificant (r = .19; p > .05). A fully crossed, two-facet generalizability (G) study design was used to examine reliability. ANOVA demonstrated that the greatest source of variance was the scoring rubric itself, accounting for 78 percent of the total variance. The smallest source of variance was the interaction between portfolio and rubric (1.15 percent). Faculty rater variance accounted for only 1.28 percent of total variance. A phi coefficient of .86, analogous to a reliability coefficient in classical test theory, was obtained in the decision study by increasing the subscales to fourteen and decreasing faculty raters to three. In conclusion, the pattern of findings from this study suggests that portfolios can serve as a valid and reliable measure for assessing student competency.

Adult↗

PubMed2025

Clinical Actionability of Genetic Findings in Cerebral Palsy: A Systematic Review and Meta-Analysis.

IMPORTANCE: Single gene variants can cause cerebral palsy (CP) phenotypes, yet the impact of genetic diagnosis on CP clinical management has not been systematically evaluated. OBJECTIVE: To evaluate how frequently genetic testing results would prompt changes in care for individuals with CP and the clinical utility of precision medicine therapies. DATA SOURCES: Published pathogenic or likely pathogenic variants in OMIM genes identified with exome sequencing in clinical (n = 1345) or research (n = 496) cohorts of CP were analyzed. A systematic literature review for evidence of effective therapies for specific genetic etiologies was performed. STUDY SELECTION: Nonstandard interventions that led to a detectable improvement in a defined outcome in individuals with variants in the gene of interest were included. DATA EXTRACTION AND SYNTHESIS: Literature was evaluated using PRISMA guidelines. A diverse, expert working group was established, scoring rubrics adapted, and scoring consensus built with a modified Delphi approach. MAIN OUTCOMES AND MEASURES: Overall clinical utility was calculated from metrics assessing outcome severity if left untreated, safety and practicality of the intervention, and anticipated intervention efficacy on a scale from 0 to 3. RESULTS: Of 1841 patients with CP who underwent exome sequencing, 502 (27%) had pathogenic or likely pathogenic variants related to their phenotype. A total of 243 different genes were identified. In 1841 patients with identified genetic etiologies of CP, 140 (8%) had a genetic etiology classified as actionable, defined as prompting a change in clinical management. Also identified were 58 of 243 genes with pathogenic or likely pathogenic variants with actionable treatment options: 16 targeting the primary disease mechanism, 16 with specific prevention strategies, and 26 with specific symptom management. The level of evidence was also graded according to ClinGen criteria; 45 of 101 interventions (44.6%) had evidence class D or below. The potential interventions have clinical utility with 98 of 101 outcomes (97%) being moderate-high severity if left untreated and 63 of 101 interventions (62%) predicted to be of moderate-high efficacy. Most interventions (72 of 101 [71%]) were considered moderate-high safety and practicality. CONCLUSIONS AND RELEVANCE: The findings indicate that actionable genetic findings occurred in 8% of individuals referred for genetic testing with CP. Evaluation of potential efficacy, outcome severity, and intervention safety and practicality indicates moderate-high clinical utility of these genetic findings. Genetic sequencing can identify precision medicine interventions that provide clinical benefit to individuals with CP. The relatively limited evidence base underscores the need for additional research.

Humans↗

PubMed1997

On-line performance assessment using rating scales.

The purpose of this paper is to report on the development of the on-line performance assessment instrument--the Assessment of Motor and Process Skills (AMPS). Issues that will be addressed in the paper include: (a) the establishment of the scoring rubric and its implementation in an extended Rasch model, (b) training of raters, (c) validation of the scoring rubric and procedures for monitoring the internal consistency of raters, and (d) technological implementation of the assessment instrument in a computerized program.

Activities of Daily Living↗

PubMed2003

Measuring inter-rater reliability of the sequenced performance inventory and reflective assessment of learning (SPIRAL).

PURPOSE: To measure the inter-rater reliability of the Sequenced Performance Inventory and Reflective Assessment of Learning (SPIRAL), a twenty-three item scoring rubric designed to assess first and second-year students' competencies such as "acquisition of knowledge," "peer teaching and communication skills," and "professional behavior" in a problem-based learning (PBL) curriculum at the University of North Dakota School of Medicine and Health Sciences. METHOD: In 2001, the authors constructed a 69-item multiple-choice questionnaire consisting of descriptions of "prototypical" (representing real students) PBL student performances. For each of the 23 SPIRAL items, the authors chose competency descriptions of an "emerging," a "developing," and an "advanced" student (69 prototypical students). These descriptions were obtained from narrative comments by PBL facilitators of real students in their classes in 2000-2001. Seventeen experienced facilitators (PBL tutors) were subsequently asked to rank the 69 prototypical students based on SPIRAL anchor points in the three competency levels. Kendall's coefficient of concordance was used to test inter-rater reliability. Modes were also determined to illustrate the extent to which the ratings of the 17 facilitators aligned with the ratings anticipated by the authors. RESULTS: Overall, inter-rater scoring for all items combined was considered to be reliable (Kendall's coefficient of concordance W =.75). Inter-rater scoring was also determined for each of the 23 SPIRAL items, with Ws ranging from.97 to.53. Facilitators rated students according to 55 of the 69 SPIRAL descriptors as the researchers anticipated. CONCLUSIONS: The scoring rubric is reliable. Further criterion-related validation study is warranted.

Clinical Competence↗

PubMed1996

Fellow of the American Dietetic Association credentialing program: development and implementation of a portfolio-based assessment.

This report has described the measurement and operational procedures used for the design, development, and implementation of the newly inaugurated Fellow of The American Dietetic Association credentialing program. The program was established to identify and certify registered dietitians who possess the characteristics of advanced-level practice. The main features of the Fellow program are as follows. It is based on the results of an empirical study that identified the characteristics of advanced-level practice. It is guided by the Fellow Assessment Plan, which specifies the content and level of dietetics practice involved and the psychometric and operational requirements for sound measurement. It is a performance-based portfolio assessment of six professional characteristics (education, work experience, professional achievement, professional roles, professional contacts, and approach to practice). It uses objective automated computer scoring and independent peer-review grading of candidate materials in relation to an explicit scoring rubric. It requires statistical analysis of the database of candidate scores, which is conducted to ensure that the scoring of candidate materials is accurate and reliable and that the assessment, as a measurement instrument, is psychometrically sound. It involves a standard-setting workshop in which a panel of subject-matter experts establish the lowest level of acceptable candidate performance on Approach to Practice; the passing standard for the other five characteristics was preestablished by the empirical study. A careful review of all evidence on the development and implementation of the 1994 Fellow assessment suggests that these procedures are technically sound and produce psychometrically defensible results. Our evaluation and commentary from CDR and CTB staff, subject-matter expert participants, and the candidates themselves suggested a number of ways to improve the Fellow program in subsequent administrations of the assessment. Changes have been implemented and CDR will continue to consider all data in its ongoing improvement efforts.

Credentialing↗

PubMed2001

Comparing holistic and analytic scoring for performance assessment with many-facet Rasch model.

This paper compares holistic and analytic scoring methods to explore how the alternative scorings can make differences for performance assessment using many-faceted Rasch model. The model is especially pertinent for analyzing performance assessment since the model can include several facets simultaneously. Forty three students' reports for social studies were scored by four raters with the holistic method and the analytic method. The result demonstrated that scoring rubrics could be improved by investigating rating scale categories. Also, the comparison of student scores between the two scoring methods revealed that the selection of scoring methods might not be significant for the relative comparison of students but it could have serious implication for the assessment of students' absolute abilities. For rater severity, analytic scoring provided more consistency than holistic scoring. These findings can be used to select and improve scoring methods for performance assessment.

Adolescent↗

PubMed2004

Assessing the written communication skills of medical school graduates.

The ECFMG Clinical Skills Assessment (CSA) was developed to evaluate whether graduates of international medical schools (IMGs) are ready to enter graduate training programs in the United States. The patient note (PN) exercise, conducted after a 15-minute interview with a standardized patient (SP), is specifically used to assess a candidate's ability to summarize and synthesize the data collected. On a yearly basis, approximately 75,000 patient notes are reviewed and scored by physician raters. Recent changes to the PN scoring rubric, combined with enhancements to quality assurance procedures, mandate that additional evidence be provided to support the intended use of PN scores. The purpose of this study was to further investigate the psychometric adequacy of PN scores. Generalizability analyses suggest that while variability in PN ratings can be attributed to the choice of rater, candidate scores are reproducible over the 10-encounter CSA. The relationship of PN scores with other related ability measures and select candidate characteristics provides additional evidence to support the validity of the written exercise.

Cohort Studies↗

PubMed2005

Assessment of ophthalmology resident on-call performance.

PURPOSE: To design and implement a valid tool for assessment of ophthalmology resident on-call performance. DESIGN: Retrospective chart audit. SETTING: Tertiary care academic ophthalmology programs. PARTICIPANTS: Ophthalmology faculty and residents at the University of Cincinnati and the University of Iowa. METHODS: A 1-page on-call assessment tool (OCAT) and scoring rubric were developed to evaluate ophthalmology resident on-call performance. A retrospective chart audit of consecutive resident on-call charts was performed at the University of Cincinnati and the University of Iowa, and resident performance was scored using the OCAT. RESULTS: A consensus of faculty comments established the face and content validity of the OCAT. One hundred ninety-one on-call consultations were assessed. Timeliness of consultation was the most common category receiving a borderline or unsatisfactory rating. Borderline ratings in knowledge-based categories (history, examination, assessment and plan, urgency rating) occurred more often for postgraduate year 2 (PGY2) residents than for PGY3 residents (P = 0.05, chi-square test). Incomplete differential diagnosis (n = 6) and lack of follow-up instruction (n = 5) were the most common deficiencies observed. CONCLUSIONS: The OCAT has face, content, and discriminative validity. It can be used to assess resident competence in patient care, professionalism, and medical knowledge. Interrater and intrarater reliability still need to be determined. The OCAT may prove to be an additional assessment tool for meeting the Accreditation Council for Graduate Medical Education competencies mandate.

Clinical Competence↗

PubMed2001

Measuring quality of life of persons with spinal cord injury: substantive and structural validation.

The purpose of this study was to evaluate the substantive and structural validity of an existing measure of quality of life (QOL), the spinal cord injury (SCI) version of the Ferrans and Powers quality of life index (QLI). To evaluate substantive validity, 11 individuals with a SCI participated in 'think aloud' interviews to determine meaningfulness of the QLI items and to identify areas requiring modification. Free sort and ranking exercises of the items were used to evaluate the structural validity of the domains and scoring rubric. Content analysis of the interview comments resulted in the addition of two items and wording revision to three items. The free sort exercise revealed that the domains as perceived by the participants differed somewhat from those of the test developer. The contribution of the satisfaction and importance sections proposed by the scoring model was not completely supported by the data from the ranking exercise. It is concluded that the modified version of the QLI reflects the perspectives of persons with SCI spinal cord injury as represented by the participants of this study. The structural validity evaluation has implications for the use of domain subscores and weighted vs. section scores. Further evaluation of the modified version is necessary before widespread use with this patient population.

Adult↗

PubMed2005

The ophthalmic clinical evaluation exercise: reliability determination.

PURPOSE: Reliable and valid tools must be developed to assess the core residency competencies identified by the Accreditation Council for Graduate Medical Education. The Ophthalmic Clinical Evaluation Exercise (OCEX) is a tool designed to assess the ophthalmology resident's competence in patient care. The OCEX has been shown to have face and content validity. This study will determine the degree to which the OCEX is reliable and has construct validity. PARTICIPANTS: Ninety-four academic ophthalmology teaching faculty from ophthalmology residency programs across the country. METHODS: Participants reviewed a video compact disc of the same resident and new patient encounter and then completed the OCEX. A scoring rubric was provided. RESULTS: Results indicate that the OCEX is a reliable tool for faculty to use to assess residency competency. The coefficient alpha statistic (a measure of reliability/internal consistency) for the OCEX as a whole was 0.81. The alpha statistics for 3 of 4 subscales that comprise the OCEX (i.e., interviewing skills = 0.65, interpersonal skills/professionalism = 0.73, case presentation = 0.70) were lower than the OCEX as a whole, but were acceptable for new scales. However, the alpha for the examination subscale (i.e., 0.27) was extremely low. Interrater reliability assessment shows that of 33 individual OCEX items, 31 (94%) had at least 85% of the raters rating the student in 1 of 2 consecutive rating categories. CONCLUSIONS: The OCEX shows both reliability and validity and, therefore, meets the Accreditation Council for Graduate Medical Education criteria for an acceptable assessment tool.

Accreditation↗

PubMed1997

Measuring the effects of problem-based learning on the development of veterinary students' clinical expertise.

PURPOSE: To investigate whether repeated exposure to simulated clinical cases, as employed in problem-based learning (PBL), accelerates the development of clinical expertise in veterinary students. METHOD: In 1995 all 122 second-year students at the Texas A&M University College of Veterinary Medicine were organized into 24 self-selected groups to complete four computer-based case simulations in veterinary neuroanatomy. A scoring rubric of four categories (poor, neutral, good, and excellent) was used to assign quality classifications for physical examination questions selected by the students. Each group's diagnostic efficiency was calculated for each case on the basis of these classifications. An analysis of variance (ANOVA) using a one-factor repeated-measures procedure was employed to examine the cumulative effects of the case simulations on diagnostic efficiency. Post hoc procedures involved the use of contrasts to determine the trend of diagnostic efficiency with repeated use of case simulations. A Pearson product-moment correlation coefficient was calculated to determine the strength of the relationship between the change in diagnostic efficiency for each group and the selected group characteristics, total hands-on veterinary experience, and average grade-point average (GPA). RESULTS: Because six of the 24 groups did not perform a physical examination in one of the cases, a rating of zero was given when no physical examination was done; the data were then analyzed with and without the six groups included, and compared. The ANOVA yielded a significant result for all 24 groups (F(3,69) = 2.75, p = .0491) and for the 18 groups that performed physical examinations in all four cases (F(3,51) = 3.03, p = .0377). Significant linear contrasts were also found for all 24 groups (F(1,69) = 7.21, p = .009) and for the 18 groups (F(1,51) = 4.25, p = .044). Improvement in diagnostic efficiency could not be correlated with GPAs and was only somewhat correlated with the total amount of prior clinical experience reported by the students. CONCLUSION: Study findings suggest that there is a significant relationship between the repeated use of case simulations in PBL and the accelerated development of clinical expertise.

Adult↗

PubMed2006

Assessing the underlying structure of the United States Medical Licensing Examination Step 2 test of clinical skills using confirmatory factor analysis.

BACKGROUND: The purpose of the present study was to assess the fit of three factor analytic (FA) models with a representative set of United States Medical Licensing Examination (USMLE) Step 2 Clinical Skills (CS) cases and examinees based on substantive considerations. METHOD: Checklist, patient note, communication and interpersonal skills, as well as spoken English proficiency data were collected from 387 examinees on a set of four USMLE Step 2 CS cases. The fit of skills-based, case-based, and hybrid models was assessed. RESULTS: Findings show that a skills-based model best accounted for performance on the set of four CS cases. CONCLUSION: Results of this study provide evidence to support the structural aspect of validity. The proficiency set used by examinees when performing on the Step 2 CS cases is consistent with the scoring rubric employed and the blueprint used in form assembly. These findings will be discussed in light of past research in this area.

Clinical Competence↗

PubMedDate not supplied

Sources of error in constructing and interpreting graphs: a study of fourth- and fifth-grade students with LD.

The purpose of this study was to examine students' understanding of and ability to interpret line graphs, using four tasks: two where students constructed a graph and two where students interpreted a given graph. The tasks were unique in that they did not simply involve the explanation of given data but included a language component that looked at students' cognitive processes and understanding while they were engaged in the tasks. The study is important because students are often able to define and even draw graphs yet perform poorly on graph interpretation items presented in high-stakes examinations. The sample for the study consisted of 91 fourth and fifth graders with and without learning disabilities (LD) from six schools in a major metropolitan area. Scoring rubrics were designed for each item. An analysis of variance by grade and group indicated no significant differences between fourth and fifth graders overall but revealed that typically achieving students performed at significantly higher levels than students with LD at both grade levels. Additional descriptive analyses looked at the patterns of errors evidenced and the nature of the students' constructions. These are discussed, and instructional implications are derived.

Achievement↗

PubMed2026

Visualizing the future: impact of holographic visualization on teaching minimally invasive veneer preparation: a randomized controlled trial.

BACKGROUND: Minimally invasive veneer preparation in prosthodontic education requires advanced spatial understanding and psychomotor precision. Although immersive technologies are increasingly used in dental education, evidence regarding the educational effectiveness of holographic visualization in prosthodontics remains limited. This randomized controlled trial evaluated the impact of holographic visualization on preparation quality and student-reported educational outcomes during preclinical training in minimally invasive veneer preparation. METHODS: Sixty undergraduate dental students were randomly assigned to receive either conventional two-dimensional photographic visualization (n = 30) or holographic three-dimensional visualization (n = 30). Following a standardized calibration session and baseline preparation, participants performed a second veneer preparation after exposure to the assigned visualization modality. Preparations were independently assessed by two blinded evaluators using a predefined scoring rubric. Within-group changes were analysed using Wilcoxon signed-rank tests, and between-group differences were assessed using analysis of covariance adjusted for baseline performance. Student-reported outcomes were compared using Mann-Whitney U tests with false discovery rate correction. RESULTS: Both groups showed significant improvements in preparation quality from baseline to post-intervention (both p < 0.001). However, no significant between-group differences were observed (adjusted mean difference: 0.60 points; 95% confidence interval: -0.48 to 1.67; p = 0.269). Overall student-reported outcomes were comparable between groups. Spatial understanding showed the largest between-group effect in favour of holographic visualization, although this difference was not significant after correction for multiple comparisons. Holographic visualization was highly accepted by students, and 70% supported its future integration into dental education. CONCLUSIONS: Although holographic visualization did not significantly improve immediate preparation quality compared with conventional two-dimensional photographs, it was highly accepted by students and showed the largest observed effect for perceived spatial understanding. These findings suggest that holographic visualization may primarily support spatial understanding and represent a valuable adjunct for teaching complex three-dimensional concepts in preclinical prosthodontic education. TRIAL REGISTRATION: German Clinical Trials Register (DRKS), DRKS00040713, prospectively registered on 22 June 2026.

Humans↗

PubMed2006

Evaluating critical thinking in clinical concept maps: a pilot study.

Today, the complexities in the health care system are challenging nurses to be skillful and knowledgeable critical thinkers and decision makers. To adequately prepare future nurses to meet the challenges, nurse educators must nurture and facilitate critical thinking. One strategy believed to promote critical thinking in nursing education is concept maps. The purpose of this pilot study was to determine the level of critical thinking in the clinical concept maps developed by second year baccalaureate nursing students. Students enrolled in a five-week clinical practicum course were asked to submit their final concept map and participate in a focus group. The data for the study included eighteen concept maps, 1 student focus group and 1 instructor focus group. The Holistic Critical Thinking Scoring Rubric (Facione & Facione, 1994) was used to measure levels of critical thinking, and content analysis was used to analyze focus group data. Results from this study indicated that developing concept maps in the clinical setting fostered critical thinking and improved clinical preparedness.

Concept Formation↗

PubMedDate not supplied

Teaching and assessing surgical competency in ophthalmology training programs.

BACKGROUND AND OBJECTIVE: The Accreditation Council for Graduate Medical Education (ACGME) has mandated implementation of six new competencies in resident training in the United States. An implementation strategy is proposed to teach and assess cataract surgical competence. PATIENTS AND METHODS: An intradepartmental Task Force for the ACGME competencies reviewed the literature for assessment tools to develop an implementation matrix for assessing surgical competence. RESULTS: "Good practices" (gleaned from the literature) were adapted for the institution's needs and tested, including (1) written and explicit goals or objectives for each stage of training; (2) substitution of a criterion-referenced (Dreyfus model) scoring rubric for a norm-referenced, peer-benchmarked global evaluation; (3) use of formative rather than summative feedback; (4) incorporation of deliberate practice (Ericsson model); and (5) portfolio-based documentation of sentinel event markers and remediation. CONCLUSION: An implementation matrix for teaching and assessing surgical competence might be useful for local compliance with the ACGME mandate.

Cataract Extraction↗

PubMed2005

Writing-skills development in the health professions.

BACKGROUND: Studies have found that students in the medical professions often lack the writing skills required during their education and career. One contributing factor to this deficiency is that writing tends to be discipline specific, rather than requiring general skills acquired in undergraduate schools. PURPOSE: To determine the extent to which a rigorous writing exercise impacted the quality of students' medical writing based on a specified rubric. METHOD: In the context of a basic science course, we developed 6 weekly writing exercises called Question of the Week, along with a rubric for scoring students' work. The rubric evaluated 6 specific aspects of students' writing including Comprehensiveness/Thoroughness, Accuracy, Conciseness, Logical Organization, Justification of Assertions, and Use of Appropriate Terminology. RESULTS: Except for Justification of Assertions and Accuracy, which did not change, scores for all categories improved between Weeks 1 and 2. Use of Appropriate Terminology was the only category for which scores increased after Week 2. CONCLUSION: The clearest indication of writing development came from students' augmented ability to use medical terminology in appropriate ways. This is an important observation, given that each Question of the Week covered a separate body system, characterized by distinctly different terms and jargon. We concluded that students need much more practice to attain the level of proficiency outlined by our rubric.

Educational Measurement↗