Program evaluation: as the program director sees it; as the evaluator sees it.
Explore the source record for details and available documents.
SEARCH · PubMed Health
Explore indexed PubMed citations for clinical trials, systematic reviews and public health research. Read source abstracts and follow each citation to its original PubMed record.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
Program evaluation in preparation for accreditation provides opportunities for the identification of program strengths, weaknesses, and improvement potential. In the present study, naturalistic inquiry, an evaluation methodology responsive to the concerns of persons with a stake in the program, was successfully used in preparation for the accreditation of an occupational therapy program. The experiences of undergraduate students who implemented the program evaluation as part of an honors elective course describe the process, outcomes, benefits, and problems associated with the use of naturalistic inquiry as a responsive program evaluation.
BACKGROUND: According to 2003 Behavioral Risk Factor Surveillance System data, South Carolina has the fourth highest rate of overall diabetes among the 50 states (9.3%) but the second highest rate among African Americans (15.5%). Nationwide, African Americans are disproportionately affected by diabetes. In addition, 40% of the African American population in South Carolina lives in a rural area, and approximately 26% live at or below the poverty level. Lack of access to health care and diabetes education are additional barriers for people with diabetes and their families. CONTEXT: Since 1997, the South Carolina Diabetes Prevention and Control Program and the Diabetes Today Advisory Council have sponsored the African American Conference on Diabetes, which targets African Americans with diabetes, their families, and their caregivers. This article describes the evolution of the conference and its evaluation. METHODS: In 2002, we conducted focus groups with 20 African American conference attendees with diabetes to 1) assess the program's effects, 2) determine how to reach more individuals, and 3) improve programming. In 2004, we incorporated the preconference and postconference Diabetes Understanding Scale survey to assess the cognitive impact of the conference on participants. CONSEQUENCES: Focus group results revealed that participants wanted to attend the conference because of the opportunity to increase their knowledge and change their behaviors through 1) education, 2) social support, 3) resources, and 4) logistics. Self-rated understanding increased significantly after the conference for each cognitive understanding item on the Diabetes Understanding Scale. INTERPRETATION: Focus group results suggested that participants who continue to attend the conference year after year may improve diabetes self-management skills. A quantitative evaluation showed that this 1-day diabetes education conference significantly increased short-term, self-rated cognitive understanding of diabetes behaviors.
This paper focuses on evaluations of programs to prevent child abuse and neglect. It discusses the quality of the evaluations and offers suggestions for improvement to evaluators, program planners, and funders. Thirteen evaluations recommended by experts and published from 1978 through 1988 were reviewed. Using the true experiment as the standard of quality, we found evaluations frequently characterized by careful attention to methodologic detail. Evaluators describe eligibility criteria; half of the studies had control groups or followed progress. However, to learn more about prevention of abuse and neglect the caliber of evaluation research must improve. Evaluators do not define abuse or neglect; they accept reports of abuse. This has the effect of shifting the responsibility to others whose concerns and reporting standards differ. Lack of definition is reflected in a paucity of valid measurements; we could not assess the characteristics of families who benefit most from programs. Evaluators also omit important topics. Although seven evaluations studied health-related variables, no comprehensive analysis of the consequences and costs of medical neglect could be found; very little information on costs and benefits of any kind was available.
Effective program evaluation is a systematic way to improve and account for public health actions by involving procedures that are useful, feasible, ethical, and accurate. The framework guides public health professionals in their use of program evaluation. It is a practical, nonprescriptive tool, designed to summarize and organize essential elements of program evaluation. The framework comprises steps in program evaluation practice and standards for effective program evaluation. Adhering to the steps and standards of this framework will allow an understanding of each program's context and will improve how program evaluations are conceived and conducted. Furthermore, the framework encourages an approach to evaluation that is integrated with routine program operations. The emphasis is on practical, ongoing evaluation strategies that involve all program stakeholders, not just evaluation experts. Understanding and applying the elements of this framework can be a driving force for planning effective public health strategies, improving existing programs, and demonstrating the results of resource investments.
Program evaluations have become an integral part of public health practice. To illustrate the program evaluation process, an evaluation of clinical outcomes for clients of the Family Case Management (FCM) program administered by the nursing division of the DuPage County Health Department, Wheaton, Illinois, is presented. Birth outcomes for 1996 clients of the program who had a previous low-birthweight (LBW) infant and a subsequent birth using the prenatal services of the FCM program were analyzed. A purposive sample of clients was selected from the FCM prenatal program. Birth outcomes including infant gestational age and birthweight, maternal weight gain, and other key variables were compared to provision of health department services. Descriptive statistical methods were used to explore the relationship between birth outcomes and prenatal care between the first pregnancy and the second pregnancy. Analysis of the data showed improvements in maternal weight gain, infant birthweight and gestational age, prenatal care access, public health nurse (PHN) contacts, and physician visits. A positive association was shown between the early entry and increased use of prenatal services to improved birth outcomes in the subsequent pregnancy.
1. A program evaluation was conducted to evaluate the effectiveness of the smoking cessation program. Substantiating data enhances the value of health promotion programs as key components of the health care delivery system. 2. Persons enrolled in the 8 week smoking cessation program and members of a comparison group were administered an evaluation tool prior to participating in the program, with a second evaluation tool upon completion of the program. Afterward, the smoking status of the members of both groups was evaluated at 3 month intervals for a period of 1 year. 3. Of the participants, 26.7% were not smoking 12 months after the program, compared to 6.9% of the comparison group. Of those participants who resumed smoking, 64% smoked less than half the amount they smoked before taking the class.
This study investigates alternate reasons for conducting program evaluation in community health care settings and their relationship with information needs. One hundred and thirty-six community health nurses representing New York state associations were the decision makers used in the survey. Results indicated that the most frequent reasons cited for conducting program evaluation were accreditation and standard control. These were followed by need for information for formative program changes and aiding decision makers. Participatory and scientific evaluation purposes were rated last. When examining information sources, respondents were unable to differentiate needs on an independent item-by-item basis; all items were considered important. When asked, however, to select the most critical items of these sources, a pattern of needs was established. Decision makers felt that program goals, identification of staff concerns, and presentations of client opinions were the most crucial types of information.
Although program evaluations promote organizational hygiene by early detection of problems, many faculty experience evaluation phobia. To overcome this phenomenon, the author discusses four major strategies for faculty and administrators.
Accountability of educational institutions is a current trend in academia. Outcomes assessment provides a method of evaluating educational programs to ensure they are meeting their objectives. This paper presents an overview of outcomes assessment and describes its relationship to and integration with program evaluation. It also describes the implementation of an outcomes assessment program at Kean College of New Jersey, Union, New Jersey. A model for assessment within occupational therapy programs, which parallels a model for intervention planning, is introduced and discussed.
Results of "Curricular Evaluation Program" (CEP) of Medical School of S. Paulo University (FMUSP) are presented. CEP numbers are obtained form students answers offered to a padronized questionnaire which included theoretical and practical aspects of curricular process. Authors asked for the opinion of curricular disciplines coordinators of FMUSP about CEP and concluded: 1. CEP has large or reasonable profitability for 95.3% of coordinators; 2. CEP questionnaire was considered adequate by 85.7% of respondents, but 42.8% mentioned deficiencies in questionnaire revitance to them; 3. CEP presents excellent performance at FMUSP departments level.
Program evaluation--the use of scientific techniques to measure the value of an agency's work--has become the focus of major attention and enthusiasm in mental health service planning and administration. However, the author points out that completed program evaluation studies are few in number and provide no evidence of great impact on the operation of programs or agencies and that unrealistic enthusiasm for program evaluation can lead to unrealistic rejection when performance fails to reach promise. He concludes that program evaluation should be applied only when it can have an impact, i.e., in a limited number of carefully selected cases.
Community-based programs are being widely adopted in the struggle to prevent chronic disease. Program evaluation of community-based programs involves a particular set of problems stemming from the variety of activities being undertaken simultaneously, the multiple intermediate goals of the programs and the rapidity with which the programs evolve. An analysis of the experience of four large community-based cardiovascular disease research and demonstration studies (Stanford Five-City Project, Minnesota Heart Health Program, Pawtucket Heart Health Program and the German Cardiovascular Prevention Project) provides valuable models, methodologies and strategies for planning and conducting evaluations of public health programs or community studies. By comparing and combining their experiences, the four programs have identified eight categories of evaluation for community studies, including formative evaluation, quality assurance, assessment of delivered dose, assessment of received dose, component program impact, intermediate outcomes, community impact and cost analysis. This paper presents information on the strategies by which each of the four programs addressed these evaluation categories.
Many approaches can be used to develop understanding of service programs. These approaches vary in how much they rely on technical vs. political procedures, who controls them, purpose, scope of content, and generality of results. Only some of these approaches are usually considered program evaluation. This diversity is illustrated by a description of the history of 10 program evaluation-like approaches to understanding as they operated during the 18-year life of the federal Community Mental Health Centers (CMHC) Program. When services were funded in federal categorical programs, information-gathering approaches focused narrowly on the program and its activities. It now appears that more widely useful information would result by shifting from the evaluation of "programs" to comparative analysis and evaluation of service systems.
Survivorship analysis is a statistical technique used here in a summative evaluation that compares outcome data from multiple-site methadone maintenance programs in three California counties. Multiple outcome measures were obtained from retrospective interviews with 297 heroin addicts 6-8 years after admission to treatment. These measures were rates of retention, incarceration, addiction, crime, dealing, and loss of employment. Survival curves for all six measures are consistent in showing that Los Angeles County had the best outcomes, followed by San Bernardino County and then Orange County.
While program evaluation is necessary for understanding and improving educational programs in nursing, it is frequently surrounded by a climate of unease and perplexity. The first article (September/October 1990) in this two-part series reviewed models of program evaluation. This second article examines the state-of-the-art of program evaluation in nursing education and proposes implementation of a responsive model of evaluation in schools of nursing. The series aims to foster more dynamic, efficient program analysis, thereby enhancing the effectiveness of nursing education programs.
The Steps to a HealthierUS Cooperative Agreement Program (Steps Program) enables funded communities to implement chronic disease prevention and health promotion efforts to reduce the burden of diabetes, obesity, asthma, and related risk factors. At both the national and community levels, investment in surveillance and program evaluation is substantial. Public health practitioners engaged in program evaluation planning often identify desired outcomes, related indicators, and data collection methods but may pay only limited attention to an overarching vision for program evaluation among participating sites. We developed a set of foundational elements to provide a vision of program evaluation that informs the technical decisions made throughout the evaluation process. Given the diversity of activities across the Steps Program and the need for coordination between national- and community-level evaluation efforts, our recommendations to guide program evaluation practice are explicit yet leave room for site-specific context and needs. Staff across the Steps Program must consider these foundational elements to prepare a formal plan for program evaluation. Attention to each element moves the Steps Program closer to well-designed and complementary plans for program evaluation at the national, state, and community levels.