Assignment #2 - Which Method is Best?

            Program evaluation has many merits, but choosing the appropriate method may not be clear initially. The Prenatal Exercise Program, an initiative meant to promote physical activity among Aboriginal women during their childbearing years, also strived to achieve the goal of decreasing the rate of gestational diabetes mellitus (GDM), which is quite high within this demographic. The most effective method to evaluate this program is Michael Scriven’s methods. Scriven approaches evaluation using either formative or summative means; in this case, summative evaluation would be appropriate. Summative evaluation is used at the end of a program to prove its effectiveness; the information provided under Program Participants and Description indicate that the program does have success, as many of the obstacles that could have challenged the women were eliminated. Because of the success, formative evaluation is not necessary, as changes are not necessary. What needs to be determined is the effectiveness of the program – is it reaching its goal of promoting regular activity? More importantly, is the long-term goal of reducing type 2 diabetes being achieved? The study took place over two years, which appears to be enough time to determine whether women regularly attended for the duration, and if they continued after the study. Because the women were given educational materials at the sessions, it would interesting to note whether they had an impact as well.  While the program appears to be positive, the worth and value needs to be evaluated, especially when there is funding involved, and so much support from the community. Interviews, surveys, and details about attendance would provide a wealth of information for all people involved. A summative evaluation would indicate whether the focus needs to change, and overall, if there is benefit to the program.

Assignment #1 - Evaluation of an Evaluation

            Various programs are implemented on a regular basis; some flourish, some fail. Evaluation of a program before failure could be essential to saving what was intended to be a successful, and often, necessary program. One method of evaluating a program is the Discrepancy Evaluation Model (DEM), created by Malcolm Provus. This model follows a specific format and is used to determine whether there is a difference between what the standard is and what has actually happened. One report, “Women’s Program Evaluation Report and Management Response” was completed in 2006, and followed the steps of the Discrepancy Model. The report focuses on The Women’s Program, which is a grant program intended “to support action by women’s organizations and other partners seeking to advance equality for women by addressing women’s economic, social, political and legal situation” (“Status of Women” 2006). The report is divided into four major sections: design and delivery of the program, impact of the program, cost-effectiveness/alternatives, and relevance.  The report has numerous strengths, many of which stem from its organization. Context and methodology is initially addressed, which provides the data sources used for the report; they include a review of program documents and administrative data, a review of program files, a survey of applicants, interviews, and case studies (“Status of Women” 2006). There are fourteen recommendations made throughout the report, the first being under the design and delivery section. The paper indicates that there are several obstacles to successful implementation, based mainly on the fact that the program uses an outcome-based approach. The recommendation that there be more accountability fits under Provus’ model; one weakness of this section is the lack of information about the outcome-based approach. It is not clear what the outcomes truly are, therefore the recommendation for greater accountability for it is not clear. Many of the recommendations address the need for transparency when it comes to performance; it is apparent that the report does intend to make improvements to the program by determining the discrepancies between what is supposed to occur and what is happening. Many of discrepancies appear to originate from the fact that the Program Procedures manual requires clarity. The recommendations call for more management follow-up to improve accountability.

            In regards to the impact of the program, recommendations appear quite basic initially; one recommendation is for greater collaboration, however, the report does not make it explicitly clear how this should happen. The latter section of the report deals with cost-effectiveness and alternatives, which again, follows Provus’ model. Interviews revealed that the Women’s Program had value in terms of cost, mainly due to the reliance on volunteer support; the cost related recommendations focus on reporting on administration costs, as well as feedback for those who apply for funding but do not receive it. The report also indicates that most of the recommendations have been accepted, and follows with an explanation about how it has or will meet the goals; completion dates are indicated, or state that they are ongoing. Overall, the report is effective, as it clearly outlines the objectives of the program, the recommendations, and whether they are accepted or not.   

References

Status of Women Canada. (2006). Retrieved 01/14, 2013, from http://www.swc-cfc.gc.ca/account-resp/pr/wpeval-evalpf/wpem-egpf-eng.pdf