Visuals and handouts:
V:5-1 Definition of Evaluation
V:5-2 Evaluation: An On-going Process
V:5-3 Three Types of Evaluation
V:5-4 Process Evaluation
V:5-5 Outcome Evaluation
V:5-6 Impact Evaluation
HO:5-1 Index Cards
HO:5-2 Evaluation Questions for Case Study
HO:5-3 Workshop Evaluation Questionnaire
HO:5-4 Workshop Reaction Form
HO:5-5 Participant Feedback
Other materials needed:
easels, newsprint, magic markers, masking tape, index cards
Transition
Throughout this training, we have attempted to help you develop the
knowledge and skills needed to promote parenting education. The topic
of this module is resource development. A comprehensive parenting
program enlists the support of the entire community and taps into its pool
or resources through networking. In turn, this strategy empowers
a program to meet its goals and objectives for substance abuse parenting
education
Brainstorm the benefits of networking
Possible responses:
Record on newsprint. For example, one essential skill is the ability to assess what an individual coalition has to offer and what it needs.
3. Ask participants to share their networking strategies.
Transition: Networking is an on-going process which we have been doing throughout this training. It is a powerful way for a group to move from talking about collaboration to actually making it happen. Resources can easily be made known so that their issue can be maximized. Each of you in this community brings to this workshop a wealth of knowledge, ideas, and resources. The purpose of the next activity is to allow you the opportunity to network with the goal of obtaining the specific resources you need today to help your organization function at a higher level.
Networking Exercise (40 minutes)
1. Form new small groups of 8 - 10.
State that it is important to meet new people who you have not yet
worked with.
2. Distribute 6 index cards to each participant. Note: business cards can also be used.
Completion of cards (5-10 minutes)
Exchange of cards (10 minutes)
Auction of unmatched cards (20 minutes)
Keep the group focused by adhering to these times!
Materials needed: 6 index cards per participant, and 10 additional index cards for each small group. Remind participants that they have a participant listing of names, addresses, and phone numbers. They can easily make contact with anyone with whom they exchange cards.
HO:5-1 Hand out Index Cards
Participants write the following information the first three cards:
Back: Their name printed for legibility.
Participants write the following information on cards #4,5,6:
Sit in a circle.Note:
One person quickly reads one Need card.
Everyone who has a Share card which matches that need, quickly gives it to this individual.
If a Need card cannot be met, move to the next person who then reads one of his/her Need cards.
Continue to repeat this process until everyone has read all of their Need cards. Note: Need cards which were not matched with a resource in the small group setting will be read to the community later in this module.
Do not slow the process by problem solving or telling war stories. Simply try to match available resources with needs. It is important that this exchange of cards takes place as quickly as possible. Time is limited for this activity.
The small group facilitator should collect all unmatched Need cards and return them to the training coordinator. This individual should group similar Need cards together to facilitate the large group activity.
Large group activity:
1. Announce that unfilled Needs from the small groups will now be read to the community and matched, it possible, with an unused Share card. The workshop coordinator or his/her selected facilitator should read off the unmatched Need cards collected from the small groups. Be sure to announce similar needs at one time to facilitate even distribution of resources.
Ask for a show of hands:
How many people had all of their needs met?
How many people had two of their needs met?
How many people got at least one of their needs met?
2. Summarize: The networking which you have started during this
workshop is a valuable source of resources. Continue networking as
often as possible when you return home.
Note: Focus on participatory evaluation, design & development issues, decision-making concerns, and importance of using evaluation.
Link to focus on which segment of the population
- Readiness
- Resources
Evaluation is the process of systematically collecting and analyzing data and using them to improve the program. It is ongoing:
Drop and discuss V:5-2 Evaluation: An On-going Process
We sometimes use a circular model to describe how a good evaluation takes place. It shows that evaluation is an ongoing process that provides continued feedback into all stages of the program, and acts to continually improve & modify the program.
HO:5-2 Evaluation Questions for Case Study
Refer back to the case studies used in Module 4. Ask the participants to go back to their small groups and discuss the four questions on HO:5-2 Evaluation Questions for Case Study. Give the small groups 10 minutes to do this exercise.
Reconvene the groups and explain that they probably discussed several
different types of evaluation in their small groups: process, outcome,
and impact.
Drop and discuss V:5-3 Three Types of Evaluation
There are three different types of evaluation that we typically talk about. They are:
Drop and discuss V:5-4 Process Evaluation
Process Evaluation:
Process evaluation describes, measures, and assesses program activities and materials. It documents what was done, when, to whom, and to how many. It answers questions like: What did we do? When? Where? To how many? What did we do well? What do we need to improve?
For example, process evaluation counts the widgets. If we want to lower the substance abuse among teenagers, we know that parent education programs can do that: we plan to conduct six parent education classes of 12 sessions each, send a monthly newsletter to all parents, sponsor three billboards, conduct a symposium on parenting, and put ads in the weekly shopper and weekly radio shows for the year. Process evaluation assesses to make sure that we did all of those things we said in our plan that we would do.
In V:5-2 process evaluation takes throughout the phases, mostly in phase 5: it records all the details of the activities.
Note: Example: we conducted 5 parent seminars, and had a total of 100 people attending
Drop and discuss V:5-5 Outcome Evaluation
Outcome Evaluation:
Outcome evaluation measures and assesses the program achievements and describes the program's immediate effects. It answers questions like; Did the program change the knowledge level, the attitudes, or the behaviors of the participants? Did the program increase the awareness of the issue it was designed to raise in the participants? We sometimes call the factors that we measure "indicators" of the larger problem that we want to attack.
For example, in the above case example, if we want to lower the use of substance abuse among teenagers, we know that lack of parent monitoring is a risk factor, and that our parent education program gives parents skills in monitoring, so our outcome evaluation seeks to measure the change in the number of parents in our program who now monitor their children more closely after the parenting program.
In the graph V:5-2, outcome evaluation results show up in phase 6: measuring the indicators, sometimes called outcome measures.
Note: Example: of the total number of people attending the parent seminars, 55% self-reported that they have increased their monitoring of their children's behaviors, in a 6-month follow-up telephone interview after the seminars.
Drop and discuss V:5-6 Impact Evaluation
Impact Evaluation:
Impact evaluation looks beyond the immediate results to assess and measure longer term effects and impact on the larger community. It seeks to answer the question: Did the parenting program decrease the use of substance abuse among teenagers in our community?
Note: Example: Two years after the seminars, plus many more parent activities, the use of alcohol & other drugs was down by 10% in the school district.
For the purposes of this workshop, we are not going to go into outcome and impact evaluation. We want to share with you three different evaluation forms that you can use in the on-going evaluation process when you are working with parent groups. In the graph V:5-2 impact evaluation results show up in phase 7; measuring long-range outcomes. This measure often is done over a period of years.
We are going to look very briefly at five different evaluation tools, all examining process evaluation.
Note: Discuss the importance of many different strategies in a comprehensive program, over many years, that will show impact evaluation.
Note: CSAP has several monographs on evaluation that expand this subject in great detail, with many examples
The first is a tool to use when starting out and choosing a specific program to use. It can be used to examine key elements of a parenting program, as we discussed in Module 3.
The next three are tools to use after an activity or training.
The fifth is one we may use tomorrow when we talk more about TOTs.
These three evaluation forms are all in your handouts:
HO:5-3 Workshop Evaluation Questionnaire
HO:5-4 Workshop Reaction Form
HO:5-5 Participant Feedback
Note: These are all examples of process evaluation forms.
Pass out each handout and briefly discuss it.