Plan your evaluation methods and measures

  • Pre-test your intervention strategies

Use focus groups or individual interviews with community members to ensure the strategies are appropriate for the intended audience. When testing the strategies, consider how well they are understood as intended, whether the information is clear, whether the information is perceived as useful, whether the participant perceives the activities to be supportive, and how well the information or activities are recalled or remembered.

  • Consider your evaluation strategy

In order to determine if your supportive relationships intervention is working, you will need to evaluate your efforts. It is important to design your evaluation in the planning phase of your intervention because you will need to be able to measure change in order to measure the impact of your intervention strategies. To measure change, you will need to have an idea of what is happening right now.

As with all interventions, it is useful to consider process, impact and outcome evaluation. Process evaluation enables you to assess if your program is being implemented as intended. You might consider collecting registration and attendance forms to determine who is, and is not, taking part in your program. This information can also provide information on how frequently individuals are participating. It is also useful to collect information on how satisfied individuals are with the various program activities and materials. With community support interventions, it is also useful to assess the process used to develop and plan the various activities. This may include an assessment of the coalition processes (e.g., decision making, conflict management) and well as specific logistics (e.g., time of meeting, adequate day care, location of meeting).

In order to assess the impact of your activities, it is important to develop intermediate outcomes (e.g., changes in attitudes; perception of support for changing particular behaviors such as readiness to use cessation products; readiness to make behavior changes) as well as long-term outcomes you will assess.  It is important to develop these intended outcomes and related evaluation questions with input from all partners including funding agencies.

You might consider collecting information through the use of standardized surveys either face-to-face conducted at the program site, in individuals’ homes, or over the phone. These surveys may collect information on changes in the levels of the various types of social support thought to impact changes in healthy behaviors (informational, tangible, or emotional/appraisal) as well as changes to tobacco use behaviors and the multiple ways these changes can be made (e.g., substitution, changes in social behavior). In addition, surveys should include items to assess exposure to interventions, utilization of materials, and It might also be useful to consider alternative ways of tracking behavior, for example, biological makers (e.g., intake of certain chemicals), and non-obtrusive measures (e.g., journals and diaries). The choices you make about which ways to track behavior will depend on a number of factors including: resources, time, personnel available, and the appropriateness of the measure for the setting.

Remember to focus evaluation on the objectives of the intervention. If the objective was to increase tangible support, it is important to assess tangible support. It may be useful to assess if these changes in support influenced readiness to change lifestyle behaviors.

  • Challenges to evaluating supportive relationship interventions

There are several challenges in evaluating supportive relationships interventions that should be considered:

  • It is often difficult to establish causality. Some individuals might have changed their tobacco use behaviors on their own or through relationships with others that were not part of the intervention. Therefore, it is important to get as much information as possible about the reason for changes made.
  • It is also difficult to attribute a change in tobacco use behaviors to one particular intervention (e.g., buddy system versus contact with a health care provider).
  • When the supportive relationships strategy is used along with other strategies (the most effective way to create change), it is difficult to figure out which intervention strategies led to the changes that were observed in the evaluation.

printer-friendly Print this window