Evaluation Types and Methods

Evaluation typically is the last consideration in planning and implementing an intervention. However it is critical that evaluation be planned from the start so necessary data or information is identified and collected.

evaluationBefore beginning the intervention, baseline data or information must be collected for the intended outcomes. The intended outcomes of your intervention were identified when preparing the intervention goals, objectives, outcomes and a logic model (see Readiness and Preparation). Information and data that were collected when conducting your needs assessment (see Assessment) may provide some or all of the baseline measures needed. Baseline measures are essential for determining if changes occurred after implementing the intervention. For example, if an intended short-term outcome is to increase access to tobacco cessation services, a baseline of the current available cessation services must be established prior to beginning the intervention so you will know if services changed after implementing the intervention. 

Evaluation must be conducted before, during and after the intervention. All three are necessary to be able to make judgments about the intervention’s effectiveness.

Evaluation conducted before an intervention is implemented is called formative. Formative evaluation is used to develop and refine the intervention content before implementing it fully with the priority population. Examples of formative evaluation and methods include:

  • Conducting focus groups with representatives of the priority population to pre-test intervention messages and materials to evaluate participant understanding and acceptance.
  • Observing the intervention protocol or procedures through pilot testing with representatives of the priority population to determine if changes are needed.
  • Collecting feedback from pre-test and pilot test participants to evaluate personnel performance and determine if additional training is needed.

Process evaluation is used to gather information during the implementation of the intervention. It measures to what degree the intervention was implemented according to the plan and how the intervention was received by the participants. These measures are typically the outputs of the activities of your Logic Model. (refer to your Logic Model prepared in Readiness and Preparation).

Process evaluation provides answers to evaluation questions such as:

  • What intervention activities were conducted? Were they conducted as planned? If not, what changes were made?
  • Who conducted the activities? What training did they receive prior to conducting the activities? Is more training needed?
  • Who and how many participated in the activities? Were they the priority population?
  • How often were participants exposed to the intervention activities?
  • What resources were used to conduct the activities? Were they sufficient?
  • How well did participants respond to the activities?

The following are example methods for capturing process evaluation information:

  • Participant attendance logs  
  • Intervention activity logs – document what, when, where and how many activities were conducted, and the resources used (including staff)
  • Intervention implementation diaries – completed by intervention implementers to identify differences in planned intervention activities and those actually conducted
  • Participant feedback forms – to identify what participants liked and disliked, what they learned, etc.

Impact evaluation may be conducted toward the end of the intervention to determine if short-term and intermediate outcomes are being achieved. This may include measuring changes in the priority population’s knowledge, attitudes or beliefs, or other factors that may lead to the desired change (refer to your Logic Model prepared in Readiness and Preparation).

As an example, one short-term outcome for increasing physical activity among your priority population may be to increase their belief about the importance of being physically active. Impact evaluation can also be useful in examining social and environmental outcomes of an intervention. For example, an increase in perceived safety in walking in one’s neighborhood may be measured against actual data on reported assaults in the community. Examples of impact evaluation methods include:

  • Collecting data on short-term and intermediate outcomes through pre- and post intervention surveys, focus groups, interviews, etc.
  • Evaluating social and environmental changes from baseline through observation, reports, and policy examination.

Outcome or summative evaluation is conducted at the conclusion of intervention activities. It allows you to determine if the intervention’s overall goals, objectives and long-term outcomes were achieved, and if the intervention should continue. Summative evaluation identifies the changes in baseline measures of long-term behavioral outcomes, such as decreased smoking and increased physical activity, as well as changes in prevalence of chronic diseases. A summative evaluation may also take into account the resources required to produce the outcomes.

Examples of summative evaluation methods include:

  • Collecting and comparing baseline and end-of-intervention outcome data
  • Conducting an analysis of the cost to produce outcomes

Select a section:

printer-friendly Printer-friendly version