Preparation

Plan your evaluation methods and measures

  • Pre-test your intervention strategies

Use focus groups or individual interviews with health care providers to ensure that the different components of the curriculum are appropriate for the intended audience. When testing these components, consider how well the information and activities are understood as intended, whether the information can be applied to different settings, whether the information is perceived as useful and how well the information is recalled or remembered. It can be very difficult to evaluate provider education interventions as they are usually part of a larger project with other intervention strategies. Therefore, changes in behavior, knowledge, attitudes, or other individual factors may be a result of one or a combination of strategies.

  • Consider your evaluation strategy

In order to determine if your provider reminder and education is working, you will need to evaluate your efforts. It is important to design your evaluation in the planning phase of your intervention because you will need to be able to measure change in order to measure the impact of your intervention. To measure change, you will need to have an idea of what is happening right now. 

As with all interventions, it is useful to consider process, impact and outcome evaluation. Process evaluation enables you to assess if your program is being implemented as intended. Provider education may include documentation of attendance at the training sessions, ability to carry out assignments or activities, recall or memory of what information was provided to the provider, how much time was spent on different training components, and the health care provider’s satisfaction with the information exchange. With provider education interventions, it may also be useful to assess the process used to develop and plan the intervention activities. This may include an assessment of the coalition processes (e.g., decision making, conflict management) and well as specific logistics (e.g., time of meeting, location of meeting).

Impact evaluation enables you to determine if you are achieving your intermediate objectives. For provider education, you could evaluate the extent to which the provider changed the types and amount of information provided to their patients, individuals experienced any changes in their visits with the health care provider and their level of satisfaction with the visits, what information they were able to recall, changes in knowledge, changes in attitudes, actions the participant has taken as a result of information provided or changes in behavior. You could also access whether the training changed policies or practices related to the health care provider or their agency/organization. This can be done by through telephone surveys or alternately, some have met face-to-face with members of the target audience and conducted interviews or focus groups to determine how much of an effect the intervention has had on the target population.

Remember to focus evaluation on the objectives of the intervention. If the objective was to increase knowledge it is important to assess knowledge, if the intent was to decrease negative outcomes then it is important to assess these outcomes. Similarly, it may be useful to assess if the intervention influenced provider readiness to change their behavior related to providing information and resources to their patients. Evaluation may include face-to-face or telephone surveys or qualitative assessments.

printer-friendly Print this window