Section 3: Evaluation Design

This section provides steps to develop evaluation questions in order to specify your evaluation design. It is important to work closely with your evaluation partners during the design phase to ensure there is buy-in for the approach that you choose. As you develop your evaluation questions, you should refer to your evaluation purpose statement (see Section 1).


Step 1: Develop evaluation questions aligned with the evaluation purpose statement

With your evaluation partners convened:

A. Brainstorm a list of evaluation questions aligned with your partnership's evaluation purpose statement.

Several approaches can be used to structure a brainstorming session with your partners, including more formal (nominal group technique, Delphi method) or informal methods.

In the idea generation phase, encourage your partners to consider the following categories of evaluation topics:

  • Partnership strengths, weaknesses, opportunities, and threats;
  • Organizing and advocacy activities;
  • Changes in policies, practices, and environments affecting pedestrian safety;
  • Educational and promotional efforts increasing pedestrian safety knowledge and skills in the population of interest; and
  • Health behaviors, health outcomes, and related social, economic, and environmental outcomes.

You can also refer partners to the following resources generated in Sections I and II to stimulate ideas, including:


Tip Icon

Tip for Success

Make sure partner concerns are included to increase applicability of the evaluation results to pedestrian safety improvements.

Example Icon

Examples from the Field

Sample Evaluation Questions1
Sample process and outcome evaluation questions for PSAPs or education and promotion evaluations.


B. Prioritize the list of evaluation questions using criteria agreed upon by partners.

To prioritize your most important evaluation questions, you and your partners should consider the following:

  • Are the evaluation questions important to all partners or subsets of partners?
  • Are the evaluation questions relevant to the audiences for the evaluation?
  • Are the evaluation questions aligned with the evaluation purpose statement and the intervention goals, objectives, and logic model?
  • Are the evaluation questions feasible to address within the current timeline, funding, and resources?
  • Will the evaluation questions inform either actionable improvements to the partnership or the intervention or evidence-based practices for the field?

As partners go through the prioritization process, be sure to track and explain the reasons why evaluation questions may be combined, refined, or discarded in relationship to the evaluation criteria. It is critical to share this rationale with partners to ensure all individuals feel their input and ongoing participation is needed and valued, particularly for those with little to no evaluation experience.

Review your final list of evaluation questions with all partners. Check that each question is consistent with the SMART guidelines described in Section 2 and that the partners responsible are assigned.

Templates Icon

Template

Worksheet to Prioritize Evaluation Questions template2
A template for rating the relevance of each evaluation question and making decisions about including vs. discarding questions based on specific criteria.

Footnotes

  1. Adapted from: HM Treasury 2011. The Magenta Book: Guidance for evaluation.
  2. Adapted from: Salabarría-Peña, Y, Apt, B.S., Walsh, C.M. Practical Use of Program Evaluation among Sexually Transmitted Disease (STD) Programs, Atlanta (GA): Centers for Disease Control and Prevention; 2007.

Next: Continue to Step 2
child at crosswalk man and child on sidewalk