Step 1: Design and Run Evaluations

In this course, we will share our perspective on why evaluation is an important tool for cities, show you how to become an advocate for evaluation, and explain what it takes to lead evaluation projects. NOTE: This BIT-led course is hosted through GovEx Academy. It's free to register.

The Data Equity Framework from We All Count is a systematic way of looking at data projects. It provides a seven-step framework to “identify all of the many equity-impacting decisions, decide how to make them equitably, and then explain them in a way that makes sense.”

The Equitable Evaluation Framework invites evaluation practitioners to consider beliefs, assumptions and preconceptions, and constantly recalibrate as they engage in their work. It encourages readers to consider culture, context and power in their evaluative work.

Use this Workbook as a helpful guide for each stage of your evaluation journey. It provides clear step-by-step instructions on how to design and implement your evaluation. You can request login credentials by emailing the Behavioural Insights Team (wwc.evaluation@bi.team)

This BIT resource helps evaluation practitioners lead project scoping conversations with partners. It includes a suggested agenda with sample questions for each of the four objectives in a scoping conversation: (1) Build excitement and interest in evaluation (2) Identify potential evaluation opportunities (3) Confirm department interest and capacity to participate in an evaluation project (4) Begin discussing feasibility considerations.

This worksheet is designed to help you identify evaluation projects. It is best suited for gathering more in depth information after an initial scoping conversation.

This worksheet is designed to help you determine if a process evaluation is a good fit for evaluating a particular initiative.

This worksheet is designed to help you determine if a randomized controlled trial is a good fit for evaluating a particular initiative.

This BIT guide is designed to help you identify internal partners and engage with them on evaluation partnership opportunities.

Use this guide to learn about ethical human subjects research. It includes a worksheet so you can work through the ethics of your evaluation.

This report identifies basic ethical principles and guidelines that address ethical issues arising from the conduct of research with human subjects.

This tool calculates statistical power for RCTs. It can help you determine how large of a sample you need for a trial or help you understand what effect size you may be able to detect with a given sample size.

This page provides an inventory of experimental approaches you may consider using in your evaluation.

This guide introduces the principles of "logic modeling" in order to enhance program planning, implementation, and dissemination activities.

This resource summarizes existing evidence on the objectives, functions and methods of implementation and process evaluation (IPE) in relation to educational interventions.

Use this Workbook as a helpful guide for each stage of your evaluation journey. It provides clear step-by-step instructions on how to design and implement your evaluation. You can request login credentials by emailing the Behavioural Insights Team (wwc.evaluation@bi.team)

This guide outlines the steps to conduct randomization for an RCT in Microsoft Excel.

This checklist outlines things you should check before launching a letter, email, or SMS RCT. Much of the information can also be used in other RCTs.

This page helps identify when and how to obtain informed consent, if your evaluation requires it.

This guide discusses their approach to designing qualitative research.

This primer is a high level overview for running interviews, focus groups, and observations as a part of a process evaluation.

This framework can be used to log and organize responses from your qualitative research activities.

This page outlines the key steps involved in data processing and cleaning. It describes best practices in file and code organization, and lists steps to make data ready for use. This resource focuses on cleaning data with Stata code, but similar principles apply to R and other data analysis tools.

This resource can help generate results if you’re running a basic analysis such as t-test or a linear regression without covariates. For more complicated analysis, we recommend using statistical software like R or STATA.

This guide outlines the steps for data cleaning & analysis in Microsoft Excel.

This spreadsheet will create a graph for simple RCT results.

Use this Workbook as a helpful guide for each stage of your evaluation journey. It provides clear step-by-step instructions on how to design and implement your evaluation. You can request login credentials by emailing the Behavioural Insights Team (wwc.evaluation@bi.team)

This guide helps you interpret your data and prepare your report back to partners.

Use this template to help you draft a report for a process evaluation.

Use this template to help you draft a report for a simple RCT.

This resource helps you interpret and understand null results.

This resource helps you make scaling decisions based on the results of your evaluation.