Social Impact Evaluation Design: How to Measure and Map Your Impact
Nonprofit funders expect your nonprofit organization to be able to demonstrate its impact. While there are many ways to approach this, building out your organization’s theory of change and your program level logic models is an important step. We will explore what a theory of change and logic model is, why it is important and how it can help your organization measure the impact of the work you are doing.
Understanding the Nonprofit Logic Model
A logic model maps out what it takes to deliver a program, service or area of work within the nonprofit organization. It details the assumptions embedded within the program and its short-, medium- and long-term outcomes. It also visually maps the program building blocks including inputs, activities, outputs. With this mapped out, you then can design an evaluation process and system to capture data on whether the program is producing the expected outcomes.
The logic model breaks out in more detail one element of an organizational level theory of change. For example, a local watershed organization may focus their work in three areas – education, citizen science and policy work – to further its mission of contributing to a healthy watershed. Each work focus area mapped out in the theory of change would then have its own logic model.
What is a theory of change?
A theory of change shows visually and in writing how your organization’s work connects to and moves your mission and vision forward. A logic model maps out what it takes to deliver a specific program, the assumptions embedded within it and its short-, medium- and long-term outcomes. Taking the time to work together to create both will make apparent gaps in logic, understanding and agreement about what you are trying to achieve within your organization. It provides you with the opportunity to work through these to come to a shared understanding and vision which strengthens your work and organization.
4 Steps to Building Your Logic Model
After you have worked collectively to create your organization’s theory of change, you can work on creating each work area, service, or program area’s logic model.
Gather the key stakeholders together to be involved in creating the program’s logic model. Stakeholders who are typically involved in the process include leadership staff, program staff, development staff and board members. Simply assigning this to a logic model staff member to do on their own misses the opportunity to ensure that there is shared agreement of the program’s goals and methods.
Together the group will fill in the key elements of the logic model. What are the inputs required to deliver the program? These may include, staff, volunteers, program participants, partner organizations, supplies, curricula, and other items. What happens during the program, i.e. what are the activities? With those activities identified, define the time frames. What do you consider short-term, medium-term, long-term? With this mapped out, move to your expected outcomes.
Pay attention to what are reasonable expected outcomes and those that are more aspirational. Think of your outcomes like a hypothesis that you will then test by beginning to capture data. These discuss the assumptions that must be true for these outcomes to happen. With this step you may identify missing elements of your program or leaps of logic that are not supported by the actual work you are doing.
Compare your completed logic model to our organizational theory of change and mission. Do the expected short-, medium- and long-term outcomes contribute to moving your mission forward?
From Theory to Practice: Implementing Your Logic Model
Once your logic model is complete, you can then decide what performance measures you will focus on and how you will collect data on those measures. These data do not all need to be quantitative. In addition to quantitative data collected through surveys, often qualitative data including gathering stories and experiences through interviews will be an important component of the evaluation design.
Measuring and Mapping Impact
With this process, there can be the temptation to start collecting data about everything and then get overwhelmed and over time the data gathering stops, or you have a lot of data that you do not have time to synthesize and make meaning of. Keep it simple and choose a few key items to track so that you have the time to do the truly important work – learning from your evaluation.
What are you learning from your team conversations?
As you collect data and synthesize it, you will learn more about the effectiveness of your program. You may identify areas in which your hypotheses were not entirely accurate. What can you learn from the information? Apply your learnings by adjusting the program. This may mean tweaking your logic model as new information is gathered.
Communicating results to your stakeholders
The information you gather will have many applications. In addition to reporting results internally to your staff and board as well as to funders, you might highlight results on your webpage, blogs, social media, and annual reports. As you are doing so, consider each audience and their interest and perspectives. What do they care about and how can you link what you are doing and your results to that? Ask this question for each audience as you decide what aspects of your results to share. Just producing one findings report misses many opportunities to tell your story and share your impact.
Overcoming 3 Common Challenges to developing a logic model
Not connecting the theoretical processes to practical application
These processes can seem a little esoteric. It helps to have an evaluation facilitator help walk you through the process. One of the common sticking points is the difference between an output and an outcome. You are likely already tracking outputs – how many people showed up to a training or the number of times you have testified before government decision makers.
But what are you expecting people to be able to do after the training – answering that question helps you start identifying outcomes. A facilitator can keep the conversation going, help everyone understand the purpose of each step and ensure all the items are captured s as the group brainstorms.
Realistic expectations of what your data are telling you
Other objections to engaging in this work have to do with whether you can prove your program, service or intervention is THE factor that creates the outcome. Organizations rarely have the resources to evaluate their programs at that level, including running double-blind tests. While you may not be able to invest in evaluation to the level that will enable you to prove your impact, you can design a system that help board and staff to continue to strengthen the program as well as gather evidence of your impact.
Does not represent the full complexity
Another pitfall is rejecting logic models because they cannot fully capture the complexity of the system that program participants are living within or are themselves. Indeed, the map is not the territory. No model will fully capture life in its full complexity. Yet it can provide a slice of that reality that your work is focused on. With these understandings in mind, you can proceed with more realistic expectations for what a logic model, monitoring and evaluation can do for your organization.
Social Impact Evaluation Design is worth the investment
Investing the time in mapping out your organization’s theory of change and logic models is an important aspect of ensuring your stakeholders are aligned on your organization’s strategy. The process can reveal and then address gaps in logic, understanding and agreement on how each aspect of your work contributes to your overall mission. Tackling this as a collective project helps educate everyone involved not only about the process itself but also about why and how you do what you do. It can contribute to breaking down silos within your organization. With the information you collect because of this work, you are also better able to tell your story of results to donors, funders, board members, volunteers, community partners and decision makers.