In this guest post, Paul Collier from Coeffect shares how you can get started measuring the impact of your MilkCrate app. Morgan Berman from MilkCrate also chimes in with some of the features the platform has to offer for making impact tracking easy.

Moving to Colorado gave me a whole new appreciation for maps. Colorado has some of the most beautiful geography in the world, and to navigate it I had to become comfortable with reading a topographic map (topo map) for the first time. I quickly learned that even the best map will leave some important details out - a short but treacherous cliff, the part of the trail that is always windy, the spring that isn’t reliable. But travelling with the map is still infinitely safer and more fun than travelling without one.

*A topographic map of the Maroon Bells near Aspen, Colorado. Accessed from caltopo.com.*

Enter the Theory of Change

As a nonprofit professional, you’ve probably heard of the term “Theory of Change” (ToC). A Theory of Change is like a constantly evolving topo map for your organization, that describes how you plan on making a lasting positive impact for the people you serve. You can define a Theory of Change at many different levels, from the app that supports part of your program, to your organization as a whole, and even an ecosystem of organizations.

At its core, a Theory of Change outlines who your program/organization/ecosystem serves, what problem you’re trying to help them solve, what activities you do to create an impact, the results you intend to achieve, and your beliefs about how change happens. Bennett’s Hierarchy is a framework I like that illustrates these concepts in a useful way. Bennet’s Hierarchy has the following elements:

Inputs: What investments of time, money, and other resources does it take?
Activities: What will you be doing?
Participation: Who do you engage, and for how long?
Reactions: What experience did your beneficiaries have? Are they satisfied?
KASA (Knowledge / Attitudes / Skills / Aspirations): Common kinds of shorter-term changes that beneficiaries may experience as a result of working with you.
Behavior or Practice Change: What did people do differently, based on the KASA they gained?
Impact: What is the ripple effect from the one individual served to other members of their family, community, or society?

Would you be surprised if I told you that most nonprofit teams aren’t entirely on the same page about these things? I’ve found that once a team is clear on it’s Theory of Change, it’s much easier to collect the right information and tell a compelling impact story. Creating a Theory of Change as a group also builds team alignment - everyone can see how their role contributes to important results.

Why does a Theory of Change Matter for your MilkCrate app?

Chances are your organization has or may at some point need to win philanthropic grants or government contracts in order to grow. Many grantmakers and government entities are now expecting their grantees to share a coherent ToC. Having this already drafted and backed up by data from a tool like a MilkCrate app or elsewhere will give funders confidence that your organization is a good investment.

As one MilkCrate foundation partner put it “[T]he app can be beneficial in showing the organization which programs have the desired impact and if that impact can be successfully expanded...[and] an organization can direct its limited resources (our grants) to better their programs. In short a non profit can operate more efficiently. It can also give the organization data showing its impact that it can use when seeking additional funds from foundations and other financial stakeholders.” To learn more about why foundations like seeing program data generated by nonprofits using MilkCrate you can read here.

The Theory of Change for your MilkCrate App

Since founding Coeffect, I’ve helped over 25 small and mid-sized nonprofits establish their Theory of Change. No two Theories of Change are the same, but there are some elements that are common for organizations who are already engaging or could engage their community through a mobile app. Consider the Theory of Change elements and indicators below as a starting point for your own conversation with your team.

Inputs

Inputs are the resources your organization leverages to make a change happen. These almost always include money and time (by staff, board members, volunteers, and others). This also often includes any key partnerships, intellectual property owned by your organization, the technical infrastructure you use (including your app), and community trust.

Input indicators are data points that illustrate the extent to which you actually have access to these resources. This could include funds raised, number of staff, number of key partners, and more.

Activities

Activities are the things your organization does to create an impact. Common activities include awareness-building (events, marketing activities, outreach), providing access to resources, making referrals to resources, connecting individuals to other peers, and whatever additional goods or services you provide to the people you serve. These can be shared and tracked using an app like MilkCrate where you can share events, actions, and content - all of which can be strung together into goals.

However, just offering activities is often not enough to get people engaged. Often, organizations need to also build trust and a belief among those that they serve that working together will be beneficial. Empathy interviews are a common approach many organizations use to determine how to design an app to maximize trust and the belief that the app will be helpful.

Another way MilkCrate supports engagement is gamification - the ability to award points for certain behaviors. Gamification signals what are “desirable behaviors” and also can build a positive form of peer pressure to engage in those behaviors.

Participation

Participation means the extent to which people are actually engaging in the activities you named in the last step. It looks different for each organization, but if you’ve built an app to serve your community, it starts with “how many people in your target audience downloaded your app?” Depending on your app, there may be different levels of participation that are important to track after the initial download - for example, completing a user profile, or taking their first action on the app. For an app focused on engaging volunteers, registering for and checking into shifts may be important indicators of participation.

Output indicators are data points that illustrate participation in your organization’s program or services. For organizations with an app, these could include the number of app downloads, the demographics of individuals downloading the app, points earned by users on the app for completing activities like reading an article, checking-in at a location, chatting with a mentor or answering a survey. With the MilkCrate system, setting and achieving Goals provides extra incentive to engage with an app in a meaningful way.

Reaction

*The data shown here are an example of a cohort analysis. This is one way to analyze engagement with your audience from an app.*

Reaction explores whether individuals were satisfied with their experience with your organization. There are many ways to measure satisfaction, from Net Promoter Score and app-store ratings to user retention and Cohort Analysis, or in-app surveys and polls. When collecting and using this kind of data, it’s important to disaggregate your results by demographics. You may find, like one of my clients, that while they were targeting lower-income beneficiaries they were getting more engagement and seeing greater growth from middle-income individuals. Insights such as these will let you know if you should be altering your approach, or doubling down on current strategies.

Outcomes (including KASA, behavior change, and impact)

Finally, we get to outcomes - the meaningful changes in knowledge, attitudes, skills, aspirations, behaviors, or conditions that you’re hoping to influence. No two nonprofits have the same outcomes, so I can’t share any generic examples. What I can share is that apps provide a great opportunity to collect this data in natural ways. For example, if your app encourages users to check-in at a local resource center, you now have some data about your beneficiaries’ awareness and use of resources.

I’ve found that for younger organizations and new programs, hosting targeted interviews can be extremely helpful in understanding how people are benefitting from whatever your organization is doing. As your organization grows and matures, monitoring quantitative data becomes important to ensure your services are being provided as planned and consistently lead to the intended outcomes.

Communicating with your Theory of Change

Generally, organizations develop two versions of their Theory of Change: 1) a graphic slide or PDF that can be explained in a presentation, and 2) a narrative that explains each part of the graphic in detail, which can be used in a grant application.

If you’re curious what a Theory of Change looks like, here are some good public examples:

Hospitality House, a housing services organization based in the Bay Area. I love their clear problem statement, and that they included their values.
Big Brothers Big Sisters, a nation-wide youth serving organization. I appreciate the simplicity of this, and the use of icons to illustrate the interconnections between outcomes.
Washington State Community Action Partnership, a poverty-alleviation organization based in the Seattle area. Their Theory of Change is a great example of a continuum from immediate crisis response to longer-term resilience-building work.
This Pinterest board contains many other great graphic examples of Theories of Change, in case you need additional inspiration.

*Hospitality House’s Theory of Change, one of many graphic Theory of Change examples.*

Putting it all Together

A Theory of Change is like a map that helps your organization navigate it’s way towards making a meaningful impact. It’s helpful for your staff to get on the same page, and a strong Theory of Change can be a big asset when applying for grants.

In this blog, we talked about the important components of a Theory of Change: Who your program/organization/ecosystem serves, what problem you’re trying to help them solve, what activities you do to create an impact, the results you intend to achieve, and your beliefs and assumptions about how change happens.

We also shared examples of what might be in your Theory of Change if an app is an important part of the support you offer to your community. And we linked out to several examples you can be inspired by.

Do you already have a Theory of Change for your MilkCrate app? If so, we would love to see it and highlight your work for the rest of our community - please share using #MilkCrateToC

Paul Collier is the founder of Coeffect. Feel free to contact him at paul@coeffect.co with any questions related to this blog. In addition, you’re also welcome to sign up for his monthly newsletter, where he shares a collection of the best resources related to nonprofit data use.

How to Build a Theory of Change for Your App