Making social change efforts more effective

Many organizations take on big problems like alleviating poverty or combatting climate change. But how can they—and their funders—know that their efforts are working? What are the best ways to measure complex interventions and determine which approaches lead to true change?

These questions are at the heart of a new book by Alnoor Ebrahim, a professor of management at The Fletcher School and the Tisch College of Civic Life. "Measuring Social Change: Performance and Accountability in a Complex World" describes how having a clear strategy can help leaders of social change organizations assess their achievements and boost their impact.

While businesses measure success with standard metrics like profits and market value, such measurements don't fit most social change groups, Ebrahim said.

"Depending on what you care about, whether it's health care or education or poverty alleviation or climate change, what's relevant to measure will differ," he said. "But what every organization needs is clarity about its strategy, which can then provide a roadmap to its most important metrics."

Tufts Now spoke with Ebrahim to understand how a focus on clear goals and methods to achieve them can help social change organizations and funders—and how the same ideas can guide individuals making decisions about their charitable giving.

Tufts Now: Nonprofit leaders sometimes think of impact measurement as a demand imposed on them by funders: "You've got to report back on this, or we won't give you more money." But you're saying it's not just about compliance, it's about strategy. Can you explain?

That tension is at the core of this book. I wanted to ask: If you were a leader of a social change organization—such as a social enterprise or a nonprofit organization—how could you drive performance measurement based on your organization's goals, rather than trying to respond primarily to the divergent demands of funders?

Three foundational questions for organizations guide this book. The first one is: What are we trying to achieve? That's your value proposition. Following from that is: How are we going to get there? That's your social change model.

Then the third piece is accountability, but it's not about accountability to funders. It's about holding yourself accountable for what you're trying to achieve and to whom you seek to serve. The question is: How will we hold our own feet to the fire? This is accountability that's driven internally by strategy, rather than by compliance with external demands. Being clear about your accountability helps you to stay focused when the external demands pull you in different directions.

If you put the answers to these three questions together—your value proposition, your social change model, and your internal terms of accountability— that's your strategy. Even in a small organization, there is likely to be considerable disagreement on each of these aspects of strategy. A key job of leadership is to ensure that everyone, from front-line workers to board members and funders, are aligned around that strategy.

How do different strategies lead to different ways to measure success?

A central contribution of the book is that it identifies four primary types of strategies for social change: niche, integrated, emergent, and ecosystem strategies. Each type of strategy requires a distinct type of performance management system.

A niche strategy requires you to be highly focused on doing one or two things well. Take the example of an ambulance service. We expect ambulances to respond quickly to emergencies and to provide quality care en route to a hospital. What should they be accountable for? They must respond quickly to an emergency (less than 9 to 12 minutes for a cardiac incident) and they should provide standardized quality care on the way to the hospital. These two metrics—response time and reliable quality—are short-term output measures, which are perfectly adequate for their strategy.

It wouldn't be reasonable to hold an ambulance service accountable for long-term health outcomes, because that is beyond their control. Health outcomes depend on a combination of the care a patient receives at the hospital, any follow-up services, and even preventive interventions like diet and exercise. An ambulance service is a "niche" in this larger healthcare chain, so its performance is best assessed in how well it delivers short-term results before passing on the patient to the next niche. There are many other services that deploy a niche strategy, such as a soup kitchen that feeds the homeless, or disaster response after a hurricane, or even an after-school education program.

But if you're thinking about long-term change, both your strategy and metrics would be different. For example, if you're committed to improving economic opportunities for urban youth, you may be providing job training and job placement, while also working with employers to support and coach the youth, and getting them back into the job market when they fall out. That's an integrated strategy—because it combines a sequence of niches to deliver a long-term outcome.

In that case, you would measure not only the short-term outputs for each of these niches, like job training and placement, but also the long-term outcomes on the lives of those individuals, like better integration into society and lifetime improvements in income. So being clear about your strategy helps you determine what you should be measuring and holding yourself accountable for.

You devote a chapter to Miriam's Kitchen, an organization serving the homeless that switched from a niche strategy to an ecosystem strategy, where it is coordinating the efforts of many groups. How is that different?

In the first two examples I gave you, the ambulance service and the job training and placement, we have a pretty good understanding of cause and effect. But Miriam's Kitchen, a nonprofit that serves the homeless in Washington, D.C., operates in a complex ecosystem where there are lots of moving parts and many other organizations trying to intervene. It's harder to determine cause and effect in such a complex, interactive ecosystem.

Miriam's was initially a soup kitchen, and it was performing very well in that niche of providing warm meals and clothing to its clients, along with referrals to other services. They were reaching a lot of people, and serving high-quality, truly delicious, meals! The former First Lady, Michele Obama, once served a Thanksgiving meal there. As a manager in such an organization, what could you reasonably measure and hold yourself accountable for? You'd measure short-term results such as numbers of people and meals served. That is all your strategy enables you to achieve.

But at some point, the leadership of Miriam's Kitchen asked themselves: Could we be doing more? They recognized that they were only one of over 100 organizations in the D.C. area that were working with homeless clients—with services ranging from providing meals and temporary shelter to substance abuse and mental health counseling. It was a fragmented ecosystem, with lots of uncoordinated niches.

Miriam's decided on an ambitious goal: to end chronic homelessness in Washington, D.C. But they knew they couldn't do this alone. They adopted a model called Housing First, or Permanent Supportive Housing, which had been pioneered in New York City. It is premised on identifying the most vulnerable people—those who are repeatedly or chronically homeless— and putting them into housing first, and then coordinating the niche services of dozens of organizations, such as substance abuse counseling, job training, meal provision, and so on. In other words, they sought to fundamentally restructure the ecosystem in order to provide a whole range of services in a much more coordinated and efficient manner. This model has been shown to be very successful in reducing chronic homelessness.

The challenge for Miriam's, or for any organization with an ecosystem strategy, is that the solution requires collective action. They can't say any one intervention solves this problem. What's the contribution of substance abuse counseling? What's the contribution of health care? What's the contribution of job training? Or of affordable housing? Or policy advocacy? So neither Miriam's, nor anyone else, can take sole credit for addressing the problem. The ultimate outcome measure—the number of chronically homeless people—is not difficult to measure. It's the strategy that's hard.

An ecosystem strategy is relevant to all sorts of social problems where no single organization can solve it alone and where cause-and-effect are hard to disentangle. Climate change, for example, requires coordinated action among all sorts of interventions, from the adoption of clean energy sources and efficiency, to forest conservation, carbon capture, and the redesign of cities, buildings, and transportation networks. Nobody can address this immense problem alone. As with the homelessness example, the key outcomes are not that hard to identify and measure—such as carbon in the atmosphere—but it's the strategy for achieving collective action that's hard.

If a social change organization adopts an ecosystem approach, and can't claim credit for the results, will funders still support it?

This is such a critical challenge because donors typically want you to measure your impact, rather than the impact of the collective. And they often want to fund direct services rather than the coordination of those services. Funders need to be aware that ecosystem-level problems require funding the infrastructure for collective action. No one can take sole credit for such work.

A handful of innovative funders are rising to this challenge by funding ecosystem "orchestrators" and by developing a portfolio of grants or investments that together can move the needle on a social problem. You wouldn't expect any of those grantees or investees to measure systemic impacts. That responsibility falls on the funder. An example of a funder with an ecosystem perspective is New Profit, founded by Tufts alumna Vanessa Kirsch.

How can your ideas help individual donors think about their charitable giving?

When we give money to organizations, we can ask what we can realistically expect an organization to deliver based on its strategy. So I ask two basic questions: Are they transparent with me about what they can reasonably deliver? Do they specify what they hold themselves accountable for, and how far along they are in getting there?

If they have a niche strategy that provides a focused service, they need to be clear about what they deliver and the quality of that service (such as meals or emergency response). If they have an ecosystem strategy that aims to address big collective action problems, they need to show that they are working with others to try to deliver on those ambitious goals. If they're promising to change the world by themselves, they're probably overselling!

Can your approach also help people consider the social impact of their careers?

Yes, and this is particularly relevant at Tufts, because of the university's orientation toward social change. Both at the undergraduate and graduate level, I think it's important for students here to ask themselves those same three fundamental questions that I frame in the book: What is it that I want to achieve, both in the short and the long term, that will contribute to society?

Second, what's my model for social change? How will I go about doing that? How will I equip myself while I'm at Tufts so that I'm best able and ready to address various kinds of social change problems?

Then, over the long term, how will I hold myself accountable for that? This requires revisiting your initial goals and your pathway periodically to ask: Am I happy with my contribution to improving my community or society? And if not, what's my strategy going forward?