How to measure social impact and see if your CSR programs are working
Watch the episode:
Prefer to listen?
What we discussed:
Karl Yeh (00:00):
So I've got two special guests today. I've got Nicole McPhail and Emily Hazell, who are the co-founders of Darwin Pivot.
And we're going to be talking about social impact data and measurement. Thank you, Nicole and Emily for being on the show today.
So let's get right into it.
So how do you go evaluating social impact?
Because for me, someone new to the space is pretty broad and there's nothing really say specifically things that you would do on a regular basis and maybe I'm wrong.
So how would you define that?
What is social impact measurement? Why is it important?
Nicole McPhail (01:06):
It's important for a lot of reasons, but before I get into that, let's just talk about what it is?
And so what it is really very nuance and specific to whatever company you're working for and what it is you're trying to achieve with your program or strategy.
So every single CSR or social impact strategy is different.
It's based on history and the company who brought it in the first place where you're getting the money to do it, because that also puts you in different business units and then you have different objectives.
So there is no one specific way that you can do this, but it's really important to do it for a few reasons. So the first is, it depends on where you are?
I guess, in your journey.
So for instance, if you have a foundation board, you might have a fiduciary duty to report out on the outputs and inputs of your program.
So exactly how much money is spent, where it's spent, who's involved, how you're benchmarking? All of those things.
What percentage of pre-tax revenue is going out because you basically have to do this and your foundation board expects this, but then there's social impact metric from say a program perspective.
So this is for the individual who's running or the team who's running these strategies.
And you have to really think about what you want to achieve within your program.
So let's go super basic understanding that there's so many aspects of CSR, but let's go into the employee giving and volunteering space.
Say you have some reason that you want to increase participation. Well, looking at that participation number on its own, isn't going to do anything for you.
You need to figure out how to break it apart, to be able to understand gaps and opportunities, and then track that over time.
And so the important piece here, and we'll get into more details in up further in our conversation.
But the important piece here is though you need to be really consistent. And you need to know why you're using the data in the first place.
So if there's a disconnect between what your strategy is, what you're solving for and what you're saying, your stakeholders care about and the metrics that are tied into your program, what's the point, right?
So I would say that's a really good place to start and go from there.
Karl Yeh (03:26):
Talking about where a good place to start would be would.
So can you tell me a little bit more about the social impact measurement framework, because I've heard that term before.
And I think that's something that I've heard a lot of social impact professionals.
That's where they would like to start get that framework going before they actually start measuring.
What is a social impact measurement framework?
Nicole McPhail (03:44):
And I feel like framework is one of those words that I think everyone is using it a little bit differently depending on their own sophistication and just again, history and how they even use certain nomenclature within their company.
So when we think about a framework, really what we're talking about is just a way that you're looking at your data and a way that you can review it over time.
So some people are calling it dashboards and then some people are also using framework in a more specific way that is related to say a standardized process, which Emily and I have tons of thoughts on that, which we don't know if it's even possible to do that just based on all of the nuances and things like that.
So in terms of a dashboard again, think about what you're trying to achieve with your strategy and then break down, how do we achieve that?
Those should be your metrics. And so that could look like for some people maybe they want to measure where they're giving back, what types of organizations that tie into company causes, what types of employees are giving back and why?
But others might be looking at it from on a more granular level.
So when we think about a framework, really what we're talking about is just a way that you're looking at your data and a way that you can review it over time.
So if you think about different behaviors that your employees are doing within a company, for one example, how can you really zero in and understand, okay, so this person isn't just giving once a year, this person is creating opportunities for their friends or colleagues to get involved several times a year.
What does that say about that person's level of engagement and what do you want to do differently or not differently to try and get more people to follow suit or the opposite, just as one example?
So it really is dependent on what you're solving for?
But being really smart about how you're thinking about that data.
Karl Yeh (05:35):
So let's say I'm a beginner, social impact professional or someone who's just starting a program from scratch.
Where do I start when measuring my social impact programs?
Nicole McPhail (05:47):
Reporting for foundation board
If you are working with a foundation board, you might have a fiduciary duty to report out on specific inputs and outputs that are very qualitative.
So how much you're giving over time, how that's changed over time, how that compares to other peers?
And this could be product, time, money.
All of those types of things would be rolled into this and then an overall investment and percentage of pretax revenue, for instance.
That would be something for a foundation board.
Reporting to your executives
If you are working with say an executive committee who are your decision makers, I would say before you even start thinking about how you're going to track your program, you need to understand what they care about most.
Because for a lot of CSR professionals getting the resources and support to run these programs is paramount to your own success.
So the community aspect of measurement, we can talk about that in a minute, but this is having this right is going to empower you to do a whole bunch of other stuff.
So for instance, say you have a bunch of executives who are care a lot about retaining employees or engaging, or even attracting employees.
Then you can start thinking about not only just measuring their engagement, but also the sentiment and how they feel about the program in the first place.
So you've got the transactional stuff like your participation, how much you're giving per region?
And then you've got the qualitative stuff. So what it means for them and how they identify with the company from that respect.
So I would say that's a part of it, or maybe they care about innovation.
So then you could start looking at, in your program, how many new partnerships have sparked that employees are helping solve for problems in the community, that type of thing then aligns.
So then you can step back and think, okay, at a high level, this is how we can create our dashboard because we're aligning it to our stakeholders.
I think sometimes we miss that step and then we're creating these dashboards and submitting them and our executives to make the decisions are going, what does this even mean? What does this mean to me?
Emily Hazell (08:11):
Maybe I could just jump in here briefly, Nicole, as you were talking, I was just thinking about my background more is in academia.
And often when you're putting together any framework or model or project, you're always asking yourself two questions.
So is it reliable? And is it valid?
So the reliability component is around that consistency of if, is this a robust measure of something over time that you can use to make better decisions?
And the ability is around, are you measuring what you actually want to measure?
So making sure that you put that stake in the ground at the beginning and knowing what you're solving for and what your objectives are and who your key stakeholders are, then frames up that initial bones of the framework to make sure that you can always go back and make those checks and balances, make sure that you're consistently being valid and reliable when you're measuring what you set out to measure.
Nicole McPhail (09:01):
So that's such a great point, Emily.
And I feel like once you do that, then you can start thinking about benchmarking to me is really related to effectively setting up a framework because also when you're presenting this information, without context, your executives are going to be like, "Well, what does this mean?"
So if we have say again, participation is just an easy one, even though we'll talk about some more in-depth measures, say your employees are participating at 11% and your partners and stakeholders are looking at this.
What does that mean to them, right?
They don't know what's good. They don't know what's bad.
So it's almost partly having a dashboard and then clearly articulating what your specific goals are. Sometimes it just means putting a stake in the ground.
So find out what some of your peers are doing, throw it out there and see how you do for the first year and then be clear about that and then modify it for subsequent years.
Emily Hazell (09:57):
And I think too, as part of that really, there has to be a process then of exploring these metrics or these data points or variables from different perspectives or different scales, so that you can start to get that deeper understanding.
So what Nicole was saying is we take a bird's eye perspective at the beginning and get this holistic perspective and then starting to drive in on some of these more localized trends or even individual trends or behaviors can then really give us this overarching, deeper understanding of the program health overall.
Karl Yeh (10:27):
So Emily, you just mentioned about social impact data.
How do I get social impact data? What are social impact indicators?
Emily Hazell (10:38):
Well, I guess it's company dependent.
So if you're using a particular software, you're going to have a lot of data points from that.
If it's a newer program or a smaller program, and you don't have any data to collect from the beginning.
It might actually be starting with some of these qualitative data points.
So actually starting to go out and survey your employees and getting those initial data points that you can start to benchmark from.
Nicole McPhail (11:02):
And I think the technology is a big part of just making it a lot when you know how Emily is talking about good data, the technology ensures that it's accurate.
And then you're able to really zero in on specific things that you're looking for. And sometimes with a technology, you might not even realize the insights that you can get.
Even just seeing how often your people visit certain pages in your technology to understand what they care about.
And then you can start to shape your program and then you can work backwards and take some of the metrics from there.
Karl Yeh (11:36):
And is it heavily reliant on self-reported data?
So like, "Oh, If I'm doing this activity, I need to input this," but I'm sure there's... I know opportunities or times when employees or people are doing something that's going to be harder to track or is that something that's one of the challenges?
Nicole McPhail (11:58):
Oh, it's a thorn in my side, it's so hard to capture everything and a lot of stuff is missed and sometimes people don't like reporting.
They like to, and especially in some cultures to keep their charitable donations and their volunteering separate, which makes it really challenging.
The other thing with reporting is so we can measure from a corporate level inputs really easily and even outputs.
So inputs for people that are newer to this would be how many hours employees are volunteering in certain regions?
And then an output would be how many say donations were being made to an organization?
The outcomes are really challenging, and this is a bit of a controversial issue, but being able to measure actual impact on the other side, I think is arguably, it's almost impossible to your point, even just around, you might not be able to collect all the data coming in.
It's the same on the way out.
And it's almost, I would say counterintuitive to expect, not for profits and charities to do this heavy lifting for you.
They're already understaffed. How can they possibly and accurately report out on the very specifics of what your company's money is doing to their program?
So that's a sidebar thought that I think is still relevant to this conversation because there's a lot of CSR practitioners that are trying to figure out how to do this.
And I would argue, why do you feel the need to do that?
Emily Hazell (13:35):
Well, maybe that's almost just the going back to what are you solving for when we're talking about, when we're looking at the program health metrics versus more outward looking, social impact metric?
As a CSR practitioner, you can really get some interesting narratives and data points and tell your story about what your people are doing and how it's impacting.
And then a lot of the times, a lot of these organizations that you're partnering with, they might have another way of reporting it that you can pull in.
So instead of trying to reinvent the wheel all the time and having it go through your channels, maybe there's already reports that they're doing that you can start to pull in on.
If you wanted to show in case some of that outward social impact versus the internal program that [inaudible 00:14:20].
Karl Yeh (14:20):
So you touched on the difference between actually measuring impact or impact itself and program health metrics.
And on the face of it, to me, is that there's measuring what your people, your business, your company's doing versus measuring the actual success of your program.
Is that a correct definition between the two?
Whats the difference between impact and program health metrics?
Nicole McPhail (14:43):
Well, the fun thing with that is how do you define success?
So going back to my original point on, so if your executives care about how your employees perceive them as an employer, maybe we know that a lot of new talent really cares about working for a company with strong social values as an example.
So if this program can accomplish that and get feedback from your people around pride, even if its qualitative data, maybe that is success.
And you also know that you're donating money to pre- vetted organizations that you are trusting them to do what's right with that money?
And you've built those partnerships for that exact reason.
So I would argue that you can think differently about the success of the program and control what you can control, which is all of the internal stuff, being strategic with where you give your money, thinking about how you're engaging your employees, how you're helping them role model?
Those social behaviors in their families and communities rather than placing success on the outcome after it reaches the not-for-profit, that is really hard to control.
Yeah, to me, it's a trust. It's a trust thing and doing what you actually can instead of playing make believe that you can do something that you can't.
Emily Hazell (16:02):
What I think to that point? You partnered with these organizations for a reason.
And so I think building that trust in knowing that if you are investing or your people are investing time and money that they know best and they know their own space and their own cause is best.
So making sure that you're putting that level of trust and accountability, and then that's how you build these really strong relationships moving forward.
Karl Yeh (16:27):
Now, there's obviously the big term of ESG, so environmental social governance.
So where does that fall into play?
Because there is a big business, there's a big push on ESG metrics, ESG standards, ESG framework. So how does ESG relate to social impact measurement or are they the same thing?
Where does ESG fit into social impact measurement?
Nicole McPhail (16:50):
Depends on you're asking, just because it's somewhat new in our industry.
It's challenging.
I saw a survey that was sent out to a whole bunch of social impact practitioners about what percentage of those senior people were actually reporting out on ESG and the real way?
And it was 2%.
So we have 98% of people who are still trying to figure it out who have different companies, have different resources internally, all wanting to be a part of this because there's positive social benefits and to be doing this and brand benefits, I guess as well.
And so right now, as it stands, I would say standardizing, something like this is pretty challenging.
And so I think that while this is exciting and it might promote some really good behaviors, it's not accessible to everyone.
So I was an in-house CSR practitioner for Fortune 500, and I wouldn't have been able to get the analysts required to actually complete this level of standardization at this point. And it was a big company.
So I say for right now, we need to remember that there's two aspects. So there's one from a corporate level of... Okay, these are due diligence, things that you should be doing as a company. It should be a part of the way you do business, but at the same point, there's also social impact metrics that you need to think about just to be doing the best you can do with the existing resources that you have, your internal programs.
What can you measure there? And what does that mean?
Sso I guess to answer, to summarize, there's a place for this, and I know there's a lot of people trying to align some of the things that they're doing with the S and ESG like, "Hey, we're giving back this money," but that's no different.
They're not really making any changes to what they're doing.
They're just aligning it similar to how we thought, think about sustainable development goals and how we can support those.
But setting that aside, you really should be focusing a lot of time on your program, health metrics, and social impact from that lens.
So you can continue to improve.
So you can continue to get that buy-in internally, and really see how you stack up against other people.
Emily Hazell (19:20):
Well, what you got me thinking about Nicole was just also, and you touch on this.
So just to later on with the standardization, what you just want to be cautious about?
And I think generally from a data analyst perspective, standardization is something that we should be striving for, but it shouldn't just be the holy grill.
We should also be talking about nuance within industry specific company, specific team specific, so that you're actually measuring apples to apples, not apples to oranges.
So for example, if we're looking at these global standardization measures of ESG, is that a fair comparison for a small startup or a small company?
And does that make sense for their local context? So sometimes we have these really high-level standardizations that make sense on a global scale, but do they make sense when you actually zoom in on a more localized context?
And so again, it's just about that context and that nuance and making sure that we're looking to these benchmarks to better our program and not just to align to something that might not make sense within the context of our company or our situation.
Nicole McPhail (20:28):
And I think there's a lot of CSR people, ones that I've talked to that are stressed about this because their executives are looking at them and going, "Hey, we're hearing about ESG, that's the same as CSR, right?"
Can you do it? And they're going, well, maybe one day, give us some resources to help us think this through. Yeah.
Emily Hazell (20:48):
Well, and it's a new paradigm, right? So everyone wants to jump on it, which is great. And it gets people excited and it gets people involved,
but it's going to develop over time and we'll have a better understanding of what those benchmarks look like over time and how best to use them within your own company or team.
Karl Yeh (21:06):
So let's shift to Darwin Pivot. So can you tell us a little bit more of how your company is helping social impact professionals?
About Darwin Pivot
Emily Hazell (21:15):
At Pivot, Nicole and I, we're both, we're very passionate about empowering our clients using novel data insights.
And so metrics are hard.
We've been talking about this now throughout this episode, and it's hard to know what to focus on. And there's a lot of information, a lot of data out there, and what should we be prioritizing and how should we be using our data?
So we really try to come in as the role to help companies understand their data better.
And these links to these metrics and how it can help them with more evidence-based decision making, moving forward, and also their overall strategy and their narrative within the CSR and social impact space.
Nicole McPhail (21:56):
So I guess it's partly the strategy, but the really cool thing that we can do is sometimes it's not about what the data is, but how you're presenting it.
So I remember going into foundation board meetings and sending the pre-reads to an executive committee that social impact is not their business and they're super busy.
So they're looking at these things and not really understanding what this is trying to convey.
And so we have a technology that basically can tell a story in a very visual way, like picture, zooming in on a map and different insights popping up to tell this story that you don't even need to be in the room for someone to understand what you're talking about so that's what we do.
And I think the last of benefit in general, and this is not about Pivot, but if you think about any time you've tried to analyze your strategy or program and say, you're looking at one metric, a single metric has often so much more when you start pulling back the layers.
And I actually, I read about this recently, a perfect example is there was a really prominent Ivy League University who was talking about how they were failing in a gender bias.
And they thought that they were accepting too many men over too many women and a data scientist goes, "Wait a minute, let's have a little bit deeper look here." And when they pulled back the layers, they realize, "Okay, so this is overall admissions." We need to look at every single program that they're applying to.
In fact, we're in favor of women, they're just not applying to some of the easier courses. So what they found out was they were hiring more women almost a little bit than men, but from the higher level data, it looked like the opposite.
So that's just one example of how you really need to be strategic and think critically about what these numbers mean, because it's not what you think on the surface level for the most part, you're the data scientist. Did I do that right?
Emily Hazell (24:14):
I think really with a lot of this, just, yeah, I think it's just being curious and exploring the data from multiple perspectives in scales or looking at it spatially or temper, looking at it in all different dimensions.
It can help you just to make sure that you're not coming in from a place of assumption or bias based on what you want the data to be telling you or the narrative you want the data to be telling you, you're letting the data speak for itself.
And so the more that you can explore and ask, follow up questions to yourself as you're going through the data, the more you'll really start to dive deep into that.
And then it starts to become like you, as Nicole said, you start to slice and dice the data in ways that you really get these. You're really driving more at the root cause when you understand these insights that you wouldn't have had when you looked at it in such an aggregate way.
And so you have to really, again, just let the data speak for itself and make sure that you're taking the time and listening to it as you move through it. So become a data nerd.
Karl Yeh (25:21):
No, we could definitely speak to this for a very long time.
And thank goodness is just part one. We're going to get a part two, which I'll leave in the description below, but if our audience wants to connect with either of you or wants to learn more or work with Darwin Pivot, what's the best place to reach you?
Nicole McPhail (25:40):
So yeah, you can find us darwinpivot.com and I'll give my email address. It's nmcphail@darwininc.org