Almost $12 ROI for every $1 in social benefits: what’s really behind the number

 

Simprints and BRAC ran a maternal and child health programme in Bangladesh. We asked Dr. Brad Wong, President of Mettalytics, the independent economist who built the economic case for it, to explain the headline number in plain language: what it means, why it’s high, and where he’d push back on it.

 

 

 

 

For every $1 invested, you found almost $12 in social benefit. What’s behind that number, and how much should we trust it?

An ROI number is attractive for a reason. There’s enormous complexity behind what an organisation does and whether it represents good value for money, and the ROI distils all of it into one thing you can repeat: for every dollar invested in Simprints technology, you get about 12 dollars in social benefit. It communicates value without anyone needing a background in economics.

Behind it, we trace how Simprints changed behaviour, iron and folic acid, tetanus injections, facility births, turn each into a change in health risk, then into lives saved, then into money. We count what both Simprints and BRAC spent on the cost side.

But every model has assumptions built in, and the more you make, the more uncertainty comes with them. So the honest reading is: best guess 12 to 1, with a real range around it.

Simprints is an enabler, not a health programme in its own right. What does that mean for its value?

Simprints amplifies whatever intervention sits behind it. Most of what it supports, vaccines, nutrition supplementation, vitamin A, is highly cost-effective, so supporting it lifts the return. But the flip side is real. In the BRAC study, the programme moved more women into facility births, and at the time not every facility could actually save a life in an emergency. So you were moving women into a more expensive place to give birth without the infrastructure to deliver the benefit.

When I took a closer look in the sensitivity analysis, the ROI actually went up when facility births went down. The technology is only ever as valuable as the care on the other side of it.

What surprised you most in the results?

The impact figures in the original report looked modest, a 5% increase here, 8% there. But impact only tells one side of the value-for-money story. When you crunch the numbers, the return was strong, and the reason is scale and cost. At the scale of the BRAC project, hundreds of thousands of people, the cost per person reached was very low. So small changes in behaviour translated into a lot of lives saved at low cost. You have to sit down with the numbers to actually detail the ROI calculations beyond the headline impact figures.

The cost of reaching each person falls sharply as the project matures. What should that tell a funder?

By the nature of this technology, you incur most of the cost at the beginning, and those early costs are fairly fixed. It’s like building a house and then refusing to pay the electricity bill. You’ve spent all the money, the walls are up, the lights are on, but if you won’t cover the small running cost, you never get the benefit of the house. The lesson for a funder is not to give up too early when fixed costs are large, because the expensive part is the start.

Where does a 12-to-1 return sit against other interventions you’ve evaluated?

It’s a very good ROI. Many interventions in global health simply don’t work; their return is less than 1. A typical one that does work returns about 3 to 4 to 1. So 12 to 1 is much stronger than typical, roughly the top 25%. The caveat I’d keep in mind is that this is a single study of a single programme. The fairer thing over time is to run several calculations and see the whole distribution.

If you had to sum up what this taught you about measuring value in digital health, what would you say?

There’s a lot of potential for digital health to improve people’s lives, but context really matters. A digital tool is only ever as valuable as what you’re using it for.

Read more