Bill Gates issued his annual letter for 2013 today, in which he makes the case for measurement as a critical tool to fighting poverty, disease and inequality.
The call for better measurement and evaluation within the aid and development community is popular, but hardly new. And, as we noted earlier in the week when Gates spilled the beans on what he was going to say this year, it’s one thing to measure something and quite another to be certain you’re looking at the right variables, getting meaningful numbers and coming up with an answer that actually provides you with a useful new course of action.
You know, this guy sees a drunk guy crawling around under a light looking for his car keys. He asks the drunk where exactly he thinks he dropped the keys. “Over there in that dark alley,” replies the inebriate. So the guy asks, why look here? Drunk guy: “The light is better.”
“…” said Raikes, blankly looking like he thought agreeing to this interview maybe wasn’t such a good idea.
My point was that measurement and evaluation are fine, but as Albert Einstein said: “Some things that are worth doing can’t be measured; And some things that can be measured aren’t worth doing.”
The eradication of polio is a top priority right now for the Gates Foundation, as Gates notes in his 2013 letter and has said many times over the years — saying so again in a speech this week in London. He thinks it can be done by 2018, a fairly bold prediction because it’s been made by others so many times over and has, so far, never happened. Said Gates:
“The number of global polio cases has been under 1,000 cases for the last two years, but getting rid of the very last few cases is the hardest part.”
Yeah, so how can measurement help?
How hard can it be to measure less than 1,000 cases of polio? How will measuring these miniscule number of cases (as compared to many other diseases, with millions of cases per year), accomplish anything much?
Is the Gates Foundation acting like the drunk under the street lamp — avoiding dark alleys like the political fuss over the CIA screwing up the polio campaign, or the debate over which vaccine to use — and basking in the delusional light of numeracy? (I didn’t actually ask it that way).
“Let’s look at Nigeria,” said Raikes, patiently. Nigeria is one of the three nations where polio stubbornly persists despite a lot of effort by Nigerian officials and health workers to stamp out the infectious disease. “If you look at the number of polio campaigns that are done each year in Nigeria. you would assume you are getting 95% plus immunity.”
“But if we had been achieving this, we wouldn’t have polio,” Raikes said. “So why do we still have polio in Nigeria? Because there’s something about the way the campaigns have been done that causes us to continually miss the same kids.”
A few years ago, he said, the foundation decided to support a project in Nigeria aimed at figuring out what was going on, why kids were being missed on the mass vaccination drives and why polio kept popping up in communities, mostly in northern Nigeria, where they had repeatedly, feverishly been doing immunizations.
It was a mapping project, using geographic information systems (GIS) and also the satellite-based Global Positioning System (GPS) to analyze the movements of the health workers, the topography and municipal boundaries along with the reported cases of polio in children.
“Look at that, notice anything?” said Raikes, pointing to a map of northern Nigeria with red dots signifying polio cases. “Most of the missed kids, the polio cases, are here on the borders between the districts or states.”
What that tells us, he said, is that the solution to improving coverage resides in better coordination between health districts. “It’s a clear example of how you can use innovations and measurement to improve the efficiency of these programs.”
So measurement and evaluation can sometimes reveal surprising new insights even into what may appear the most mundane or deceptively simple of tasks, Raikes said. But he agreed that getting good information at the baseline is still critical to success and often a problem.
“I was in the Kano (Nigeria) last November and the lady working at the clinic told me they had 158 percent coverage,” Raikes said. (For you non-numerical types, that’s not statistically meaningful). But buried in the bad statistics, he said, was useful information so it would have been wrong to not dig in a bit deeper.
Setting targets and measuring progress is critical to success at every level in the fight against poverty and disease, Raikes said, and not evaluating, trusting in good intentions, is no longer acceptable to most donors.
But simplicity is also key, he said. The reason the much-celebrated 8 Millenniusm Development Goals (MDGs) have achieved as much as they have over the past decade, Raikes said, can be largely attributed to the fact that they were simple and, most of them, relatively easy to measure.
Given the major progress made on many of the MDGs and heading toward the targeted completion date of 2015, almost every organization appears to want to get their cause on the line-up for the next set of international goals. Fighting 8 goals focused on poverty and disease is no longer good enough. Many want the next set of MDGs to emphasize ‘sustainability’ (whatever that means) and human rights, political change and so on.
“Because the MDGs have been so successful, we now have the challenge that everybody wants to add their pet thing to the agenda,” Raikes said. That might sound equitable and fair, but it also puts the entire enterprise at risk.
“The beauty of the MDGs were largely because they were so simple, and measurable.” Raikes said.