An interesting conversation took place in mid-July between Bill Easterly of NYU; Holden Karnofsky and Stephanie Wykstra of GiveWell; and an unnamed funder. Easterly and Karnofsky penned a pair of blog posts that shared some of the highlights of the conversation. It is interesting in terms of how the two sides perceived the conversation in light of their disagreement on whether or not to make recommendations based on academic research.
Easterly, who has emerged as one of the critics of the much lauded randomized control trial (RCT) explains his point of view at the start of the conversation.
As Angus Deaton has repeatedly emphasized, RCTs give an average result. Treatment effects vary a lot depending on the context. When we average over a lot of them it’s almost certain that we’re getting some negative treatment effects, even when the average is a positive and significant result. You want a safeguard against having one enormous beneficiary with everyone else losing. You want a safeguard against harming a lot of people unacceptably.
But researchers don’t want their job to be more difficult than it is. If you ask for not only a RCT but also a guarantee that it’s not concealing unacceptable harm, you’re making it harder, and RCTs are already expensive and hard to begin with. It’s inconvenient for the researchers to acknowledge these problems.
Development happens when people have the opportunity to choose what they want, choose whether or not to give consent for an intervention that affects them, protest if they don’t like what’s being done to them and have a mechanism to exit if they don’t like what’s being done.
(snip)
There are a lot of thing that people think will benefit poor people (such as improved cookstoves to reduce indoor smoke, deworming drugs, bed nets and water purification tablets) that poor people are unwilling to buy for even a few pennies. The philanthropy community’s answer to this is “we have to give them away for free because otherwise the take-up rates will drop.” The philosophy behind this is that poor people are irrational. That could be the right answer, but I think that we should do more research on the topic. Another explanation is that the people do know what they’re doing and that they rationally do not want what aid givers are offering. This is a message that people in the aid world are not getting. The rational choice paradigm has never been fully accepted in the development community. We should try harder to figure out why people don’t buy health goods, instead of jumping to the conclusion that they are irrational.
The GiveWell pair disagreed with this assessment .
We have some sympathy with this view and agree that more evidence would be welcome, but we are probably less hesitant than Prof. Easterly is to conclude that people simply undervalue things like insecticide-treated nets.
Brett Keller observes that irrationality about one’s health is common in the developed world. In the developing world, there are substantial additional obstacles to properly valuing medical interventions such as lack of the education and access necessary to even review the evidence. The effects of something like bednets (estimated at one child death averted for every ~200 children protected) aren’t necessarily easy for recipients to notice or quantify.
We’ve previously published some additional reasons to provide proven health interventions rather than taking households’ choices as the final word on what’s best for them.
Easterly also expresses concern about the harm that can be caused by aid. He suggests that promoting individual rights may indirectly lead to positive outcomes that exceed the benefits of individual interventions. This is a place where the pair disagree almost entirely saying, “We don’t believe in a “first, do no harm” rule for aid. Instead, we try to maximize “expected good accomplished.” It is easy to overestimate benefits and underestimate possible harms, and we try to be highly attentive to this issue, but we believe that it isn’t practical to eliminate all risks of doing harm, and putting too high a priority on “avoiding harm” would cause aid to do less good overall.”
It is an interesting point of view. I have tended to lean towards the side of ‘do no harm’ when it comes to aid, but the pair make a valid argument in saying that all choices are not so black and white. It also provides an insight into how GiveWell evaluates programs. Organizations that distribute bednets sit high in GiveWell recommendations and it appears to be because they provide the maximum possible outcome at the lowest price.
The core difference in the argument is how Easterly and GiveWell see their roles in aid. Easterly, as the title of his blog post suggests, fancies himself a dissident explaining, “Even if we dissidents were wrong, it would still be important that people like us challenge the mainstream consensus to make them rethink what they’re doing.” On the other hand, GiveWell exists to provide advice. They take in the information that is available, analyze it and make recommendations based on what they believe to be the best course of action.
While the disagreement is important and interesting, it is a sign of a tougher look at the efficacy of aid programs. Even though GiveWell and Easterly diverge on what to do with the information at hand, both are making it a goal to evaluate long held interventions. Dean Karlan and Caroline Fiennes made the same point in the Stanford Social Innovation Review earlier this month writing, “The current rows are therefore a sign that international development is moving beyond “just knowing because I saw it with my own eyes” into properly understanding what works. We need more and better data to enable more quality debates on many subjects about development—debates that get settled, not by personalities or popularity or politics, but by the evidence.” Further to the point, these debates are taking place in a public space. Social media has become a powerful tool for the debates to take place, for people to participate and for them to be shared more broadly (as I am doing here).
