An interesting conversation took place in mid-July between Bill Easterly of NYU; Holden Karnofsky and Stephanie Wykstra of GiveWell; and an unnamed funder. Easterly and Karnofsky penned a pair of blog posts that shared some of the highlights of the conversation. It is interesting in terms of how the two sides perceived the conversation in light of their disagreement on whether or not to make recommendations based on academic research.
Easterly, who has emerged as one of the critics of the much lauded randomized control trial (RCT) explains his point of view at the start of the conversation.
As Angus Deaton has repeatedly emphasized, RCTs give an average result. Treatment effects vary a lot depending on the context. When we average over a lot of them it’s almost certain that we’re getting some negative treatment effects, even when the average is a positive and significant result. You want a safeguard against having one enormous beneficiary with everyone else losing. You want a safeguard against harming a lot of people unacceptably. Continue reading