So I just wrote this essay about all the reasons I think international development, as we currently carry it out, can never achieve its own objectives. One thing I didn’t have room for was the ideas I’m excited about, development projects that meet, at least partially, our outsized expectations of them. Here are two such ideas, and why I think projects like these—technical, slow, un-viral—are the future of development.
Payment by Results
For those of you who’ve never implemented a development project, here’s how it works: You write a proposal to a donor. They agree, in principal, to fund your idea. Then you negotiate what your ‘indicators’ will be. These are the data points the donor will use to determine if you did what you set out to do, whether your project was successful.
Let’s say you’re proposing a project in Zambia, you want to decrease malaria rates. You get the European Commission (EC) to give you $1 million to train Zambian nurses to go house to house handing out malaria treatments, training mothers on symptoms and doses.
You and the EC come up with some indicators they’ll use to evaluate your project after it’s finished: You have to provide 10 training sessions per year, they have to be attended by at least 20 nurses, and all the nurses have make 1,000 home visits within a year of the training.
These sound pretty robust, right? The donor is saying, if we’re gonna give you money, you have to spend it like you told us you would.
But look how each of those indicators is tied to the process, not the outcome. Maybe 20 nurses attended your training, but none of them worked in clinics in high-malaria regions, or they read the newspaper during the training, or they only attended because they wanted the per diems. None of those indicators are related to the thing you actually set out to do.
I’ve done projects in Sub-Saharan Africa where our indicators were the number of Facebook likes we got, how many pages our summary reports were, how many trips to the field we made. Donors send auditors to get the sign-in sheets from our events and copies of PowerPoints we gave.
It’s not just individual projects that fall into the gap between inputs and results. Lant Pritchett’s The Rebirth of Education: Schooling Ain't Learning documents how the international push for improved school attendance—as opposed to improved literacy, professional skills, and cognitive ability—led to overburdened teachers and crowded schools. According to a 2012 UNECSO study, 130 million kids—about one-quarter of the total worldwide—finished elementary school without basic literacy and math skills. In Nigeria, 52 percent of girls who finished six years of education were still illiterate, a rate that actually increased from 2002, right in the middle of the worldwide push for universal enrollment.
Payment by results does this upside-down: You’re paid for the result. How you get there is up to you.
So if the same project was administered under payment by results—sometimes called pay-for-performance aid or cash-on-delivery aid—you’d do a baseline survey before you started the project. One thousand Zambian kids, let’s say, die of malaria each year in the region where you’re carrying out your project. If, after you’re finished, that number has fallen to 500, you get the $1 million. If it doesn’t, you don’t.
As an employee of an international NGO, someone who spends a lot of time saving, scanning, and filing receipts for coffees in African airports, I love this model. I like that it gives us space to be creative. If we were being judged on our outcome rather than our process, we’d be free to pivot midstream. Maybe the training events aren’t working, and we should meet with nurses during their shifts at the hospitals. Maybe home visits aren’t reaching working mothers, so we should do village-wide weekend events. Because our indicators are related to activities rather than outcome, we couldn’t change our approach if it wasn’t working.
It also gives us the incentive to be cost-effective. If we spend $800,000 on the project and we get the $1 million grant, we can spend the surplus hiring more people, tweeting about our results, doing a fundraising drive. Under the current model, it’s the exact opposite: I have to go to the field, I have to give those training sessions; otherwise, I don’t get reimbursed.
The charity GiveDirectly has gotten a lot of attention lately for simply giving cash to poor people, no questions asked. The idea is that poor Kenyans have the best information on how poor Kenyans should spend their money, and aid agencies and Western donors should just them the means to do so and get out of the way. Payment by results is a step toward applying the same model to development charities themselves.
Not that payment by results is perfect. We could fake those improved death statistics, for one thing. Or we could spend half our budget bribing a politician to increase spending in our district, get our better death rates through graft.
In a survey of the evidence for and against payment by results (spoiler: there isn’t much), the NGO coalition Bond pointed out that this model puts all the financial risk on NGOs, and would encourage them to favor “tested” development projects rather than trying something riskier or more innovative. “One UK NGO,” the report notes, “reported having internal discussions on whether to include disabled children in the target group for an intervention funded through payment by results when their contract would not have paid them the additional cost required.” For me, it’s the untested-ness of payment by results that gives me the greatest trepidation about using it on a large scale. It sounds great, sure, but so did all the other development projects in the unmarked shed behind where they give the TED Talks.
Still, results-faking, profit maximizing, stat-juking, it’s not like those are new risks. Facebook likes aren’t exactly a perfect measure of impact, after all, and I could already be faking my sign-in sheets and my hotel receipts. The reason I don’t has nothing to do with accountability to my donor. It’s because I genuinely want my projects to succeed, not just to look like they do. I’d love it if a donor gave me the freedom to find that out for myself.
The ‘Data Revolution’
A friend of mine works at Amazon.com. Her job is to monitor the activity on the site and make tweaks, down to the millisecond, to maximize how much people buy. Thousands of people at Amazon do the same thing: This is how they know exactly which shade of yellow the “buy” button should be to make you click it. Every time we talk about work, I feel this chasm between how much she knows about her job and how much I know about mine.
Like I say in my essay, we basically have no idea what makes kids in poor countries go to school or not, why their moms vaccinate them or don’t. While Amazon is making tweaks to its business model, we’re reversing ours every time a new study gets published.
Much of the reason for this, boringly, is that we simply do not have very good data on the developing world. Statistics from poor countries are notoriously noisy and imprecise, clouded by political incentives and baseless external projections. In Madagascar, for example, a census hasn’t been carried out since 1993. The 2006 Nigerian census was mired in controversy, politicians accused of inflating numbers to increase political, ethnic and religious representation for their districts. “We do not really know our population,” the chairman of the country’s National Population Commission said at the time.
Morten Jerven’s book Poor Numbers (wonks only please) notes that many of the economic statistics you read from Sub-Saharan Africa—GDP, prices, income levels—are bold extrapolations from meager data points. The UN has price figures, for example, for 47 Sub-Saharan African countries, covering 1991 to 2004. But less than half of 1,410 observations have actual data behind them. The rest are projections, assumptions, a finger in the wind. For 15 of the countries, the UN has never received any data at all. This is how Guinea is either the seventh poorest country in Africa (out of 45) or the eleventh richest, depending on which source you’re using. Jerven compares the three main rankings of per capita GDP and sees Liberia jump 20 places.
Here’s the World Bank’s chief economist for Middle East and North Africa in 2011, calling Africa a ‘statistical tragedy’:
Today, only 35 percent of Africa’s population lives in countries that use the 1993 UN System of National Accounts; the others use earlier systems, some dating back to the 1960s. … Consider the case of Ghana, which decided to update its GDP last year to the 1993 system. When they did so, they found that their GDP was 62 percent higher than previously thought. Ghana’s per capita GDP is now over $1,000, making it a middle-income country.
Nigeria did the same thing in 2013, rebased its economic statistics, and saw its GDP jump up by 89 percent. These are the statistics development projects are based on, what they are trying to change. It’s like trying to lose 20 pounds without a mirror and stepping onto a different scale every day.
This is where “the data revolution” comes in. Amanda Glassman, a member of the Data for Africa Working Group, notes that most of the development statistics—how many people can read, who is at risk of starvation—come from household surveys, many of which are carried out by international monitoring and evaluation teams checking to see whether NGOs are spending donor money wisely (there’s those indicators again).
The problem with these surveys is that they’re not aligned between donors. So the Gates foundation team comes to a village, asks everybody their age and their weight and what they’ve been vaccinated for. Next month, Oxfam comes and asks them their height and their education level and how well they can read. The next month the World Bank comes and … you get the idea.
Meanwhile, the clinics and schools and local statistics offices, the bodies that are actually mandated to gather this kind of information, are cut out of the process. They don’t have the staff or budgets to carry out their own investigations, and none of the donors report their findings back. If Bill Gates or the World Bank or whoever finds high rates of TB, they’re not obligated to give this information to the agencies responsible for addressing it. The official line is basically “we’ll take it from here, guys.”
This is understandable in the short-term. Local agencies don’t have the staff to solve large-scale problems, and they might be undertrained, corrupt, or flapping in the political winds. In this Development Drums podcast, the interviewees mention that statistics offices get calls from politicians, ordering them to make the numbers look like they’re improving.
But in the medium- and long-term, it means that African authorities stay under-resourced, de-capacitated, out of the loop. Replacement of local authorities by international NGOs might even be partly responsible for Liberia and Sierra Leone’s slow response to the Ebola outbreak: After decades of being bypassed by international health charities, local public health services didn’t know about, and weren’t able to respond to, conditions in their own country. When international foundations come in with their own statistical programs and skip the local authorities, the locals are cut out of information about their own countries.
But now we know about this problem! Since Jerven’s book, the dearth of development data has gone from obscure and insoluble to urgent and achievable. The Data for Africa Working Group report identifies efforts by the Gates Foundation, USAID, Rockefeller Foundation, UN, and World Bank to improve statistical agencies all over Africa.
Like the last idea, this one also warrants some skepticism. This is not the first time rich countries have sent experts to poor countries with the aim of improving their institutions. Technical assistance, as they call it in development lingo, has been found to be one of the least effective forms of aid.
But the potential of a more unified approach to gathering data is profound. Statistics are how development agencies diagnose problems and identify effective solutions. “Big data” gets overhyped these days, and real-time data collection in rural areas sounds like exactly the kind of development gimmick that will become the next One Laptop Per Child. But when it comes to the basic numbers—population, economics, living conditions, many parts of Africa would benefit from any improvement at all.
Neither of these ideas is all that sexy, they’re not going to get shared on UpWorthy or make you reach into your pocket for your PayPal password or whatever. They’re methodological, technical, there’s no fancy new technology or smiling celebrity at their prow. We’re not talking game-changers here, more like game-slightly-tweakers.
But the more I look at development, the more I think the age of the game-changer is over. Sixty percent of the world’s poor live in middle-income countries; only 14 percent of them are in fragile of conflict-prone ones. The countries still getting aid are getting less and less of it. Charles Kenny, who wrote an entire book about how much better the developing world is now than it used to be, points out that in the 1990s, 40 percent of aid-receiving countries relied on donations for more than one-tenth of their budgets. Now, that’s below 30 percent, and dropping.
Not that we should ignore the Afghanistans and Burundis of the world, but by 2030, up to 41 countries are going to move into the middle-income bracket. Increasingly, their challenge, as ours, will be the distribution of resources, not the creation of them. The development technologies of the future aren’t going to be boreholes and school buildings. They’re going to be labor inspectors, census bureaus, government administrators, state pensions: All the boring stuff that makes our own countries function.
So yeah, that’s why I like these ideas. One of them says, either help us or go home. The other says, if you’re going to be here, know the problem and whether what you’re doing is solving it.
In 1998, Amartya Sen won a Nobel Prize, in part, for showing that a famine had never occurred in a functioning democracy. It’s never that there isn’t enough food to go around; it’s that authoritarian governments don’t set up the mechanisms to provide it, at a decent price, where it’s needed.
The more I learn about development, the more I think the same principle applies to prosperity itself.