Policymakers face challenges when trying to identify the right targets for antipoverty programmes. This column assesses whether the data typically available to policymakers in sub-Saharan Africa are up to the task. Commonly used proxy means tests are found to perform worse than simpler methods in identifying poor households. Moreover, analyses of nutritional status reveal substantial inequality within households, suggesting that household-based measures are not very effective in identifying disadvantaged individuals.
It has often been noted that the world’s aggregate poverty gap – the total monetary gap by which poor people fall below the poverty line – is modest when one uses poverty lines typical of low-income countries. For example, Lowrie (2017) writes that “[o]ne estimate, […] recently calculated that the global poverty gap […] is roughly what Americans spend on lottery tickets every year, and it is about half of what the world spends on foreign aid.” The implication sometimes drawn is that only a modest sum of money is needed to eliminate global poverty – to bring all poor people up to the international poverty line. But is this correct?
In two new research papers we have tried to assess whether the data typically available to policymakers in sub-Saharan Africa – now the poorest region of the world by most measures—are adequate for the task of reliably identifying the poor, and targeting money to them (Brown et al. 2016, 2017). Specifically we ask:
- How well can we identify poor households with the type of data routinely used by policymakers?
- How well can we identify poor individuals using such data?
Targeting poor households
Identifying the poor is often complicated by a lack of reliable data (or imperfect information about which households are poor) and limited administrative capabilities. In attempting to address these issues, practitioners across the world have increasingly turned to some form of proxy means test to identify poor households. The idea is simple. A score is given to each household in the relevant population based on a (typically small) set of readily observable household characteristics. The weights on these characteristics are given by the regression coefficients for household consumption or income as a function of those characteristics. The regression is calibrated to survey data and then used to make the out-of-sample predictions about household consumption (or income) for the population.
There has been some debate about proxy means tests in the policy-oriented literature. Supporters claim that it is more reliable than other options found in practice. Critics point to seemingly poor predictions about who is poor and who is not.
In reading the policy literature, one finds that the methodological inadequacies of standard proxy means tests are not fully appreciated by practitioners. Standard regression-based calibration of the test score will tend to work less well toward the extremes of the distribution of household consumption. By its design, a standard regression line passes through the means of the data. One can expect the method to have a tendency to overestimate living standards for the poorest (and underestimate them for the richest). This is clearly an undesirable feature of this method for targeting an antipoverty programme.
We have studied the performance of this popular method in a number of African countries (Brown et al 2016). Our results point to both strengths and weaknesses of the method. Proxy means tests can substantially reduce inclusion errors (i.e. including the non-poor) in an antipoverty programme; in most cases the inclusion error rate can be at least halved. But this comes at the cost of substantial exclusion errors (excluding the poor). And when the objective is to reduce poverty, policymakers should be more worried about exclusion errors. Better methods exist. The method we find to generally perform best is a ‘poverty-quantile regression,’ which forces the prediction model through the poverty line rather than the mean.
When judged in terms of the impact on poverty for a budget set equal to the aggregate poverty gap, we find that the most widely used form of proxy means test does only slightly better on average than a universal basic income in which everyone gets the same transfer, whatever their characteristics. Even under seemingly ideal conditions, the ‘high-tech’ solutions to the targeting problem with imperfect information do not do much better than age-old methods using state-contingent transfers or even simpler basic income schemes. We find that an especially simple demographic ‘scorecard’ based on characteristics such as age and gender, can do almost as well as the proxy means test in reducing poverty. Indeed, allowing for lags in implementing such tests (which are common in practice), the simpler categorical targeting methods perform better on average in bringing down the poverty rate.
We were surprised that proxy means tests only allow small (or even negative) gains in reaching poor people compared to simpler methods. For practitioners deciding on targeting methods going forward, we suspect that other criteria besides targeting accuracy should take precedence in the choice, such as administrative capabilities and cost, the need for transparency, incentive effects, and the scope for fine targeting to undermine political support for social policies.
Targeting poor individuals
While it is widely appreciated that poverty is an individual deprivation, household data are almost invariably used to infer individual poverty. In other words, a poor household is assumed to contain poor individuals. It is almost always the case when calculating poverty rates and formulating anti-poverty policies that all individuals within the household are assumed to have the same level of economic welfare.
Our second paper tries to throw light on how well widely used household-based measures perform in identifying disadvantaged individuals (Brown et al. 2017). For its part, the World Bank has made reaching poor families – as often identified by the poorest two quintiles of people based on household consumption per person – the main objective of its social protection operations. However, missing data on individual-level poverty present a significant hurdle to examining this issue. Individual-level consumption is not easily collected, and it is difficult to determine how income earned by individuals is shared across other household members.
One dimension of individual welfare that can be observed in many surveys is nutritional status as indicated by anthropometric measures. Undernutrition can stem from inadequate caloric intakes or deficiencies in protein or micronutrient intakes, or from illness that impedes nutritional absorption. Such nutritional deprivations are of direct and immediate concern, and there is also evidence of longer-term social and economic costs, especially low-birth weight and chronic undernutrition in childhood.
Our paper provides a comprehensive study on the relationship between household wealth and individual nutritional status for 30 countries in sub-Saharan Africa. We find a reasonably robust household-wealth effect on undernutrition indicators for women and children (that is, the incidence of undernutrition tends to fall as household wealth rises). Nonetheless, about three-quarters of underweight women and undernourished children are not found in the poorest 20% of households. This is consistent with evidence of considerable intra-household inequality. Furthermore, countries with a higher overall incidence of undernutrition tend to be those where a larger share of the undernourished are found in non-poor families.
Our findings suggest that to have any hope of reaching undernourished women and children, policy interventions in this setting will either require much more individualised information or they will need to have broad coverage (rather than policies finely targeted to poor households). This is especially so in countries with a high incidence of undernutrition.
Rather than folding nutrition objectives into household-targeted antipoverty programmes, emphasis should be given to specific nutritional interventions that target all women and children, such as comprehensive school feeding (with explicit nutrition supplementation), maternal health care, and universal sanitation services.
Conclusions
There are potentially many constraints on antipoverty policies, including incentives and political economy. In this column we have focused on what is arguably the most basic constraint – information. Policymakers need to have realistic expectations of what can be accomplished given the data actually available.
Our results do not suggest that standard data sources on poverty used by policymakers are very effective in identifying poor household or poor individuals. There is some scope for using better data and better methods, and our work has pointed to some specific options. However, the idea that we can easily eliminate poverty by finely targeted transfers that fill all the poverty gaps can be ruled out even before we start to think about the (serious) incentive problems that such a policy could generate.
References
Brown, C, M Ravallion and D van de Walle (2016) “A poor means test? Econometric targeting in Africa”, NBER, Working Paper 22919.
Brown, C, M Ravallion and D van de Walle (2017) “Are poor individuals mainly found in poor households? Evidence using nutrition data for Africa”, World Bank, Policy Research Working Paper 8001.
Lowrie, A (2017) “The future of not working”, New York Times Magazine, 23 February.