The Aid Effectiveness Literature: Is It Over Yet?

Author: Maria Perrotta Berlin, SITE.

After several decades of studies, the academic community still does not have an answer to whether foreign aid affects growth, and in which direction. Part of the reason for such an outcome may lie in a wide variety of models, techniques and data used. However, the main reason is probably that the broad spectrum of effects is difficult to disentangle when looking at the question at an aggregated level.

The aid effectiveness literature (AEL), a genre of studies using cross-country panel regressions to estimate the impact of aggregate development assistance flows on economic growth in the recipient country, is large and mostly inconclusive. The results vary widely in size and sign, and have often been proven not robust or reversed by new estimations. In short, no robust evidence has been found for aid contributing to development, at least not at the macro level.

Still new studies and in particular working papers keep being published. A plot of the number of top-journal publications and working papers over time (Figure 1) reveals an upward trend that continues unabated without any sign of decline.

So a natural question is why is this literature so big and still growing?

The question

Part of the reason the AEL keeps expanding resides in the (perceived) importance of the question. Aid flows from DAC donor countries (OECD’s Development Assistance Committee, which represents a very large share of global aid) totaled USD 135 billion in 2013, the highest level ever. Aid is also an important component of recipient countries’ budget. In many cases, it can exceed half of GDP. For example, annual foreign aid has accounted for approximately 64% of Liberia’s GDP on average since 2010. These magnitudes imply that the question on whether to continue disbursing (accepting) aid is a first order policy question for donors’ (recipients’) governments.
Figure 1. Trends in AEL publications

Source: From Google scholar

Many answers

Even limiting ourselves to the most recent reviews of the field, we can observe a broad variety of conclusions. In the popular book “The Great Escape”, Princeton’s Angus Deaton dismisses the idea that aid might contribute to growth as an illusion, and one that in practice hinders improvements in the lives of the poor (Deaton, 2013). However in his review of the book, Georgetown’s (and former World Bank) Martin Ravallion counters that instead “an objective assessment of the evidence (…) suggests a more positive role for aid in the developing world’s ongoing efforts to escape poverty” (Ravallion, 2014).

Two renowned aid-research teams also disagree on their view of the consensus. The German team (Nowak-Lehmann et al., 2012) concludes their review stating that “aid has an insignificant or minute negative significant impact on per capita income“, while the Scandinavian (UNU-WIDER) team’s review “suggests a positive and statistically significant long-run effect of aid on income” (Lof et al., 2014) and finds in particular that “the macro evidence in the recent (post-2008) literature is more supportive of positive impacts” (Arndt et al., 2014). This last conclusion is in turn challenged in a meta-analysis study (Doucouliagos et al., 2014), which concludes that “the improvement is an artifact (of publication bias). […] The average effect of aid on growth is trivially small“.

The Pitfalls and the Methods

The reason for this lack of consensus is ultimately that, in the words of Michael Clemens and coauthors (2012), “the aid-growth literature does not currently possess a strong and patently valid instrumental variable with which to reliably test the hypothesis that aid strictly causes growth.”

Aid is to a large extent allocated to low performing countries, making low growth associated with higher aid quantities on average. This simple remark makes the causal link from aid to GDP growth impossible to establish by looking at simple partial correlations between these two quantities. At the same time, an experimental approach allocating aid disbursements on a random basis is probably unthinkable, not the least for ethical reasons. This leaves researchers with the need to use econometric techniques that allow for an isolation of the causal effect of aid.

The current modus operandi to address the aid causality issue is the instrumental variable approach. However, as both data availability and quality, and empirical methods improve over time, ever-new takes at answering the question on aid effectiveness are attempted. A few recent alternative approaches are worth mentioning as very compelling. Brückner (2013) estimates separately the effect of growth on aid (getting at the reverse causality) and then removes it from the growth equation. The already cited study by Clemens and coauthors (2012) treats (lagged) aid as exogenous, and being predetermined with respect to future shocks in growth, after controlling for country fixed effects. Finally Galiani and coauthors (2014) rely on a quasi-experiment: the discontinuity around an income threshold used by the World Bank to allocate its concessional lending. All of these studies find positive effects of aid.

Our Experience and the Problem of Choice

In Frot et al. (2010), we also find positive effects of comparable size, using a new instrumental variable. We argue that it is a significant improvement relative to past IV approaches. Recent reassessment of the empirical analysis revealed, though, that our results are not robust.

Here are some of the sources of the variability in our results, and presumably of the results across different studies:

• Type of aid flows. Many types of aid flows are registered and can potentially be chosen as object of analysis: loans and grants, commitments and disbursements, tied and in kind, more or less programmable by the recipient, including or excluding debt service, emergency relief, and so on.
• Sources of aid flows and aggregation. Most flows are voluntarily reported by the donors, at different aggregation levels (from project or sector level and up or directly as gross totals). The multiplicity and voluntary character of reports creates incongruities and more generally measurement errors in the variables used. Moreover, this compounds with the well-known measurement error issues in GDP, increasing the noise.
• Econometric model. Given the size of the literature, a very large number of estimators, control sets and specifications have been used over time.
• Lag structure. When aid inflows can be expected to produce a measurable effect on economic growth is not clear a priori, and obviously depends among other things on the type of flow, its intended purpose, and its actual destination.

Theory offers little guidance over this broad set of choices. It is therefore hard to establish in advance what is the most suitable empirical strategy in all these details, exposing the researcher to the risk of confirmation bias (preference accorded to the results that one expects to find, either positive or negative).

A recent working paper holding a similar view on the AEL as the one expressed in this brief is Edwards (2014), also summarized in Edwards (2014b): the interested reader can find a more detailed historical perspective and rigorous description of the methodological and technical issues.

Conclusion

The question on the effectiveness of development aid is academically interesting and policy-relevant. The formulation of the problem in terms of aggregate measures of aid and aggregate outcomes such as growth is however argued to make a serious empirical investigation unnecessarily hard. This is probably the reason why contributions keep accumulating but without no definitive answer being offered. Disaggregated components of aid flows, or specific aid programs and interventions, as well as particular, well-defined outcomes would constitute a more suitable object of analysis, for a number of reasons (as argued in Qian, 2014). A more recent literature is therefore doing just that, although this approach is not devoid of shortcomings either.

Probably the “big question” about what happened to all the foreign aid disbursed for decades will keep titillating the investigative minds, and the easily accessible data and powerful econometric software will keep producing new contributions for a while still. It is useful to keep in mind that absence of evidence is not evidence of absence.

▪

References

• Arndt, C., S. Jones, and F. Tarp (2014). Assessing foreign aid’s long run contribution to growth and development. World Development.
• Brückner, M. (2013). On the simultaneity problem in the aid and growth debate. Journal of Applied Econometrics 28(1), 126–150.
• Clemens, M. A., S. Radelet, R. R. Bhavnani, and S. Bazzi (2012). Counting chick- ens when they hatch: Timing and the effects of aid on growth*. The Economic Journal 122(561), 590–617.
• Deaton, A. (2013). The great escape: Health, wealth, and the origins of inequality. Princeton University Press.
• Doucouliagos, H. and M. Paldam (2014). Finally a breakthrough? the recent rise in the size of the estimates of aid effectiveness. Working paper.
• Edwards, S. (2014). Economic Development and the Effectiveness of Foreign Aid: A Historical Perspective, NBER Working Paper No. 20685
• Edwards, S. (2014, b). Economic Development and the Effectiveness of Foreign Aid: A Historical Perspective, VOXEu Policy Brief available online at http://www.voxeu.org/article/development-and-foreign-aid-historical-perspective
• Frot, E. and M. Perrotta (2010). Aid Effectiveness: New Instrument, New Results? Working Paper No. 11, SITE.
• Galiani, S., S. Knack, L. C. Xu, and B. Zou (2014). The effect of aid on growth: Evi- dence from aquasi-experiment. World Bank Policy Research Working Paper (6865).
• Lof, M., T. J. Mekasha, and F. Tarp (2014). Aid and income: Another time-series perspective. World Development.
• Nowak-Lehmann, F., A. Dreher, D. Herzer, S. Klasen, and I. Martínez-Zarzoso (2012). Does foreign aid really raise per capita income? a time series perspective. Canadian Journal of Economics/Revue canadienne d’économique 45 (1), 288–313.
• Qian, N. (2014). Making progress on foreign aid. NBER Working Paper No. 20412
• Ravallion, M. (2014). On the role of aid in the great escape. Review of Income and Wealth, 60: 967–984.