Tag: Randomized field experiments

Managed Competition in Health Insurance Systems in Central and Eastern Europe

Posted on October 28, 2013 | Policy Brief

This policy brief summarizes common trends in the development of health care systems in the Czech Republic, Slovakia, and Russia in late 1990s–early 2000s. These countries focused on regulated competition between multiple health insurance companies. However, excessive regulation led to various deficiencies of the model. In particular, improvements in such quality indicators of the three health care systems as infant and under-five mortality are unrelated to the presence of multiple insurers or insurer competition.

A number of transition countries in Central and Eastern Europe and the former Soviet Union introduced health care systems with compulsory enrollment, obligatory insurance contributions unrelated to need and coverage according to a specified package of medical services. This so-called social health insurance (SHI) model (Culyer, 2005) is regarded as a means for achieving universal coverage, stable financial revenues, and consumer equity (Balabanova et al. 2012; Gordeev et al., 2011; Zweifel and Breyer, 2006; Preker et al., 2002). While most transition countries chose to only have a single health insurance provider on the market, the Czech Republic, Slovakia, and Russia allowed competitive (and often private) insurers in the new system. However, the evidence from the three countries shows excessive regulation of health insurers and limited instruments for insurer competition within indebted post-reform health care systems (Naigovzina and Filatov, 2010; Besstremyannaya, 2009; Medved et al., 2005). Consequently, the three countries may have been over-enthusiastic in putting large emphasis on market forces in the reorganization of health care systems in economies with a legacy of central planning (Diamond, 2002).

This brief addresses the results of Besstremyannaya (2010), which assesses the impact of private health insurance companies on the quality of health care system. While various performance measures reflect different goals of national and regional health care systems (Joumard et al., 2010; Propper and Wilson, 2006; OECD, 2004; WHO, 2000), aggregate health outcomes directly related to the quality of health care are commonly infant and under-five mortality (Lawson et al., 2012; Gottret and Schieber, 2006; Wagstaff and Claeson, 2004; Filmer and Pritchett, 1999). Consequently, Besstremyannaya’s (2010) analysis regards mortality indicators as variables reflecting the overall quality of health care system.

The estimations employ data on Russian regions in 2000-2006. The results indicate that regions with only private health insurers have lower infant and under-five mortality. However, given the low degree of competition on the social health insurance market in Russia, we hypothesize that this effect is mostly driven by positive institutional reforms in those regions. Indeed, incorporating the effect of institutional financial environment, we find that the impact of private health insurers becomes insignificant.

Development of a Social Health Insurance Model in the Czech Republic, Slovakia, and Russia

At the beginning of their economic transition, the Czech Republic, Slovakia, and Russia established a model for universal coverage of citizens by mandatory health insurance (Balabanova et al., 2012; Medved et al., 2005; Sheiman, 1991). The revenues of the new SHI system came from a special payroll tax and from government payments for health care provision to the non-working population. The main reason for combining certain features of taxation-based and insurance-based systems was the desire to establish mandatory health insurance as a reliable source of financing in an environment with unstable budgetary revenues (Lawson and Nemec, 2003; Preker et al., 2002; Sheiman, 1994). The insurance systems instituted in the three transition countries correspond to the major SHI principles implemented in Western Europe: contributions by beneficiaries according to their ability to pay; transparency in the flow of funds; and free access to care based on clinical need (Jacobs and Goddard, 2002).

The Czech Republic, Slovakia, and Russia placed emphasis on regulated competition, decreeing that SHI should be offered by multiple private insurance companies with a free choice of the insurer by consumers. Managers of private insurance companies were assumed to perform better than government executives (Lawson and Nemec, 2003; Sinuraya, 2000; Curtis et al., 1995), so an intermediary role for private insurance companies was seen as a key instrument for introducing market incentives and improving the quality of the health care system (Sheiman, 1991).

However, the activity of health insurance companies in the three countries was heavily regulated, since the content of benefit packages, size of subscriber contributions, and the methods of provider reimbursement were decided by government, and tariffs for health care were frequently revised (Lawson et al., 2012; Rokosova et al., 2005; Zaborovskaya et al., 2005; Praznovcova et al., 2003; Hussey and Anderson, 2003). In particular, Russian health care authorities enforced rigid assignments of areas, whose residents were to be served by a particular health insurance company (Twigg, 1999) and imposed informal agreements with health insurance companies to finance providers regardless of the quality and quantity of the health care (Blam and Kovalev, 2006). As a result, the three countries experienced an initial emergence of a large number of health insurance companies, followed by mergers between them, resulting in high market concentration (Sergeeva, 2006; Zaborovskaya et al., 2005; Medved et al., 2005).

In Russia, the Health Insurance Law (1991) specified that until private insurers appeared in a region, the regional SHI fund or its branches could play the role of insurance companies. Therefore, several types of SHI systems emerged in Russian regions in the 1990s and early 2000s: the regional SHI fund might be the only agent on the SHI market; the regional SHI fund might have branches, acting as insurance companies; SHI might be offered exclusively by private insurance companies; or SHI might be offered by both private insurance companies and branches of the regional SHI fund (Figure 1). The variety of SHI systems reflects the fact that many regions opposed market entry by private insurance companies (Twigg, 1999). Indeed, the boards of directors of regional SHI funds usually included regional government officials (Tompson, 2007; Tragakes and Lessof, 2003) who were reluctant to reduce government control over SHI financing sources (Blam and Kovalev, 2006; Twigg, 2001). The controversy with health insurance legislation created a substantial confusion at the regional and the municipal level (Danishevski et al., 2006).

Figure 1. Health insurance agents in Russia in 2000-2006, (number of regions)

This context suggests that Russian regions provide an interesting study field to address the impact of private health insurance companies on the quality of health care system. In particular, the wide variety of SHI systems across Russian regions, as well as the gradual introduction of the health insurance model in Russia provide a sufficient degree of variation in practices and outcomes to allow for a well-specified empirical analysis.

Data and Results

In our analysis we use data on Russian regional economies between 2000 and 2006 (as based on data availability). Our measures of health outcomes are given by the pooled regional data on infant and under-five mortality. Our key explanatory variable is the presence of only private health insurers in the region. Arguably, the coexistence of public and private health insurance companies does not enable effective functioning of private health insurers owing to their discrimination by the territorial health insurance fund. Therefore, in the empirical estimations we focus on the presence of only private health insurers in the region, regarding it as a measure of effective health insurance model. The analysis also employs a variety of important socio-economic and geographic variables influencing health outcomes (per capita gross regional product (GRP), share of private and public health care expenditure in gross regional product, share of urban population, average temperature in January).

The results of the first set of our empirical estimations demonstrate that the presence of only private health insurers in a region leads to lower infant and under-five mortality. Furthermore, an increase in the share of private health care expenditure in GRP leads to a decrease in both mortality indicators. The result is consistent with numerous findings about the association between personal income and health status in Russia (Balabanova et al., 2012; Sparling, 2008).

Prospective reimbursement of health care providers is associated with a decrease in infant and under-five mortality. The finding suggests the existence of a quasi-insurance mechanism in the Russian SHI market. Operating in an institutional environment where provider reimbursement is based on prospective payment, private insurance companies in effect shift a part of their risk to providers (Glied, 2000; Sheiman, 1997; Chernichovsky et al., 1996).

Table 1. Factors leading to decreased infant and under-five mortality in Russia

Notes: * indicates that the coefficient is statistically significant in a parametric regression

Although our analysis shows that the presence of only private health insurers is statistically associated with improvements in infant and under-five mortality, we believe that the influence is indirect. Namely, the overall positive institutional environment in the region may result in both a decrease of mortality indicators and a lower coercion of regional authorities towards the presence of private health insurance companies.

To test this hypothesis, we use financial risk in a region as a measure of institutional environment and incorporate it in the analysis through an instrumental variable approach. (We measure financial risk by an expertly determined rank ordered variable by RA expert rating agency; this variable reflects the balance of the budgets of enterprises and governments in the region, with lower ranks corresponding to smaller risk.)

In line with our hypothesis, the results suggest that the presence of private health insurance companies now becomes insignificant in explaining infant and under-five mortality.

Discussion

The existing literature suggests that the improvement in infant and under-five mortality in the Czech Republic, Slovakia, and Russia can be attributed primarily to an increase of health care spending (Gordeev et al. 2011; Besstremyannaya, 2009; Lawson and Nemec, 2003) rather than being an effect of the social health insurance model with multiple competing insurers. It should be noted that insufficient government payments for the non-working population and a decline of the gross domestic product in the early transition years left SHI systems in the three countries indebted (Naigovzina and Filatov, 2010; Sheiman, 2006; Medved et al., 2005), which undermined the development of the managed competition in the health care provision.

In Russia (and also in the Czech Republic and Slovakia) there is little competition between insurers, and surveys show that the main factors causing consumers to change their health insurance company are change of work or residence, and not dissatisfaction with the insurer (Baranov and Sklyar, 2009). The fact that law suits on defense of SHI patient rights are rarely submitted to courts through health insurers (Federal Mandatory Health Insurance Fund, 2005) may also be evidence of the failure of Russian health insurance companies to win customers on the basis of their competitive strengths.

Summary and Policy Implications

The above findings as well as the other mentioned literature suggest that improvements of infant and under-five mortality in the Czech Republic, Slovakia, and Russia are not associated with the positive role of managed competition in the social health insurance system. In particular, in Russia the decrease in infant and under-five mortality is likely to be related to financial environment, rather than the existence of insurance mechanisms or competition between health insurance companies. One possible explanation of this absence of effect may come from the excessive regulation of the private insurance markets, as well as the insufficient competition between insurers. Importantly, the health insurance reform, implemented in Russia in 2010, both addressed underfinancing (by raising payroll tax rates) and took a step towards fostering provider competition, by allowing private providers to enter the social health insurance market (Besstremyannaya 2013). However, insurance companies are still not endowed with effective instruments for encouraging quality by providers, which may greatly undermine their efficiency.

▪

References

Balabanova D, Roberts B, Richardson E, Haerpfer C, McKee V. 2012. Health Care Reform in the Former Soviet Union: Beyond the Transition. Health Services Research 47(2): 840-864.
Baranov IN, Sklyar TM. 2009. Problemy strakhovoi modeli zdravookhraneniya na primere Moskwy i Sankt-Peterburga (Problems of insurance model in health care: the example of Moscow and Saint Petersburg). In X International Conference on the Problems of Development of Economy and Society, Yasin E.G (ed), Moscow: Higher School of Economics, vol.2.
Besstremyannaya GE. 2013. Razvitie systemy obyazatelnogo meditsinskogo strakhovaniya v Rossijskoi Federatsii (Development of the Mandatory Health Insurance system in the Russian Federation) Federalizm 3: 201-212
Besstremyannaya GE. 2010. Essays in Empirical Health Economics. PhD thesis. Keio University (Tokyo).
Besstremyannaya GE. 2009. Increased public financing and health care outcomes in Russia. Transition Studies Review 16: 723-734.
Blam I, Kovalev S. 2006. Spontaneous commercialization, inequality and the contradictions of the mandatory medical insurance in transitional Russia. Journal of International Development 18: 407–423.
Culyer AJ (2005) The Dictionary of Health Economics, Edward Elgar.
Danishevski K, Balabanova D, McKee M, Atkinson S. 2006. The fragmentary federation: experiences with the decentralized health system in Russia. Health Policy and Planning 21: 183–194.
Gordeev VS, Pavlova M, Groot W. 2011. Two decades of reforms. Appraisal of the financial reforms in the Russian public healthcare sector. Health Policy 102(2-3): 270-277.
Hussey P, Anderson GF. 2003. A comparison of single- and multi-payer health insurance systems and options for reform. Health Policy 66: 215-228.
Jacobs R, Goddard M. 2002. Trade-offs in social health insurance systems. International Jthenal of Social Economics 29(11): 861-875.
Lawson C, Nemec J, Sagat V. 2012. Health care reforms in the Slovak and Czech Republics 1989-2011: the same or different tracks? Ekonomie a management 1, 19-33.
Lawson C, Nemec J. 2003. The political economy of Slovak and Czech health policy: 1989-2000. International Political Science Review 24(2): 219-235.
Medved J, Nemec J, Vitek L. 2005. Social health insurance and its failures in the Czech Republic and Slovakia: the role of the state. Prague Economic Papers 1:64-81.
Praznovcova L, Suchopar J, Wertheimer AI. 2003. Drug policy in the Czech Republic. Jthenal of Pharmaceutical Finance, Economics and Policy 12(1): 55-75.
Preker AS, Jakab M, Schneider M. 2002. Health financing reforms in Central and Eastern Europe and the former Soviet Union, in Funding Health Care: Options for Europe, Mossalos E., Dixon A., Figueras J., Kutzin J. (Eds.), European Observatory on Health Care Systems Series: Open University Press, 2002.
Rokosova M, Hava P, Schreyogg J, Busse R. 2005. Health care systems in transition: Czech Republic. Copenhagen, WHO Regional Office for Europe on behalf of the European Observatory on Health Systems and Policies.
Sheiman I. 1991. Health care reform in the Russian Federation. Health Policy 19: 45–54.
Sheiman I. 2006. O tak nazyvaemoi konkurentnoi modeli obyazatelnogo meditsinskogo strahovaniya (On so-called competitive model of mandatory health insurance). Menedzher Zdravoohraneniya 1: 52-58.
Sheiman I. 1997. From Beveridge to Bismarck: Health Financing in the Russian Federation’. In Innovations in Health Care Financing, Schieber G. (ed.), Discussion Paper 365, 1997, Washington DC: The World Bank.
Sinuraya T. 2000. Decentralization of the health care system and territorial medical insurance coverage in Russia: friend or foe? European Jthenal of Health Law 7:15–27.
Sparling AS. 2008. Income, drug, and health: evidence from Russian elderly women. PhD dissertation. University North Carolina at Chapel Hill, UMI Dissertations Publishing.
Tompson W. 2007. Healthcare reform in Russia: problems and perspectives. Working Papers 538, OECD Economics Department
Tragakes E, Lessof S. 2003.Russian Federation, Health Care Systems in Transition, The European Observatory, WHO, Europe.
Twigg J. 1999. Obligatory medical insurance in Russia: the participants’ perspective. Social Science and Medicine 49: 371–382.
Twigg, JL. 2001. Russian healthcare reform at the regional level: status and impact. Post-Soviet Geography and Economics 42: 202–219.
Zaborovskaya AS, Chernets VA, Shishkin SV. 2005. Organizatsiya upravleniya i finansirovaniya zdravoohraneniyem v subjektah Rossijskoi Federatsii v 2004 godu (Organization of management and finance of healthcare in Russian regions in 2004)
Zweifel P, Breyer F. The economics of social health insurance. In The Elgar Companion to Health Economics, Jones A. (ed.), Edward Elgar, 2006.
Wagstaff A. 2010. Social health insurance reexamined. Health Economics 19: 503–517.

Accountability in Russia

Posted on December, 2012 | Policy Brief

This policy brief summarizes two recent research papers that are related to obstacles to political accountability in modern Russia and potential ways to overcome these obstacles. The first paper provides a rigorous assessment of the extent of electoral fraud in Moscow city during the parliamentary elections held on December 4, 2011. Using random assignment of independent observers, we estimate the actual share of votes for the incumbent United Russia party to be at least 11 percentage points lower than the official count (35.6 percent instead of 46.5 percent). A less rigorous, but more realistic estimate is 21 percentage points. These results suggest that electoral accountability in Russia is limited. The second paper demonstrates that even in an environment with low electoral accountability and limited freedom of media, alternative accountability mechanisms may emerge. In particular, anti-corruption campaigns in social media may affect the valuation of state-controlled companies, so that market forces put a disciplining effect on the managers of SOEs. We study consequences of blog postings of a popular Russian anti-corruption blogger and shareholder activist Alexei Navalny on the stock prices of state-controlled companies. In an event-study analysis, we find a negative effect of company-related blog postings on both daily abnormal returns and within-day 5-minute returns. We use the incidence of distributed denial-of-services (DDoS) attacks to show that the effect is not driven by the endogenous timing of blog postings. We also show that there are long-term effects of certain types of posts on stock returns, trading volume, and volatility. Overall, our evidence implies that blog postings about corruption in state-controlled companies have a negative causal impact on stock performance of these companies.

To study the extent of electoral fraud we employ data from a large-scale field experiment that allows us to estimate the amount of electoral fraud in the city of Moscow during Russian parliamentary elections in December 2011. In particular, we exploit randomized assignment of independent observers to polling stations. Prior to the parliamentary elections the independent NGO Citizen Observer (Grajdanin-nabludatel) trained more than 500 volunteer observers in the city of Moscow. The observers were sent to 156 randomly selected polling stations. The polling stations were selected using a systematic sampling technique. In particular, polling stations were divided by electoral districts. Within each district, polling stations were sorted according to their official number assigned by Central Election Committee. Every 25^th polling station within electoral district starting from the 1^st was assigned for observation, resulting in a sample of 185 polling stations. The Citizen Observer’s network recruited enough observers to cover 156 of the 185 polling stations, which corresponds to 4.9 percent of the 3,164 ordinary polling stations in Moscow.[1] To make sure that this procedure does not lead to a biased sample because of some hidden periodicities we check that in the previous parliamentary elections in 2007 polling stations selected using a similar procedure were not different from other polling stations.

Comparison of the share of votes received by different parties and the turnout between polling stations with independent observers from Citizen Observer (treatment group) and without observers (control group) is presented in Figure 1. The results indicate that the presence of observers led to a decrease in the share of votes for United Russia of 10.8 percentage points and the turnout at the polling stations with observers was lower by 6.5 percentage points.

Figure 1. Vote Shares in 2011

Notes: The figure is reproduced from Enikolopov, Ruben, Vasily Korovkin, Maria Petrova, Konstantin Sonin, and Alexei Zakharov (forthcoming) “Electoral Fraud in Russian Parliamentary Elections in December 2011: Evidence from a Field Experiment.” Proceedings of the National Academy of Sciences.

The above results are likely to provide a lower bound on the extent of the electoral fraud, since the presence of observers at the polling stations did not fully prevent fraud. To provide more information on the extent of the fraud, we divide all treatment stations into three groups: those in which observers reported no serious violations (75 polling stations), those in which serious violations were reported, but the observers received the final protocol (43 polling stations), and those in which all observers were not able to get the official protocol of the vote count (38 polling stations), which happened if the observers were dismissed from the polling station or the heads of electoral commissions illegally refused to give a signed copy of the protocol.

Figure 2 shows the distribution of vote shares for United Russia at polling stations from these three groups. For observations in the control group the distribution seems to be bimodal with two peaks – one around 25 percent of votes and another one around 55 percent of votes. The distribution for the precincts with observers also has two peaks, with the first one around 25 percent of votes. Note, however, that the second mode of this distribution, around 50 percent of votes, is noticeably smaller as compared with the control group. Moreover, for the polling stations in the treatment group in which observers reported no serious violations the distribution becomes unimodal with the peak around 25 percent of votes for United Russia. Thus, the results are consistent with the following hypothesis: the distribution of vote shares for United Russia in the control group is simply a mixture of two distributions that correspond to polling stations without large electoral fraud (for which the distribution is centered around 25 percent of votes) and polling stations with substantial electoral fraud (for which the distribution is centered around 55 percent of votes). Note also that a similar pattern is observed for the distribution of turnout across three groups of precincts, but not for the distribution of vote shares for other parties.

Figure 2. Distribution of votes for United Russia

Notes: The figure is reproduced from Enikolopov, Ruben, Vasily Korovkin, Maria Petrova, Konstantin Sonin, and Alexei Zakharov (forthcoming) “Electoral Fraud in Russian Parliamentary Elections in December 2011: Evidence from a Field Experiment.” Proceedings of the National Academy of Sciences.

To assess the overall influence of the electoral fraud in Moscow on the outcome of Russian parliamentary elections, we also estimate the total number of votes that United Russia received due to electoral fraud. As both vote share of a ruling party and turnout were affected by electoral fraud, we look at the number of votes for each party as a share of registered voters in precincts with and without observers. Based on these numbers, our conservative estimate of the number of votes, which United Russia received at the ordinary precincts in Moscow due to electoral fraud, is equal to 635,000. This is a lower bound for the size of electoral fraud as it assumes that the presence of observers fully prevented any fraud, and at least anecdotal evidence suggests that it is not always the case. If we use results from the polling stations in which observers report no serious violations as an alternative estimate, the number of stolen votes increases up to 1,090,000.

The results presented above indicate that because of electoral fraud, voting does not constitute an efficient mechanism to replace those in power, and, therefore, electoral accountability in Russia does not work to discipline politicians in the office. Other means to hold politicians and public officials accountable are also limited, since traditional media is often censored and politics is generally not competitive. We ask the question whether in such environment there is any alternative ways to hold public officials accountable, and, in particular, if new media, such as blogs, can make a difference. Specifically, we study whether blog postings of a popular Russian blogger, shareholder activist, and, subsequently, one of the leaders of emerging opposition to President Putin’s regime, Alexei Navalny, have had an impact on stock performance of the companies whose wrongdoings he uncovered and made public.

First, we show that daily abnormal returns of the companies Navalny wrote about were significantly lower after Navalny’s posts about them. The results hold if we control for mentions of these companies in other types of media (business newspapers, online newspapers, and blogs) and for company-year and year-month fixed effects. In addition to looking at daily abnormal returns, we show similar results for 5-minute abnormal returns even controlling for trading-day fixed effects (see Figure 3). The magnitude of this effect is quite sizable with a daily decline of 0.5 p.p. after an average blog posting, and a daily decline of 0.9 p.p. after an important blog posting.

Figure 3. 5-minute Abnormal Returns and Navalny’s Blog Postings, Non-Trading Time (Evenings and Weekends) Excluded

We also provide evidence that the impact of blogging on stock performance is causal. Although the results described above are consistent with the negative impact of blogging, they could be explained, e.g., by selective exposure. To identify the causal effect of blog postings we use an external variable, distributed denial-of-service (DDoS) attack on a blog service, as a source of exogenous variation. During the period under study (between January 2008 and August 2011), these DDoS attacks, allegedly, were not specifically targeting the Navalny’s blog, but they affected the accessibility of the whole blog platform, and the Navalny’s blog was also affected. As a result, DDoS attacks either prevented Navalny from writing a post or prevented his readers from reading his blog, but there was no obvious reason why they might influence fundamental determinants of stock prices of the companies Navalny wrote about.

In a reduced form model, we find significant positive effect of DDoS attacks on daily abnormal returns of the companies Navalny wrote about. This effect is stronger for the companies Navalny was more focused on (the latter result holds even with DDoS attack fixed effects). Quantitatively, the effect of DDoS attack is similar to the absence of the post or to the presence of the post with no information about the company in question. We also show that though DDoS effect is increasing in Navalny’s attention to the companies he was writing about, it is not increasing in the amount of general news attention to these companies.

Finally, in addition to the short-term effects we just described, we look at the longer-term one-month effects of blog postings. We find that although there were no long-term effects of the ordinary postings, there were negative and significant long-term effects of the most important postings, as proxied by at least 5 mentions of a company in the post. In addition, during the month after a blog posting, there was a larger volatility of stock returns and a larger trading volume. It appears that the number of transactions, controlling for trading volume, was significantly larger in both the short-term and longer-term perspective. Smaller average transactions are consistent with more individual, in contrast to institutional trading, which suggest that short-run effects of blog posting are driven by attention effects, rather than provision of new information. Overall, all our results are consistent with a negative causal impact of blog postings on stock performance of state-controlled companies, and imply that potentially there is a disciplining effect on the behavior of public officials who manage these companies. Thus, our results suggest that posting in online social networks can affect the stock performance of state-controlled companies, and, as a result, can become an unusual alternative mechanism to putting additional checks on the behavior of government officials even if political competition remains limited, and traditional media remain controlled.

The report is based on two papers: Enikolopov, Ruben, Vasily Korovkin, Maria Petrova, Konstantin Sonin, and Alexei Zakharov (2012) “Electoral Fraud in Russian Parliamentary Elections in December 2011: Evidence from a Field Experiment.” Proceedings of the National Academy of Sciences, 109 (52); Enikolopov, Ruben, with Maria Petrova and Konstantin Sonin “Do Bloggers Have any Real Influence? Event Study of Blog Postings by a Russian Activist Shareholder and Blog Service DDoS Attacks,” CEPR Working Paper.

[1] The sample excludes 210 precincts that had a special status, as they were located in hospitals, military units, or pre-trial detention facilities. These polling stations were excluded from the analysis since sending observers there was not always possible, and it was not clear if these polling stations were sufficiently similar to each other to use randomization. The number of votes cast at these polling stations, however, stood at only 1.8 percent of total votes in Moscow.

Development Programs and Security in Afghanistan

Posted on November 28, 2011 | Policy Brief

20111128 Policy Brief Image Afgan man with a donkey walking down the road

This policy brief summarizes the results of recent research which studies the effect of a development program in Afghanistan on the security situation there. We use a large-scale randomized field experiment to examine the effect of the largest development program in Afghanistan on the economic wellbeing of villagers and their attitudes toward the government and the security situation. We find that implementation of the program leads to significant improvement in villagers’ economic wellbeing as well as in their attitudes towards the government. The program also leads to an improved security situation in the long run. However, these positive effects on attitudes and security are not observed in districts with high levels of initial violence.

Development programs have long been used to promote economic and political development. In recent years, however, they have assumed yet another role: they have been used to promote security in countries fighting fierce insurgencies, such as Afghanistan and Iraq. The approach contends that such projects, which are commonly used by the domestic government and allied entities to provide basic services and infrastructure, improve economic outcomes, build support for the government, and ultimately reduce violence as sympathy of the population for the insurgency wanes. The idea of using development projects as a counter-insurgency strategy is becoming more and more influential and now constitutes a major component of the new U.S. counterinsurgency doctrine (U.S. Army/Marine Corps, 2006).

The study tests whether this approach works in the context of the National Solidarity Program (NSP) in Afghanistan. NSP is the largest development program in the country and has already brought almost $1 billion in aid to more than 26,000 Afghan communities. Under the supervision of the program communities elect a council, which assumes responsibility for implementing infrastructure projects (e.g. building wells or repairing roads) that are chosen by the villagers and are funded by block grants from the NSP.

To measure the effects of the program, the study uses a field experiment conducted in 500 villages across 10 Afghan districts spanning all parts of the country except for the southern provinces, where security levels were insufficient for the study to be carried out. The experiment divided the villages randomly into two groups of the same size, one of which received the program in autumn 2007, while the other group was to receive the program four years later. Before the start of the program the villages in these two groups were virtually identical, so their comparison over the course of these years shows the effect of the program on the life of village communities. The study uses the results of the extensive survey conducted in these villages two years after the start of the program as well as military information on security incidents around the villages during this period.

Our findings indicate that NSP has a strong positive effect on people’s economic wellbeing and on their attitudes towards the Afghan government (both at the central and local level). NSP also appears to improve attitudes toward NGOs and, to some extent, coalition forces on the ground. Respondents in NSP villages have significantly more positive attitudes toward government figures at almost all levels, including district and provincial governors, central government officials, the President of Afghanistan, Members of Parliament and government judges. Magnitude of effects varies from between 8 percentage points for Members of Parliament to 4 percentage points for the national police. NSP also has a positive effect on the attitudes of villagers toward NGOs and soldiers of the International Security Assistance Force (ISAF). The results for the summary measure indicate that NSPs improve villagers’ attitudes by 13 percent of a standard deviation. However, results for the two eastern districts, which experienced high initial levels of violence, are completely different. There is no positive effect of NSP on attitudes toward any government bodies, ISAF soldiers, or NGOs, and the effect on attitudes towards many figures is, in fact, significantly negative.

The results also indicate that villagers have more positive perceptions about security in NSP villages. There is no evidence, however, that the program affects the number of security incidents around villages recorded by NATO coalition forces (ISAF) in the short run (the first 15 months after the start of the program) or the number of security incidents reported by villagers in the survey. However, NSP does reduce the probability of security incidents in the long-run. The probability that a security incident will occur in one- and ten-kilometer radius around a village is smaller in treatment villages by 2 and 4 percentage points, respectively. For a three-kilometer radius, the probability is lower by 2 percentage points, but not statistically significant. In the two eastern districts, the short-run effect is similar to the average effect, but there are no statistically significant differences between treatment and control villages in long-run effects.

Overall, the empirical evidence suggests that strategies for winning the “hearts and minds” through the provision of development projects are working, but only in relatively secure regions. The development program improves the attitudes of the civilian population toward the government and makes them more likely to think that the government is working in their best interest, which in turn makes them less likely to support the insurgents. The fact that we observe the effect on security only in the long run suggests that support for the government reduces violence mainly by reducing the number of people willing to join the insurgents, rather than by increasing the population’s willingness to share information with the government. The results also suggest that development programs can prevent the spread of violence in relatively secure regions, but they are not effective in reducing violence in regions that are already experiencing significant security problems.

Overall, the results suggest that the benefits of development programs are not limited to the provision of direct economic and social benefits. They can also contribute to long-term sustained development by preventing the spread of violent internal conflicts, which are the core problem in many developing countries.

Disclaimer: Opinions expressed in policy briefs and other publications are those of the authors; they do not necessarily reflect those of the FREE Network and its research institutes.

▪

What Do Recent Insights From Development Economics Tell Us About Foreign Aid Policy?

Posted on October 3, 2011 | Policy Brief

Policy Brief Image Representing Insights from Development Economics

The short answer is: quite a lot, but different parts of the literature offer different recommendations. The problem is that these different recommendations are partly in conflict, and that political and bureaucratic incentives may reinforce these frictions when putting aid policy into practice. It follows that reforms aiming at improving aid effectiveness have to find a way to deal with this conflict and also balance the tendency of institutional sclerosis within bureaucratic agencies against short sighted incentives of politicians.

The currently predominant field of development economics focuses on impact evaluation of different economic and social interventions. These studies are all micro-oriented, looking at the impact on the level of the individual or household, rather than at the nation as a whole. One example is evaluations of the effects of different interventions on school participation, such as conditional cash transfers, free school meals, provision of uniforms and textbooks, and de-worming. Other well-known studies have looked at educational output, moral hazard versus adverse selection on financial markets, how to best allocate bed-nets to prevent malaria, and the role of information in public goods provision and health outcomes.

What has sparked the academic interest in these types of impact evaluations is the application of a methodology well known from clinical trials and first introduced in the field of economics by labor economists, randomized field experiments. The purpose of impact evaluation is to establish the causal effect of the program at hand. Strictly speaking this requires an answer to the counterfactual question; what difference does it make for the average individual if he is part of the program or not. Since an individual cannot be both part of, and not part of, the program at the same time, an exact answer to that question cannot be reached. Instead evaluators must rely on a comparison between individuals participating in the program and those that do not, or a before and after comparison of program participants. The challenge when doing this is to avoid getting the comparison contaminated by unobservable confounding factors and selection issues. For instance, maybe only the most school motivated households are willing to sign up for conditional cash transfer programs, so a positive correlation between program involvement and school participation may all be due to a selection bias (these households would have sent their children to school anyway). In this case participation is what economists refer to as “endogenous”, individual characteristics that may impact the outcome variable may also drive participation in the program.

To get around this problem, the evaluator would want strictly “exogenous” variation in the participation in the program, i.e. individuals should not get an opportunity to self-select into participation or not. The solution to this problem is to select a group of similar individuals/households/villages and then randomize participation across these units. This creates a group of participants in the program (the “treated”, using the language of clinical studies) and a group of non-participants (the “control group”) who are not only similar in all observable aspects thought to possibly affect the outcome, but who are also not given the opportunity to self-select into the program based on unobservable characteristics. Based on this methodology, the evaluator can then estimate the causal effect of the program. Exactly how that is done varies, but in the cleanest cases simply by comparing the average outcome in the group of treated with that in the group of controls.

So what has this got to do with aid policy? A significant part of aid financing goes of course to projects to increase school participation, give the poor access to financial markets, eradicate infectious diseases, etc. Both the programs evaluated by randomization, and the randomization evaluations themselves, are often financed by aid money. The promise of the randomization literature is thus that it offers a more precise instrument to evaluate the effectiveness and efficiency of aid financed projects, and also helps aid agencies in their choice of new projects by creating a more accurate knowledge bank of what constitutes current best practices. This can be particularly helpful since aid agencies often are under fire for not being able to show what results their often generous expenditures generate. Anyone who has followed the recent aid debate in Sweden is familiar with this critique, and the methodology of randomization is often brought forward as a useful tool to help estimate and make public the impact of aid financed development projects.

Limits to Randomization

Taken to the extreme, the “randomization revolution” suggests that to maximize aid effectiveness all aid should be allocated to clearly defined projects, and only to those projects that have been shown through randomization to have had a cost-effective causal effect on some outcome included in the aid donors objective (such as the millennium development goals). Yet, most aid practitioners would be reluctant to ascribe to such a statement. Why is that? Well, as is typically the case there are many potential answers. The cynic would argue that proponents of aid are worried that a true revelation of its dismal effects would decrease its political support, and that aid agencies want to keep their relative independence to favor their own pet projects. Better evaluation techniques makes it easier for politicians and tax payers to hold aid agencies accountable to their actions, and principal-agency theory suggests that governments then should put more pressure on agencies to produce verifiable results.

There are other more benevolent reasons to be skeptical to this approach, though, and these reasons find support in the more macro oriented part of the literature. In recent papers studying cross national differences in economic growth and development almost all focus is on the role of economic and political institutions. The term “institutions” has become a bit of a catch-phrase, and it sometimes means quite different things in different papers. Typically, though, the focus lies on formal institutions or societal norms that support a competitive and open market economy and a political system with limited corruption, predictability and public legitimacy. Critical components include protection of property rights, democracy, honest and competent courts, and competition policy, but the list can be made much longer. Also this time the recent academic interest has been spurred by methodological developments that have permitted researchers to better establish a causal effect from institutions to economic development. Estimating cleanly the effect of institutions on the level or growth rate of GDP is complicated since causality is likely to run in both directions, and other variables, such as education, may cause both. What scholars have done is to identify historical data that correlates strongly with historic institutions and then correlated the variation in current institutions that can be explained by these historical data with current day income levels. If cross national variation in current institutions maps closely to cross national variation in historical institutions (“institutional stickiness”) and if current day income levels, or education rates, do not cause historical institutions (which seems reasonable) then the historical data can be used as a so called “instrument” to produce a cleaner estimate of the causal effect of institutions.

Note that randomization and instrumentation are trying to solve the same empirical challenge. When randomization is possible it will be superior if implemented correctly (because perfect instruments only exist in theory), but there is of course a fairly limited range of questions for which randomized experiments are possible to design. In other cases scholars will have to do with instrumentation, or other alternatives such as matching, regression discontinuity or difference-in-difference estimations to better estimate a causal effect.

A second insight from this literature is that what constitutes successful institutions is context specific. Certain economic principles may be universal; incentives work, competition fosters efficiency and property rights are crucial for investments. However, as the example of China shows, what institutions are most likely to guarantee property rights, competition and the right incentives may vary depending on norms and historical experiences among other things. Successful institutional reforms therefore require a certain degree of experimentation for policy makers to find out what works in the context at hand. To just implement blueprints of institutions that have worked elsewhere typically doesn’t work. In other words, institutions must be legitimate in the society at hand to have the desired effect on individual behavior.

Coming back to aid policy, the lesson from this part of the literature is that for aid to contribute to economic and social development, focus should be on helping partner country governments and civic society to develop strong economic and political institutions. And since blueprints don’t work, it is crucial that this process involves domestic involvement and leadership in order to guarantee that the institutions put in place are adapted to the context of the partner country at hand, and has legitimacy in the eyes of both citizens and decision makers. Indeed, institution building is also a central part of aid policy. This sometimes takes an explicit form such as in financing western consultants with expertise in say central banking reform or how to set up a well-functioning court system. But many times it is also implicit in the way the money is disbursed, through program support rather than project support (where the former is more open for the partner country to use at their own priorities), through the partner country’s financial management systems and recorded in the recipient country budget. Also in the implementation of projects there is an element of institution building. By establishing projects within partner government agencies and actively involving its employees, learning and experience will contribute to institutional development.

Actual aid policy often falls short of these ambitions, though. Nancy Birdsall has referred to the impatience with institution building as one of the donors’ “seven deadly sins”. The impatience to produce results leads to insufficient resources towards the challenging and long term work of creating institutions in weak states, and the search for success leads to the creation of development management structures (project implementation units) outside partner country agencies. The latter not only generates no positive spill-overs of knowledge within government agencies, but can often have the opposite effect when donors eager to succeed lure over scarce talent from government agencies. The aid community is aware of these problems and has committed to improve its practices in the Paris declaration and the Accra Agenda, but so far progress has been deemed as slow.

Micro or Macro?

So, I started out saying that there is a risk that these two lessons from the literature may be in conflict if put into practice for actual aid policy. Why is that? At a trivial level, there is of course a conflict over the allocation of aid resources if we interpret the lessons as though the sole focus should be on either institutional development or best practice social projects respectively. However, most people would probably agree that there is a merit to both. In theory it is possible to conceive of an optimal allocation of aid across institutional support and social project support, in which the share of resources going to project support is allocated across projects based on best practices learned from randomized impact evaluations. In practice, however, it’s important to consider why these lessons from the literature haven’t been implemented to a greater extent already. After all, these are not completely new insights. Political economy and the logic of large bureaucratic organizations may be part of the answer. Once these factors are considered, a less trivial conflict becomes apparent, showing the need to think carefully about how to best proceed with improving the practices of aid agencies.

As mentioned above, one line of criticism against aid agencies is that they have had such a hard time to show results from their activities. This is partly due to the complicated nature of aid in itself, but critics also argue that it is greatly driven by current practices of aid agencies. First of all there is a lack of transparency; information about what decisions are made (and why), and where the money is going is often insufficient. This problem sometimes becomes acute, when corruption scandals reveal the lack of proper oversight. Secondly, money is often spent on projects/programs for which objectives are unclear, targets unspecified, and where the final impact of the intervention on the identified beneficiaries simply can’t be quantified. This of course limits the ability to hold agencies accountable to their actions, so focus instead tends to fall on output targets (have all the money been disbursed, have all the schools been built) rather than the actual effects of the spending. So why is this? According to critics, a reason for this lack of transparency and accountability is that it yields the agencies more discretion in how to spend the money. Agencies are accused of institutional inertia, programs and projects keep getting financed despite doubts about their effectiveness because agency staff and aid contractors are financially and emotionally attached.

In this context, more focus on long run, hard to evaluate institutional development may be taken as an excuse for continuing business as usual. Patience, a long run perspective and partner country ownership is necessary, but it cannot be taken as an excuse for not clearly specifying verifiable objectives and targets, and to engage in impact evaluation. It is also important that a long term commitment doesn’t have to imply an unwillingness to abandon a program if it doesn’t generate the anticipated results. It is of course typically much harder to design randomized experiments to evaluate institutional development than the effect of say free distribution of bed-nets. But it doesn’t follow that it is always impossible, and, more importantly, it doesn’t preclude other well founded methods of impact evaluation. The concern here is thus that too much emphasis on the role of institutional development is used as an excuse for not incorporating the main lesson from the “randomization revolution”, the importance of the best possible impact evaluation, because actual randomization is not feasible.

The concern discussed above is based on the implicit argument that aid agencies due to the logic of incentives and interests within bureaucratic institutions may not always do what is in their power to promote development, and that this is made possible through lack of transparency and accountability. The solution would in that case seem to be to increase accountability of aid agencies towards their politicians, the representatives of the tax payers financing the aid budget. That is, greater political control of aid policy would improve the situation.

Unfortunately, things aren’t quite that easy, which brings us to the concern with letting the ability to evaluate projects with randomized experiments being a prerequisite for aid financing. We have already touched upon the problem that programs for institutional development are hard to design as randomized experiments. It follows that important programs may not be implemented at all, and that aid allocation becomes driven by what is feasible to evaluate rather than by what is important for long run development. But there is also an additional concern that has to do with the political incentives of aid. The impatience with institution building is often blamed on political incentives to generate verifiable success stories. This is driven by the need to motivate aid, and the government policies more generally, in the eyes of the voters. It follows that politicians in power often have a rather short time horizon, that doesn’t square well with the tedious and long run process of institution building. Putting aid agencies under tighter control of elected politicians may therefore possibly solve the problem outlined above, but it may also introduce, or reinforce, another problem, the impatience with institution building.

Unfortunately, the perception that randomization makes it possible to more exactly define what works and what doesn’t, may have further unintended consequences if politicians care more about short term success than long term development. We know from principal-agent theory that the optimal contract gives the agent stronger incentives to take actions that contribute to a project if it becomes easier to evaluate whether the project has been successful or not. Think now of the government as the principal and the aid agency as the agent, and consider the case when the government has a bias towards generating short run success stories. In this case the introduction of a new technology that makes it easier to evaluate social projects (i.e. randomization) will make the government put stronger incentives on the aid agency to redirect resources towards social projects and away from institutional development. This would not be a problem if the government had development as its only objective, because then the negative consequences on effort at institution building would be internalized in the incentive structure. But in a second best world where politics trump policy, the improved technology may have perverse and unintended consequences. Greater political control will lead to less focus on institutional development than what is desired from a development perspective. A very benevolent (naïve?) interpretation of the motivation behind aid agencies’ tendencies to design social projects such that their effects are hard to quantify could thus be that it decreases the political pressure to ignore institutional development.

Concluding Remarks

The challenge to heed the two lessons from the literature thus goes beyond the mere conflict of whether to allocate the resources to institutional development or to best practice social projects once political economy and bureaucratic incentives are considered. Improved agency accountability may be necessary to avoid “institutional sclerosis” in the name of institution building and make sure that best practices are followed, but too much political meddling may lead to short sightedness and a hunt for marketable success stories. It is even possible, that the “randomization revolution” may make matters worse, if it becomes an excuse for neglecting the tedious and long term process of institution building and reinforces the political pressure for short term verifiable results.

What is then the best hope for avoiding this conflict of interest? That is far from a trivial question, but maybe the best way to make sure that agency accountability towards their political principals doesn’t lead to impatience with institution building is to form a broad-based political consensus around the objectives, means and expectations of development aid. The pedagogical challenge to convince tax payers that aid helps and that they need to be patient remains, but at least the political temptation to accuse political opponents of squandering tax payers money without proven effects and to pretend to have the final solution for how to make aid work, should be mitigated. But until then the best bet is probably to stay skeptical to anyone claiming to have the final cure for aid inefficiency, and to allow some trust in the ability of experienced practitioners to do the right thing.

Recommended Further Reading

Acemoglu, D., S. Johnson and J.A. Robinson (2001) “The Colonial Origins of Comparative Development: An Empirical Investigation“, American Economic Review 91(5), 1369-1401.
Banerjee, A. (Ed.) (2007), “Making Aid Work”, MIT Press.
Bannerjee, A. and E. Duflo (2008), “The Experimental Approach to Development Economics”, NBER Working Paper 14467.
Birdsall, N. (2005), “Seven Deadly Sins: Reflections on Donor Failings”, CGD Working Paper 50.
Birdsall, N. and H. Kharas (2010), “Quality of Official Development Assistance Assessment”, Working Paper, Brookings and CGD.
Duflo, E., R. Glennerster and M. Kremer (2007), “Using Randomization in Development Economics Research: A Toolkit”, CEPR Discussion Paper 6059.
Easterly, W. (2002), “The Cartel of Good Intentions: The problem of Bureaucracy in Foreign Aid”, Journal of Economic Policy Reform, 5, 223-50.
Easterly, W. and T. Pfutze (2008), “Where Does the Money Go? Best and Worst Practices in Foreign Aid”, Journal of Economic Perspectives, 22, 29-52.
Knack, S. and A. Rahman (2007), “Donor fragmentation and bureaucratic quality in aid recipients”, Journal of Development Economics, 83(1), 176-97.
Rodrik, D. (2008), “The New Development Economics: We Shall Experiment, but how Shall We Learn?”, JFK School of Government Working Paper 55.

Disclaimer: Opinions expressed in policy briefs and other publications are those of the authors; they do not necessarily reflect those of the FREE Network and its research institutes.