Tag: ICRG

In recent years, variously defined indicators of corruption from different sources have aimed at raising awareness about corruption and to provide researchers with better data for analyzing the causes and consequences of corruption. Most of them have achieved spectacular popularity, and are regularly cited in news reports on corruption around the world. However, in a 2006 study for the World Bank, Stephen Knack warns that the particular properties and limitations of these indicators are often neglected by data users, often leading to wrong interpretations and sometimes puzzling disagreements about the actual situation in a country or a region and its changes over time. The first part of this brief summarizes the main conclusions of this study; the second part presents updated data from different sources on recent corruption trends in the new EU members and the neighbors to the east, as a clear exemplification of the issues discussed.

Existing corruption indicators differ in many ways: where the original information or evaluation comes from, how they are built, who are their constituencies or audiences, as well as which of the many aspects of corruption they intend to capture. For these reasons, no single indicator or data source is best for all purposes.

The corruption indicators can be subdivided into three main groups: those based on surveys, either of firms or households, those reporting expert assessments, and finally, the recently popular composite indexes.

Two examples of firms’ surveys that will be presented below are the Business Environment and Enterprise Performance Survey (BEEPS) and the World Economic Forum (WEF) “Executive Opinion Survey”. Similar enterprise surveys have been conducted by the World Bank and in the IMD World Competitiveness Yearbook. However, BEEPS and WEF are more systematic and better comparable across countries and years, have broader coverage and disclose more information about their definitions and methodology, which makes them, in a sense, more research-friendly.

Surveys are relatively well-suited for evaluating the administrative corruption since they measure the prevalence of corruption as experienced by users of government services. They can also measure some aspects of state capture by asking about perceived undue influence over laws and regulations that affect business. However, surveys are definitely less effective in assessing the prevalence of corrupt transactions that occur entirely within the state, for example when politicians bribe bureaucrats or when funds are illegally diverted. Many types of conflict of interest are also not easily captured by surveys. For example, the equity stakes of public officials or employment promises to them by the firms (World Bank, 2000).

Expert assessments of corruption have been most widely used for comparisons across countries and over time because of bigger coverage in both dimensions. A large and growing number of organizations provide such assessments. Some examples are Freedom House’s Nations in Transit (NIT), the International Country Risk Guide (ICRG), the World Bank’s Country Policy and Institutional Assessment (CPIA). Corruption ratings from these sources are based on the assessment by a network of correspondents with country-specific expertise. In some cases, the final ratings are subsequently determined centrally by a smaller group of people. The organizations that are behind these indicators may be very different, with potential implications for what their ratings are measuring. Some are advocacy NGOs. Others are for-profit companies marketing their product to multi-national investors and paying subscribers. Most subscribers to the ICRG, for example, are more interested in conditions faced by foreign investors than in those faced by local residents. Corruption ratings produced by development agencies are also potentially influenced by their constituents (if for example they take into account the consequences for funds allocation decisions or relations with local partners).

An important difference as compared to the firms or households surveys is that corruption assessments place less emphasis on experience and more on perceptions. Moreover, the respondents in a firms’ survey can be asked more specific and objective questions because they comprise a more homogeneous group. For example, a typical question can be “Was an informal gift or payment expected or requested to this establishment, in reference to the application for an electrical connection?” (from the BEEPS 2009 questionnaire). Instead, a questionnaire directed to a group that includes public officials, academics, journalists, etc. must frame questions in such a way that they can be answered meaningfully by all of them, which necessitates broader questions.

More recently, composite indexes have gained popularity. Well known examples include Transparency International’s widely-cited “Corruption Perceptions Index” and the World Bank Institute (WBI) “Control of Corruption” index (Kaufmann, Kraay and Mastruzzi, 2008). Although the statistical methods used to produce them vary somewhat, both indexes standardize several corruption indicators such as ICRG, CPIA and even survey outcomes, to place them on a comparable scale, then aggregate them, so as to obtain a single value for each country. As a result, composite indexes suffer from the same problem as the corruption measures from individual sources such as ICRG, NIT or CPIA: if any component of a composite index is constructed in an opaque manner, the composite index will be opaque as well. Further limitations are introduced by the process of aggregation. Composite indexes have no explicit definition, but instead are defined implicitly by what goes into them. The sources used in constructing these composite indexes change over time, and from country to country in a given year. For any pair of countries the index values are very likely to reflect differing implicit definitions of corruption.

The standardization procedure used to place different indicators on a common scale precludes the ability to track changes meaningfully over time.  A final issue with the composite indexes is the interdependence of expert sources. If expert assessments display high correlations driven by the fact that they consult each other’s ratings – or that they all base their ratings on the same information sources – this can undermine the main premise of the aggregation methodology that more sources produce more accurate and reliable estimates. The addition of another expert-based source containing little new information – relying on the same information sources as its competitors, or even checking their ratings – can actually reduce the accuracy of the composite index.

A general caveat in the use of corruption indicators, beyond the weaknesses of individual types discussed above, concerns the importance of their intended use. For some purposes, broader measures may be preferable: for example, a researcher studying the relation between corruption and economic growth may have no particular view on exactly which aspects of corruption most impair growth, and is hence content with a general measure. For other purposes, however, narrower measures may be required. For example, a donor funding projects in a country may be interested in a measure of corruption in public procurement, while a donor providing budget support might prefer a measure of the likelihood of funds diversion to unintended purposes. The design of effective anti-corruption reforms requires narrow measures to identify specific problem areas and track progress over time, and so on.

Finally, it is important to remember that some indicators are more suitable than others for measuring changes over time. Broad, multi-dimensional indicators are potentially problematic in this respect, because there is no way to ensure that the implicit weights given to the various dimensions do not change over time. Some indicators have no fixed and explicit criteria provided for each ratings level, so there is no way of ensuring that the same numerical rating means the same corruption level from one year to the next.

With this background in mind, it is easy to understand why, while it is often possible to form a broad assessment on the general situation and trends in corruption, different sources might often disagree markedly on specific countries, and in particular on which countries have improved and which have not. The evidence from different sources on recent corruption trends reported below provides a clear example in this respect. We are going to focus on the new EU members (Estonia, Bulgaria, Romania, Slovenia, Slovakia, Czech Republic, Hungary, Poland, Lithuania, Latvia), indicated as EU-group, and the non-Baltic former Soviet Republics (Armenia, Azerbaijan, Belarus, Georgia, Kazakhstan, Kyrgyzstan, Moldova, Russia, Tajikistan, Turkmenistan, Ukraine, Uzbekistan), indicated as CIS-group [1].

Levels and Trends in Corruption for the New EU-Members and the Eastern Neighbors

The Business Environment and Enterprise Performance Survey (BEEPS) is a nationally-representative survey of business firms assessing corruption and other problems faced by businesses in the ECA region. The BEEPS is sponsored by the European Bank for Reconstruction and Development (EBRD) and the World Bank, and has covered almost every country in the region since 1999. The two most recent waves with a good coverage of our countries are 2005 and 2009. The surveys typically contain multiple questions pertaining to narrower aspects of corruption, and so do the BEEPS.

Looking at the outcomes of the BEEPS, the most dramatic change between 2005 and 2009 is in the “bribe tax”, the share of annual sales paid in “informal payments or gifts to public officials to get things done”. The average in the new EU members increased more than four-fold from .72% to 3.11% of firm revenues. A positive value for the bribe tax was reported by 28.12% of firms in 2005, increasing to 62.1% in 2009, although this might simply reflect an increasingly open attitude in answering the survey. The corresponding increases for the CIS-group are more moderate, from 1.31% to 4.26% bribe tax and from 40.7% to 60.1% of firms declaring positive values. The only country where the bribe tax did not increase is Poland, although data for 2009 are not available for Belarus, Georgia, Tajikistan, Ukraine and Uzbekistan. The biggest increases are reported in Estonia and Slovenia, although they started from the lowest levels within the EU-group (0.29 and 0.17 respectively). These are two countries that, as we will see later, are consistently singled out as the best performers by the other indicators. This apparent contradiction might be due to a different reporting attitude in these countries. Similarly, the lowest level of bribe tax in the CIS-group for 2009 is reported in Russia (1.31), while the highest levels (8.8) in Azerbaijan, a country that according to other indicators is doing relatively well.

Besides the bribe tax, among the numerous other questions on corruption issues in the BEEPS, most show evidence of a modest improvement.  For example, in 2005 about 13.4% (24%) of firms in the EU- (CIS-) group reported that paying bribes was frequently, usually or always necessary to get things done, and this figure is down to 6.75% (18.8%) in 2009.  Most questions about specific public services also show evidence of a decline in the incidence of bribe paying, e.g. when paying taxes, dealing with customs and the courts.

The assessment on the fairness of the courts got worse in both areas, but at the same time it is considered a big obstacle by fewer businesses as compared to 2005. Also, the share of businesses that admit to paying a kickback payment to obtain a government contract, and the share of sales required for this payment, decreased over this period, markedly for the new EU members, though only slightly for the former Soviet Republics.

Slovenia and Estonia are the champions also in this respect, as well as Armenia in the CIS-group. Kickback payments are most expensive in Latvia (3.06% of sales) and Russia (4.65%). Corruption was however cited as the biggest obstacle to doing business by an increasing share of firms everywhere [2]. In the CIS-group as a whole, the share of firms that consider corruption the biggest obstacle to business increased from 6.4% in 2008 to 8.16% in 2009. The individual countries with the biggest shares are Romania (9.5%) and Azerbaijan (17.8% of firms), respectively in the first and second group. Ironically, the biggest increase between 2005 and 2009 is in Poland, the only country where the reported bribe tax actually decreased. This highlights how tricky it is to aggregate the information from these sources, given that very different aspects of the situation in a country are captured by each item.

More difficulties emerge with respect to evaluating change over time since different measures often move in opposite directions for a given country. For example, both Hungary and Azerbaijan experienced the biggest increases in bribe tax, but also a sharp decrease in kickback payments for government contracts and it is hard to balance the one against the other. This is also a reason why the picture emerging from these data does not necessarily agree with the aggregate indicators discussed below, although they are in part based on the very same outcomes of the surveys.

The World Economic Forum (WEF) “Executive Opinion Survey” is another cross-country survey of firm managers. The sample in each country is selected with a preference for executives with international experience, who tend to be from larger and exporting firms. The questions are designed to elicit “the expert opinions of business leaders” on corruption and other issues, and focus much less on direct firms’ experiences. Moreover, the aim is solely to produce country-level measures of the business climate, and not firm-level analyses. Cross-country rankings on several corruption questions from this survey are published in WEF’s annual Global Competitiveness Report. Ratings are computed as the simple average of all executives’ responses.

The 2011 WEF data include 7 variables related to corruption, all scaled from a low value of 1 to a high value of 7: Diversion of public funds, Irregular payments and bribes, Judicial independence, Favoritism in decisions of government officials, Burden of government regulation, Transparency of government policymaking and Ethical behavior of firms. The sample includes a total of 142 countries, including many developed countries, and covering most of the countries we have been addressing above. Both the average rating and the average rank are slightly higher for the EU-group, but the similar average hides quite a bit of variation between different countries and in different dimensions.

In particular, the CIS-group ranks higher with respect to both the extent to which government regulation is perceived as a burden for business, and the perceived transparency of policymaking, and the two averages are extremely close when it comes to the assessment of favoritism in officials’ decisions. The largest difference between the two groups seems to be the prevalence of irregular payments and bribes, in accordance with the evidence from the BEEPS.

Nevertheless, some countries in the second group, like Georgia and Tajikistan, have a higher average ranking than most of the new EU members and position themselves extremely well even in global terms in some dimensions. For example, Georgia is number 7 in the world with respect to the (absence of) burden of regulation, although not many more reach the upper quartile or even the upper half of the ranking. On the other hand, some of the new EU countries do very poorly in some respects, like the Slovak Republic ranking 135th (of 142) in terms of favoritism by public officials and the Czech Republic being 124th in diversion of public funds.

Compared to 2010, the EU-group saw a slight worsening in their rating, while the CIS-group improved. More in detail, half of the countries in the first group went down, including some quite substantial drops (Estonia, Poland and Slovenia, down by more than 1 point) while the others improved, though not spectacularly. All but one country (Georgia) in the second group improved their average rating from 2010, the biggest progress taking place in Azerbaijan with 1.5 points.

As opposed to the surveys discussed above, the NIT, CPIA and ICRG each provide a single measure of corruption, intended to reflect a mix of various aspects of corruption.

The NIT index is mostly concerned with the impact of corruption on business. It measures the corruption with on a 1-7 scale, 1 being the best possible rating and 7 being the worst, with quarter-point increments allowed.

The ranges of variation in the ratings during the last five years for the two regions do not overlap at all: all of the new EU countries positioned themselves always below a score of 4, while all the countries in the CIS-group stayed well above this threshold. This implies that the best performers within this group (Georgia and Armenia) have a consistently lower rating than the worst performing EU countries (Bulgaria and Romania). However, the trend over time in this period is very similar across the two regions. Both the averages are very flat, with a slight upward trend (i.e. to the worse). In the EU-group, this reflects the fact that five countries saw worsening in their rating, three saw no change at all and only two (Estonia and Lithuania) a slight progress. The lowest (and hence best) score is Estonia and Slovenia’s 2.25. Also in the other group only two countries – Armenia and Georgia – improved their rating. They also have the lowest scores in the region, 5.25 and 4 respectively. Six countries kept a stable rating while four got worse. The highest (and hence worst) score, 6.75, goes to Turkmenistan and Uzbekistan.

The CPIA question “Transparency, Accountability and Corruption in the Public Sector”, is assessed on a 1-6 scale, where a lower level corresponds to a worse situation in terms of corruption. This index focuses on less developed countries, so the EU-group is not covered. The most recent available data are for the period 2008-2011, during which four out of the six developing regions in the world improved.

In contrast to the stagnation with slight worsening described by the NIT, the ECA region is the one that sees the steepest improvement in the CPIA rating, increasing to 2.87 in 2011. This contrasting assessment can be explained by the fact that only six of our CIS-group countries are included in the CPIA sample: Armenia, Azerbaijan, Georgia, Kyrgyzstan, Moldova and Uzbekistan. If we look at the average only in those six, also the NIT rating improved by about the same relative amount (1.5% of the value range). The two indexes do not agree, though, on the individual countries that they reward with a higher or punish with a lower score. In particular, only Georgia improved in both ratings, while Uzbekistan, for example, got a better CPIA score but a worse NIT score; Armenia and Azerbaijan, that respectively improved and worsened in the NIT assessment, are completely stable in the CPIA, and the opposite is true for Moldova.

Unlike the CPIA, the ICRG sample includes most developed countries. The focus of the ICRG is to establish the relative incidence of corrupt transactions. Its corruption ratings range from a minimum value of 0 to a maximum of 6, where higher rating corresponds to a better situation.

The latest available data are however not as recent as for the other indicators discussed here. In the three years up to 2007, the mean rating remained stable in both groups, around 2.5 in the new EU members and 1.8 in the former soviet countries, although only three of eleven countries from this group are included. Also in this case, the two ranges of values for the two regions do not overlap.

In the EU, Lithuania’s rating went down while Poland’s went slightly up. Estonia and Slovenia are again the best performers together with Hungary. The lowest rating in the region goes to Bulgaria, just as in the NIT evaluation, together with Latvia. All the three countries in the CIS-group made improvements, although from dismally low levels. This is not in contrast to the other assessments, since the data refer to an earlier period. The highest score of the three is Moldova’s (low) 1.5.

Both of the widely-known composite indexes of corruption (TI and WBI index) show large differences between the EU members and their eastern neighbors. The average score, varying from 1 to 10 and from -2.5 to 2.5, respectively, are much higher for the first than for the second group. Similarly the ranks – from 1 (best) to 182 (worst) for TI, reversed scale from 0 (worst) to 100 (best) for WBI – reflect a much worse situation in the CIS-group. However, the former Soviet countries improved their WBI rank between 2009 and 2010, as opposed to the new EU members which saw a slight drop. Although changes over time for these indexes should be taken with caution, this is coherent with the 2010-2011 comparison in the WEF.

The two indexes also agree on best and worst performer, respectively; Estonia and Bulgaria in the first group (Slovenia was best performer in 2009 according to WBI) and Georgia and Turkmenistan (on par with Uzbekistan according to TI) for the second. Both the largest improvement (Lithuania) and the largest backslide (Slovenia) from 2009 happened in the EU-group, but a larger share of the CIS-group countries experienced improvements, which is reflected by a smaller drop in the average score. The main difference between the two indexes is that WBI uses more sources and reports a value even for cases when only one source is available (TI requires a minimum of three sources), obtaining as a consequence a broader coverage. Otherwise, the two indexes are quite correlated, and subject to the same problems.

Summing up, all the indicators agree, not surprisingly, that the situation looks much brighter in the EU-group than in the CIS-group. Although, it is not clear that they are keeping up the good work in the most recent years. There is relatively more evidence of improvement over time in the CIS-group, despite the dismal starting point. Only few countries emerge unequivocally as good or bad performers. One example being the coherently positive performance of Georgia; for most of the other countries, the picture is mixed.

Given the variety and breadth of indicators, this conclusion was very much expected. Corruption is such a broad and multidimensional phenomenon that different indicators and different assessments are bound to result in different, often contrasting pictures. Unless one is very clear on which specific aspect is in focus, and sticks consequently with one particular measure, any conclusion based on general comparisons of corruption indicators both between countries and over time should be taken with serious cautiousness.

▪

References

• Kaufmann, Daniel, Aart Kraay, and Massimo Mastruzzi. (2008). “Governance Matters VII: Aggregate and Individual Governance Indicators, 1996-2007.” World Bank Policy Research Working Paper 4654. Washington: World Bank (June 24).
• Knack, Stephen. (2006). Measuring Corruption in Eastern Europe and Central Asia: A Critique of the Cross-Country Indicators. World Bank Policy Research Working Paper No. 3968, Washington,D.C.
• World Bank (2000). Anticorruption in Transition: a Contribution to the Policy Debate. Washington DC: World Bank.
• European Bank for Reconstruction and Development, Business Environment and Enterprise Performance Survey (BEEPS), http://www.ebrd.com/pages/research/economics/data/beeps.shtml
• World Economic Forum (WEF), Executive Opinion Survey, https://wefsurvey.org/index.php?sid=28226&lang=en&intro=0
• Freedom House, Nations in Transit, http://www.freedomhouse.org/report-types/nations-transit
• PRS Group, International Country Risk Guide, http://www.prsgroup.com/icrg.aspx
• Transparency International, Corruption Perception Index, http://www.transparency.org/research/cpi/overview
• World Bank Group, Country Policy and Institutional Assessment, http://www.worldbank.org/ida