Tag: Procurement

Decentralization, E-Procurement and Efficiency of Public Procurement in Ukraine

20190311 Decentralization, E-Procurement and Efficiency Image 01

This brief is based on a research that investigates if there’s a synergy effect of procurement and decentralization reforms in Ukraine on procurement efficiency. The analysis shows the similarity between new and old administrative units procurement performance. Although the analysis does not provide evidence of a significant synergy effect, such a similarity could be considered as something positive (due to the lower market power and capacity of the newly created administrative units) that should be analyzed further.

Decentralization in Ukraine

In April 2014, the Ukrainian government launched a systemic decentralization reform – a delegation of a significant part of resources and responsibilities from oblast and raion-level executive branches of the government to the local self-government level. The key issue of the reform was the creation of a strong basic level of local self-government in line with the European Charter of Local Self-Government. This is done through the creation of amalgamated hromadas (AH), the merger of several settlements with a single administrative centre.

An AH is governed by a council of the amalgamated hromada (CAH), a representative body of a local self-government. It is elected by residents of territorial communities and is responsible to independently resolve local issues, develop and approve AH budgets. Particularly, the government redistributed tax revenues and expanded the system of state subsidies (medical and educational subvention, subvention on development of amalgamated hromadas etc.) that could be used according to AH decisions.

In 2015, 159 AH were established. As of September 2018, 831 AH made up by 3,796 hromadas with 7 million residents (decentralization.gov.ua).

Public E-Procurement

Contemporaneously, a reform of public procurement has been implemented. According to the Law on Public Procurement of Ukraine, since August 2016, public procurement must be announced and executed through ProZorro, a public procurement web portal administered by the state enterprise. The e-procurement system consists of a central database, an auction module and commercial marketplaces (Figure 1).

In order to participate in public tenders, bidders can choose one of 22 commercial marketplaces (7 companies that provided initial investment for the project became the first marketplaces). The commercial marketplaces are web resources managed by private companies that provide access to the electronic procurement system.

Figure 1. ProZorro Architecture

The electronic procurement system does not completely cover the procurement cycle. Actually, it only covers the tendering process while planning and contract execution are mostly out of the system (plans are published online). Moreover, the existing legislation provides opportunities to manipulate tendering process by switching between different procedures. Within the ProZorro system there are 6 main procurement procedures that can be used by procuring entities depending on the volume and specifications of their needs.

Selection of procedures is based on a Threshold principle. There are three thresholds (Figure 2):

  • Lower Threshold (LT). Contracting authorities are not obliged to report procurements in the electronic system if the total value of procurement is lower than UAH 50,000.
  • Higher Threshold (HT). Contracting authorities are not obliged to use open competitive procedures if the total value of tender is lower than a defined level. This level is equal to UAH 200,000 for goods and services and UAH 1.5 million for construction.
  • Euro Threshold (ET). The value of tender that requires applying the strictest competitive auction procedure. The euro threshold corresponds to thresholds used in EU public procurement legislation and is different for goods and services and construction works.

Transparency and efficiency indicators

Typically, a procurement process is divided into three stages: pre-tender stage, tender stage and post-tender stage. To measure efficiency and transparency of AH procurement, we constructed a system of eleven indicators that evaluate each stage of the procurement process (Table 1).

Table 1. Transparency and efficiency indicators used in the report

  • Avoiding ProZorro. Both this and the following indicator would be associated with a decreased transparency, and, while not necessarily evidential, raise suspicions about procurement done in a less efficient and more collusive/corrupt way. AH are not inclined to avoid the ProZorro system. The share of procurements outside of ProZorro per AH is smaller than the corresponding indicators for other administrative units (Raion State Administrations, RSA, and unamalgamated hromadas, UH).
  • Avoiding higher and euro thresholds. An analysis of ProZorro data shows that for AH, 84% and 11% of AH have at least one “suspicious” case for each corresponding threshold. For, RSA the indicators are 73% and 31% respectively.
  • Unanswered questions. Having a productive dialogue with suppliers is of crucial importance for the success of the procurement. It helps to adjust the tender documentation so that it does not include discriminatory demands. Analysis shows that this practice is not dominant: relatively small proportion of AH and UH uses it.
  • Level of competition. Higher competition is normally assumed to imply more efficient procurement deals. There is no difference in competition across administrative units and products (measured by the number of bidders per tender).
  • High disqualification rates can be a consequence of ill prepared tender documentation with unclear technical specification or it can be a consequence of suppliers’ inexperience. It can also be a sign of corruption, when a tender committee is trying to find any reason to disqualify ‘unwanted’ suppliers. The analysis shows that disqualification is not a significant problem and, in fact, there are no significant differences across administrative units and products.
  • Success rates. To successfully complete competitive procurement, the contracting authority has to determine the technical description of a good and its expected value based on their budget and market analysis. It also has to prepare and publish tender documentation and answer questions of potential suppliers. Finally, after the auction, the contracting authority has to evaluate documents of the auction winner and sign a contract. Failure in each of these steps will lead to unsuccessful or cancelled procurement. There is no significant difference across administrative unit groups in terms of procurement success rate
  • Abnormal saving rate. Generally, a high saving rate (the difference between tender expected and contracted values) is regarded as a positive indicator, however, a too high rate is suspicious. It can be a sign of an inadequate expected value or an abnormally low price (suspicious behavior on behalf of the supplier). For the purpose of this study, we consider a saving rate abnormal if it is greater or equal to 30%. The analysis shows that AH had a significantly lower share of tenders with abnormally high saving rate than RSA. On average, 0.6% (in terms of value) of AH tenders are suspicious, for RSA this indicator equals 6.1%
  • Contract termination. Frequent contract termination is a sign of significant inefficiencies in the procurement function of contracting authorities. The share of terminated contracts (as a percentage of the total contract value) is approximately similar for AH and RSA. On average, one AH has 5% of contract value terminated, while RSA indicator equals to 6%.
  • Fixing the price with additional agreements. Although the Ukrainian Law on Public Procurement gives the right to amend the price per unit indicated, this right can be misused. It could lead to significantly higher costs. RSA are strikingly different from the other two groups – on average 20% of the RSA contract value stems from contracts with amended prices. This difference is the consequence of the different structure of goods and services procured by AH and UH on the one hand and RSA on the other.
  • Share of largest supplier. Generally, it is considered to be a good practice, when contracting authorities are not overly dependent on one supplier. Approximately 30% of contract value of average AH and RSA belongs to one supplier. For UH this indicator is even higher (on average 48%) but it could be the consequence of the smaller number of contracts signed by UH.

Effect on prices

If contracts are successfully executed, the price of a good usually summarizes the efficiency of the procurement process.

There are many factors that affect the prices of goods in public procurements. On the one hand, AH (a) “realized” that they spent their own money and thus, they have more incentives to save and (b) have more power to choose where to spend. On the other hand, there are some factors that have the opposite effect: (a) because of low quantity demanded, the tenders announced are not interesting for large companies that could potentially provide lower price, and (b) the procurement officers could have insufficient capacity to negotiate lower price. Although, it is impossible to evaluate all these factors, we can assess their outcome – the contract price of a good.

For this analysis we looked at the prices on homogeneous goods such as food (potato, butter, eggs) and fuel (petrol A95, petrol A-92, diesel.

Table 2 summarizes the prices on the goods received by hromadas and compares it to the prices received by other types of entities (UHs and RSAs).

The data shows that for food products, AH average prices are lower or not different from UH, and slightly higher or not different from prices received by RSAs.

Even though there are some differences in prices of Petrol A-95 (partially due to inefficient planning and contracting at periods of higher prices), in general, the price level is very similar between all the entities.

In most cases, despite some warnings, there were no significant gaps between AH’s prices and UH or RSA. Moreover, the more competitive is the market of goods procured, the closer are prices received by different administrative units.

Table 2. Prices of goods by administrative units in 2017


The analysis shows the similarity between AH and RSA in terms of number of procurements, success and disqualification rates as well as competition level and share of terminated contracts. However, in cases when it is allowed by the Procurement Law, AH are more likely than RSA to choose direct selection of a supplier than a competitive procedure. Such behavior can be caused by a lack of professionalism (or even corruption), a desire to select a local trustworthy company or just because it is easier and faster to conduct uncompetitive procedure below the threshold.

On the other hand, AH are less inclined (in comparison to RSA) to avoid the ProZorro system (by using procurements below UAH 50 K) and to sign additional agreements that increase the price. Such behavior is potentially punishable by law. It can be suggested that procurement officers of AH only recently started to work with tenders above HT and are therefore more conscious of possible negative consequences of such actions.

The price per unit is the key indicator that summarizes information on procurement efficiency. Although AH show varying price efficiency, their prices of procured goods, in general, are not worse than in other administrative units’ groups. The more competitive the market, the closer are prices (especially in the case of fuels). Even if some gaps were observed, these differences are decreasing over time. Better planning can help to receive lower prices (better estimation of needs and choosing appropriate  periods for procurement).

Currently, the analysis does not provide an evidence on a significant synergy effect of decentralization and procurement reforms.  There are no significant differences between old and new administrative units. However, usually new communities have lower market power and capacity, and “no difference” could be considered as a positive sign that should be analyzed further.


Disclaimer: Opinions expressed in policy briefs and other publications are those of the authors; they do not necessarily reflect those of the FREE Network and its research institutes.

Reputation for Quality and Entry in Procurement: Is there a Trade-Off?

20140224 Reputation for Quality and Entry Image 01

How much weight should be given to past performance indicators when selecting contractors? Does a large weight assigned to suppliers’ previous performance deter entry by new, innovative suppliers that have no track records for the very reason that they are new? If yes, how should we take this into account when designing procurements for firms or governments? This note describes recent research that sheds light on these questions crucial for every government and organization.


How should past performance be accounted for when selecting (public or private) contractors? On one hand, giving large weight to suppliers’ previous performance in assigning contracts may improve incentives in procurement; on the other hand, it may deter entry by new, innovative suppliers for the very reason that they are new. Are there ways to structure procurement rules and procedures to minimize or eliminate these costs? How can well performing suppliers be rewarded, but not at the expense of losing the most innovative start-ups, that could pose important positive externalities on the buyer and on society overall? These questions are important ones and every procurement manager, in the private and public sector, should know how to answer, or at least how to think about, them. Unfortunately, if one looks at the leading management and operations textbooks, or at public procurement textbooks, it is hard to find a line that could help in making these crucial decisions. The only procurement book that at least mentions these crucial questions, to our knowledge, is the Handbook of Procurement (2006; see Ch. 18, by Dellarocas et al.). However, even that handbook falls short in providing evidence-based or – more generally – research-based guidance for these questions. This is the case because there is practically no research dealing with these everyday problems. One recent exception is an experimental study recently undertaken by Butler, Carbone, Conzo and Spagnolo (2013) that will be discussed in depth in the reminder of this brief.

Public Policy Relevance  

Before getting into the results of this recent study, let me provide some background information that will give an idea of the relevance of these questions for public policy.

Public procurement currently accounts for between 15% and 20% of the GDP in developed countries (see http://cordis.europa.eu/fp7/ict/pcp/key_en.html).  In 2011, the total public procurement market in the EU – i.e. the purchases of goods, services and public works by governments and public utilities – reached a size of approximately €2,500 billion, corresponding to 19 percent of GDP (see e.g. http://ec.europa.eu/internal_market/publicprocurement/docs/modernising_rules/public-procurement-indicators-2011_en.pdf). As Table 1 below shows, despite year-to-year fluctuations, there has been an overall increase in procurement expenditures relative to 2007 levels, both absolutely and proportionately. The increasing shift in focus in EU innovation policies, from “push-based” mechanisms like R&D subsidies/tax breaks towards demand-led “pull” mechanisms, like Pre-Commercial Procurement, is likely to further increase the volume of this market.

Table 1: Total EU procurement expenditure on works, goods and services
In EUR billion











As a % of GDP











Source: DG MARKT. 2011. Public Procurement Indicators.

The enormous size of this market notwithstanding, we know relatively little about whether, when, and how buyers should use reputational indicators based on past performance in selecting among sellers, and whether the use of such indicators necessarily reduces the ability of new sellers—i.e., sellers with no history of past performance—to enter the market.

It is well known that reputational mechanisms that reward past performance are important governance tools that complement (and sometimes substitute for) contracts in private transactions (Calzolari and Spagnolo 2009). Private buyers, however, are typically only concerned about the price and quality of the good they buy. Regulators in charge of public procurement, instead, are usually also concerned that the public procurement process is transparent and open for obvious accountability reasons. The need to prevent favoritism and corruption has led lawmakers around the world to ensure that open and transparent auctions where bidders are treated equally—even when in some crucial dimensions they have very different track records—are used whenever possible.

The trend in the US

The costs of limiting discretion to ensure public buyers’ accountability – such as the possibly large cost of not allowing reputational forces to complement incomplete procurement contracts – were stressed by Kelman (1990), who pushed for a deep reform of the US procurement system when he was the head of public procurement during the Clinton administration. The reform was targeted at reducing the rigidity of procurement rules in the Federal Acquisition Regulations and allowing public buyers to adopt more flexible purchasing practices common in the private sector, including giving more weight to suppliers’ past performance. Since the Federal Acquisitions Streamlining Act of 1994, US federal departments and agencies are expected to record past contractors’ performance evaluations and share them through common platforms for use in future contractor selection.

However, the US Senate recently expressed the apparently widespread concern that past performance-based selection criteria could hinder new and small businesses’ ability to enter and compete effectively, leading to an intriguing, but inconclusive report by the General Accountability Office.

This is not to say that US regulators were not concerned with the ability of small and medium enterprises (SME) – sometimes the most innovative part of the economy – to enter public procurement markets. In the US, this long-held concern led to large programs like the Small Business Act, with its rules limiting the bundling of public demand in very large procurements and establishing the Small Business Agency, and the ‘set aside’ (procurement only open to SMEs) common in many types of procurement auctions. However, the worry that past-performance based selection may contribute to the exclusion of novel and smaller firms only arose in the last couple of years.

The (opposite) trend in the EU

The European Union has instead been moving precisely in the opposite direction. An important concern driving procurement regulation in Europe since the Treaty of Rome has been helping the process of common market integration by increasing cross-border procurement, i.e., the amount of goods and services each EU Member State buys from contractors based in other states. The EU Procurement Directives that coordinate public procurement regulation in the various European states have been limiting the use of past-performance information in the process of selecting among offers—a feature that came under broad attack during the 2011 consultation for the revision of the EU Directives (see Replies to the Consultation on the 2011 EU Green Book on Public Procurement regulation). The EU regulators appear to have been always convinced that using reputational indicators as a criteria for selecting contractors leads to manipulations in favour of local incumbents, at the expense of cross-border procurement and market integration.  (see e.g., http://ec.europa.eu/internal_market/publicprocurement/modernising_rules/consultations/index_en.htm).

Only very recently – now that the US is moving towards reconsidering the effects, and possibly limiting the use of past-performance indicators – has the EU started to move (again in the opposite direction to the US) towards leaving more space to these indicators, perhaps as a reaction to the comments they received in recent consultations.

Finally, to have an idea of the lack of research-based knowledge guiding policy in this field, note that in some cases EU regulation already acknowledges the crucial importance of past-performance based reputation for some types of procurement. For example, the European Research Council (ERC) provides funding to top researchers in Europe, who are selected through peer review, and the track record of the researchers is usually the main awarding criterion. ERC funding is distributed almost exclusively based on reputation criteria in order to support the best and the brightest. Other European instruments for the procurement of research, such as the FET-OPEN program, are based on a strictly enforced, completely anonymous evaluation instead, without obvious reasons justifying this opposite approach. On the dedicated homepage of these programs it states: “The anonymity policy applied to short proposals has changed and is strictly applied. The part B of a short STREP proposal may not include the name of any organization involved in the consortium nor any other information that could identify an applicant. Furthermore, strictly no bibliographic references are permitted.”

Reputation and Entry in Procurement

In a recent research paper (Butler, Carbone, Conzo and Spagnolo, 2013) we have been trying to fill at least part of this knowledge gap and offer some initial evidence-based guidelines for future policy.

The study

We build a simple model of repeated procurement with limited enforcement and potential entry and implement it in the laboratory. We focus on reputation as an incentive system to limit moral hazard in the quality dimension as well as on the effect of reputation on selection through entry. We assume that some costly-to-produce quality dimension of supply, although observable to the parties, is too costly to verify for a court to be governed through explicit contracting and is therefore left to reputational governance. We make the additional assumption that there is a potential entrant firm that is more efficient than all incumbent firms. In this context, we study how quality, price, entry and welfare change with the introduction of a simple and transparent reputational mechanism. This mechanism rewards an incumbent firm that chooses to provide (costly) high-quality production with a bid subsidy in the subsequent procurement auction, and may also award a bid subsidy (of varying size) to an entrant with no history of production.

Note that in the case of public procurement and of firms’ vendor rating systems, we are talking about reputational mechanisms based on public rules, known and accepted by suppliers. Formal mechanisms and rules give commitment power to the buyer and can be designed in many different ways. A common mistake is to assume that reputational mechanisms must be designed along the line of the eBay feedback system, in which new sellers start with “zero reputation”. However, a buyer with some commitment power concerning the rules for information aggregation and diffusion and for selecting suppliers may well award a positive rating to new entrants—e.g. the maximum possible rating, or the average rating in the market, putting entrants at less of a disadvantage—and ensure that this is taken into account by the scoring rule that selects the contractor, even if the contractor has never before interacted with the buyer. Indeed, private corporations often have vendor rating systems in which all suppliers start off with the same maximal reputational capital—a given number of points—and then lose points when performing poorly and are suspended for some time if their reputational capital falls below a certain low threshold. This type of vendor rating system creates an advantage for new suppliers, most likely stimulating rather than hindering entry, suggesting that it is possible to design a reputational mechanism in public procurement that simultaneously sustains quality and entry.

The results

First of all, the study shows that concerns about reputation-based selection hindering entry are justified: naively introducing a “standard” reputational mechanism in which only good past performance is rewarded with a bid subsidy in the following procurement auction increases quality provision, but it also significantly reduces entry.

In contrast to this first result, the study goes on to show that properly designed reputational mechanisms in which new entrants, with no history of past performance, are awarded a moderate or high reputation score—as is often done in the private sector—actually foster rather than hinder entry while, at the same time, delivering a substantial increase in high quality goods provision.

The third important result of this study is that the total cost to buyers (buyer’s transfer) does not increase when a reputational mechanism is introduced, even though (costly) quality provision increases. The introduction of bid subsidies for good past performance appears to benefit the buyer/taxpayer by increasing competition for incumbency, driving winning bids down sufficiently to offset the potential increase in procurement costs due to bid subsidies.


Considered together, the findings in Butler et al. (2013) suggest that there need not be a trade-off between reputation and entry in procurement, and that the debates both in the EU and the US are rather misplaced. The results suggest that the dual goals of providing incentives for quality provision and increasing entry and cross-border procurement – in the EU or elsewhere – are achievable through an appropriately designed reputational mechanism. Policy makers should therefore probably stop quarrelling about whether a generic past-performance based reputational mechanism should be introduced, and instead focus on how such a mechanism should be designed.


  • Butler Jeff, Carbone Enrica, Conzo Pierluigi and Giancarlo Spagnolo. 2013. “Reputation and Enty in Procurement.” CEPR Discussion Paper No. 9651.
  • Calzolari, Giacomo and Giancarlo Spagnolo. 2009. “Relational Contracts and Competitive Screening.”  CEPR Discussion paper No. 7434.
  • European Commission. 2011. Green Paper on the modernisation of EU public procurement policy Towards a more efficient European Procurement Market. Available at http://ec.europa.eu/internal_market/consultations/index_en.htm
  • European Commission. 2011. Green Paper on the modernisation of EU public procurement policy Towards a more efficient European Procurement Market. Synthesis of Replies.  Available at http://ec.europa.eu/internal_market/consultations/docs/2011/public_procurement/synthesis_document_en.pdf
  • Government Accountability Office. 2011. (GAO-12-102R, Oct. 18, 2011). Prior Experience and Past Performance as Evaluation Criteria in the Award of Federal Construction Contracts. Available at  http://www.gao.gov/products/GAO-12-102R
  • Kelman, Steven. 1990.  Procurement and Public Management: The Fear of Discretion and the Quality of Government Performance: American Enterprise Institute Press.
  • Yukins, Christoher. 2008. “Are IDIQS Inefficient? Sharing Lessons with European Framework Contracting.” Public Contract Law Journal,  37(3): 545-568.