Tag: World Bank

Buyer Competence and Procurement Renegotiations

Posted on December 2, 2019 | Policy Brief

20191201 Buyer Competence and Procurement Renegotiations FREE Network Policy Brief Image 01

This brief deals with the extent to which a more competent public bureaucracy can contribute to better economic outcomes. It addresses this question in the context of public procurement, governments’ purchase of goods and services from private contractors, which accounts for about 15% of GDP in most economies and is on the rise. The efficiency of the procurement process directly influences the prices and quality of many government-provided goods and services that are crucial to social welfare objectives and sustained economic growth. Several issues challenge this efficiency. Media attention is typically on episodes of corruption, which can of course be a major source of waste. Here, we focus on a less glamorous, often overlooked, but potentially even more important source of waste, the lack of procurement competence.

Public procurement is a complex task. Contracting authorities must know market characteristics, design and implement efficient award mechanisms, balance risks and incentives in drafting contracts, effectively manage the contracts in the execution phase, etc. Effective procurement, in particular for complex services or works, requires teams endowed with legal, marketing, engineering, and economic/strategic expertise. The World Bank‘s Benchmarking Public Procurement 2017 compares the quality of the legal and regulatory environments of 180 countries and reveals the existence of great heterogeneity in the quality of the procurement processes across countries. Saussier and Tirole (2015) focus on the case of France, documenting that 63% of the staff of French contracting authorities do not have a legal profile, and only 39% have qualifications specific for managing public purchases.

Recent research focusing on prices of standardized goods showed that (lack of) buyer competence among public buyers could make an even bigger impact on the waste of public funds than corruption. For example, Bandiera, Prat and Valletti (2009) estimate that Italian public buyers would save 21 percent of their expenditures if they all paid the same as the buyers at the 10th percentile of the estimated procurement price distribution. Savings could reach 1.6-2.1 percent of the Italian GDP per year. They then estimate that bureaucratic inefficiency also linked to incompetence is the main cause of waste, accounting for 83 percent of total estimated waste, compared to only 17 percent due to corruption. In a similar vein, Best, Hjort and Szakonyi (2017) report that over 40 percent of within-product price variation on standardized goods in Russia in 2011-2015 can be ascribed to the bureaucrats and organizations in charge of procurement. They estimate that if the least effective quartile of bureaucrats and organizations had the effectiveness of the 75th percentile, the Russian government would save around $13 billion per year – roughly one fifth of the total amount spent on health care by the Russian government at federal, regional, and municipal level combined.[1]

The role of competence in complex procurement

This problem is becoming even more serious now that, being under fiscal pressure after the crises, many governments are promoting the use of public procurement not only as a tool to save budgets – sometimes at the expense of quality – but also to achieve more complex objectives like fostering innovation, protecting the environment, and promoting social objectives, a multiplicity of goals that per se makes the procurement mission even more complex.

Little is known about the importance of procurement competence in more complex procurements, not least because it is very difficult to measure performance in these environments. In our paper (Decarolis et al. (2019)) we try to make a step in this direction by focusing on works and services, typically more complex than goods. We use data from the US, probably the country with the most well-developed system of production and certification of procurement competences. Thus, our estimates of the effect of lack of competences should provide a lower bound of most other countries.

We combine, for the first time, three large databases: contract-level data on procurement performance in the Federal Procurement Data System (FPDS); bureau-level data from a survey conducted by the Office of Personnel Management since 2002 on federal employees, the Federal Employee Viewpoint Survey (FEVS); and Federal Workforce Data (FedScope) containing information on characteristics of the public workforce at the employee level.

To quantify the extent to which the government-bureau-level competencies determine procurement outcomes, we use the first database to construct procurement performance measures and the second dataset to build measures of procurement offices’ competence. We then use the third database to construct instruments that help us addressing important endogeneity issues. Our identification strategy exploits the exogeneity of death events involving public officials to allow for a causal interpretation of bureau competence on procurement performance.

Measurement Challenges

Indeed, there are three main challenges that our analysis needs to overcome. The first is how to measure procurement performance. Unit price comparisons have been used for standardized goods, but they are not suitable for the more complex procurements we focus on as they are heterogeneous in many non-recorded dimensions and their contracts are often incomplete. We use FPDS instead to construct three proxies of performance based on time delays, cost overruns, and the number of renegotiations. Although the first two measures are widely used in the literature, we are careful to take into account that cost overruns and delays may be due to new or additional work requested by the public buyer, in which case they should not be viewed as indicative of a poor outcome. We, therefore, consider only those which have occurred to deliver the work or service that was originally tendered. The third performance measure, the overall number of renegotiation episodes, is new and aims at capturing Williamson’s “haggling costs,” which are a pure deadweight loss present whatever the reason behind the renegotiation and have been shown to be economically sizeable for complex contracts. Our data reveals a surprising and persistent heterogeneity along these three dimensions across US federal bureaus.

The second challenge is the measurement of bureaucratic competence. Other papers in the field have measured it using buyer fixed effects. We use a novel approach based on the mentioned survey of employees’ subjective evaluations (FEVS). The survey is extremely rich, and we chose the most general question as an overall measure for competence (How would you rate the overall quality of work done by your work unit?). Responses to this question should be seen as measures of the overall efficacy of the workflow and processes within the bureau, hence proxying for the ideal measure of competence on the many different aspects relevant to procurement. An extensive set of robustness checks support our idea of measuring competence through the FEVS data.

The third measurement problem is the association between more complex contracts and more competent buyers: the most competent buyers may consistently produce poor performance because they are allocated the most complicated procurements. This point is well illustrated in a case study showing that the performance of the agencies that are worst in terms of competence (the Department of Veterans Affairs and the Department of Justice) is superior to that of the two most competent agencies (the NASA and the Nuclear Regulatory Commission) in terms of both delays and cost overruns. This striking inversion indicates that any straightforward regression of performance on competence would grossly underestimate the impact of competence.

We, therefore, develop an instrumental variable strategy exploiting exogenous changes in competence. We use FedScope to build instruments for bureaus’ competence based on the deaths of specific types of employees: bureau managers and white-collar employees who are relatively young and earn a relatively high wage. The idea is that more competent offices adopt better managerial practices, routines and processes that are more resilient to risks, such that of an unexpected loss of a key employee, and less dependent on specific individuals. This is precisely what the first stage of our IV strategy documents. Our instruments perform well in terms of their statistical properties and they allow us to estimate a causal effect of bureau competence on procurement outcomes that is an order of magnitude larger than the corresponding OLS estimate.

Results

We find that one standard deviation increase in competence reduces the number of days of delay by 23 percent, cost overruns by 29 percent and the number of renegotiations by half. This implies that if all federal bureaus were to obtain NASA’s high level of competence – corresponding to the top 10th percentile of the competence distribution – delays in contract execution would decline by 4.8 million days, and cost overrun would drop by $6.7 billions over the entire sample analyzed. We also find a consistently negative effect of greater competence on the number of renegotiations: one standard deviation increase in competence causes 0.5 (39%) and 0.8 (71%) fewer cost renegotiations and time renegotiations, respectively.

Finally, we try to understand what exactly makes a bureau ‘competent’ using the FEVS data to identify three different components: cooperation among employees, incentives and skills. Separately estimating their causal effects is unfeasible with instruments like the two described above as the validity of the exclusion restriction, which can be argued to be satisfied when measuring a broadly defined notion of bureau competence, is unlikely to hold for more specific components. However, we provide multiple pieces of evidence suggestive that cooperation is the key driver behind the positive effects of bureau competence. This finding conforms with the view that successful procurement requires appropriate coordination of a multiplicity of tasks involving different individuals. We also consider the extent to which the role of cooperation is due to the presence of capable managers, able to lead a group to effective cooperation, exploiting the heterogenous effects obtained through instruments considering the deaths of different subgroups of employees. We find that the deaths that matter the most are those of relatively young and best paid white-collar employees.

These results point at the large potential improvement in the performance of public contracts that could be achieved by investing more resources in increasing the competence of contracting authorities, even in a country with long-established procurement training and certification institutions such as the US. In Europe, recent policy initiatives see the introduction of qualification systems for public procurers as a necessary response to the generally lower procurement competence coupled with the greater discretion granted by the 2014 Procurement Directives. Our results on the role of cooperation suggest that certification programs would be also useful at the level of the procuring office, and should include features such as the organisation of the acquisition process and the prevailing management practices, as is often done for private firms.

References

[1] See also, Bucciol et al (2017) who study procurement of standardized medical devices purchased by local Italian purchasing bodies, finding that the price for the same medical devices paid by Italian public buyers differ substantially, and that the differences are explained by ‘buyers fixed effects’ capturing all specific buyers characteristics, including their competence levels.

Disclaimer: Opinions expressed in policy briefs and other publications are those of the authors; they do not necessarily reflect those of the FREE Network and its research institutes.