A Decade of Russian Cross-Domain Coercion Towards Ukraine: Letting the Data Speak

Russia’s coercion towards Ukraine has been a topic of major international events, meetings and conferences. It regularly makes the headlines of influential news outlets. But the question remains open – do we really understand it? We diligently collect and analyze data to make informed decisions in practically all domestic issues but is the same done for international relations? This research paper introduces a number of tools and methods that could be used to study Russia’s coercion towards Ukraine beyond its most visible manifestation, looking into latent trends and relations that could reveal more.

Introduction

For the past decade, Ukraine has been in the headlines of the major world news outlets more frequently than ever before. Ukrainian-Russian relations have been and still remain the topic of international summits and other events. The occupation of a part of Ukraine’s territory has been denounced and Russia’s coercion towards Ukraine is now generally accepted as a fact. But what do we really know about the underlying empirics and dynamics and how can this multi-domain assertiveness be measured and tracked? This paper presents a number of data-driven approaches that allow looking beyond the headline stories to identify and track various dimensions of Russia’s coercion towards Ukraine and the dynamics of its development.

Academic Interest

Mapping the landscape of scholarly literature reveals a number of interesting results. First, the body of works studying Russia’s coercion towards Ukraine remains relatively modest. It quintupled in 2014 but afterwards the interest started tapering off. A search for papers on this topic in Scopus and Web of Science with a very precise query (to increase the accuracy of search) and publication time of 2009-2019 returned 155 papers most of which were published in or after 2014.

Figure 1. Scholarly publications on Russian-Ukrainian relations.

Source: WoS and Scopus, 2009-2019

A closer look at the content of these works with the use of a bibliometric software called CiteSpace shows that the majority of papers focus on Putin, once again emphasizing the significant role of his personality in Russia’s coercion towards Ukraine. The second largest cluster has the “great power identity” as its main theme and presumably looks beyond actions of Russia to identify its ideological grounds. Another group of publications is devoted to sanctions, pointing to their important role in studying Russian-Ukrainian relations.

Figure 2. The landscape of topics in scholarly publications on Russian-Ukrainian Relations.

Expressions of Coercion

The “practical” side of Russia’s coercion towards Ukraine is also frequently associated with the personality of Vladimir Putin and his attitudes towards Ukraine. To analyze this perception further, we created a corpus of speeches of Russian presidents published on the Kremlin website, filtered them to keep only those that mention Ukraine, divided them into pre-2014 and 2014 and after, and then analyzed them using an LDA topic modeling algorithm. This algorithm is based on the assumption that documents on similar topics use similar words. So, the latent topics that a certain document covers can be identified on the basis of probability distributions over words. Each document covers a number of topics that are derived by analyzing the words that are used in it. In simple terms, the model assigns each word from the document a probabilistic score of the most probable topic that this word could belong to and then groups the documents accordingly.

Figure 3. Speeches of Russian presidents before 2014, LDA topic modeling.

Figure 4. Speeches of Russian presidents in 2014 and after, LDA topic modeling.

Quite surprisingly, we discovered that the overall rhetoric of speeches is very similar for the two periods. Although some speeches do differ and the later corpus includes new vocabulary to reflect some changes (i.e “Crimea”, “war”) the most common words remain practically the same. Thus, regardless of the apparent shift in relations between the two countries, Russian leadership still relies on the same notions of collaboration, interaction, joint activities, etc. The narrative of “brotherhood” between the nations persists despite and beyond the obvious narrative of conflict.

To include a broader circle of Russia’s leadership we also looked at the surveys of the Russian elite conducted regularly by a group of researchers led by William Zimmerman and supported by various funders over the years (in 2016 – the National Science Foundation and the Arthur Levitt Public Affairs Center at Hamilton College). Seven waves of the survey already took place; the most recent one in 2016. The respondents were the representatives of several elite groups (government, including executive and legislative branches, security institutions, such as federal security service, army, militia, private business and state-owned enterprises, media, science and education; for practical reasons from Moscow only).

The survey revealed a number of interesting observations. For instance, while the prevailing Russian opinion on Russia’s occupation of Crimea had been that it was not a violation of international law, a closer look at the demographic characteristics of respondents shows that they were not as coherent as it might seem from the outside. While the “green” answers from respondents with backgrounds such as media or private business may have been anticipated, the number of members of the legislative and especially executive branch and the military that had at least some doubt on the legality was surprisingly quite sizable, and they even demonstrated some support of the “violation of law” interpretation.

Figure 5. Elite and public opinion on Russia’s annexation of Crimea.

Comparing these elite opinions to the public opinion poll by Levada Center conducted in the same year shows that even the general public is slightly more likely to choose the most extreme “full legality” option than the respondents from the executive branch.

Beyond the elite or general opinion polls, we tried to develop a metric that might allow us to track Russian sensitivities towards Ukraine. For that, we examined two different ways of expressing “in Ukraine” in Russian language: ‘на Украине’ (the ‘official’ Russian expression) vs. ‘в Украине’ (the version preferred by Ukrainians). [In English, this can be compared so saying ‘Ukraine’ vs ‘the Ukraine’.]

Our first visual plots how many search queries were done on Google Search with both versions over the last decade.

Figure 6. Search queries for “в Украине” (green) versus “на Украине” (red), Google Trends, 2009-2019.

We can clearly observe that during less turbulent times the more politically sensitive version is much more common. This however drastically changes during the peaks of Russia’s coercion towards Ukraine when the number of searches with the less politically correct term increases significantly.

A different trend can be observed if we look at official media publications stored in the Factiva database. We estimated the ratio of search volumes for each term and observed that until the beginning of 2013, about a third of articles and news reports used “in Ukraine”. This changed around January 2013 when the ratio starts to decrease for “in Ukraine” searches and plummets to a mere 10% of outlets still preferring this term.

Figure 7. The ratio of “в Украине” to “на Украине” occurrences in large Russia media (2009 – 2019), Factiva.

Tracking Coercion Itself

What is the track record of Russia’s actual coercion over this decade? For this, we turn to a few recent datasets that try to systematically capture verbal and material actions (words and deeds): the automated event datasets. The largest one of those, called GDELT (Global Database of Events, Language, and Tone), covers the period from 1979 to the present, and contains over three quarters of a billion events. It is updated every fifteen minutes to include all “events” reported in the world’s various news outlets. To exclude multiple mentions of the same event by one newswire, the events are “internally” deduplicated. The events are not compared across newswires.

An event consists of a “triple” coded automatically to represent the actor (who?), the action (what?) and the target (to whom?) as well as a number of other parameters such as type (verbal or material; conflict or cooperation; diplomatic, informational, security, military, economic), degree of conflict vs cooperation etc. Other similar datasets include ICEWS (Integrated Crisis Early Warning System) and TERRIER (Temporally Extended, Regular, Reproducible International Events Records). For this analysis, we filtered out only those events in which Russia was the source actor and Ukraine was the target country. We present two metrics: (1) the percentage of all world events that this subset of events represents and (2) the monthly averages of the Goldstein score, which captures the degree of cooperation or conflict of an event and can take a value from -10 (most conflict) to +10 (most cooperation). Also, to add a broader temporal perspective, we looked beyond the last decade. It can be clearly seen that the number of events before 2013 was significantly lower, especially in “material” domains. Some verbal assertions from Russia towards Ukraine happened during the Orange Revolution and so-called “gas wars”.

The situation changes radically starting from 2013. The proportion of events increases with some especially evident peaks (i.e. during the occupation of Crimea). The verbal events remain quite neutral while the actions towards Ukraine move from some fluctuations to steadily conflictual.

Figure 8. Russia’s negative assertiveness towards Ukraine, 2000-2019.

Measuring Influence

We have seen that the past decade was exceptional in the scale of Russian assertiveness towards Ukraine. But what do we know about Russia’s influence on Ukraine and Ukraine’s dependence on Russia? Influence measures the capacity of one actor to change the behavior of the other actor in a desired direction. In an international context this often concerns the relations between countries. Influence can be achieved by various means, one of which is to increase the dependence of the target country upon the coercive one. This strategy is frequently employed by Russia willing to regain and/or increase control over the former post-Soviet countries. The Formal Bilateral Influence Capacity (FBIC) Index developed by Frederick S. Pardee (Center for International Future) looks at several diplomatic (i.e. intergovernmental membership), economic (trade, aid) and security (military alliances, arms import) indicators allowing to identify the level of dependence of one country upon another. This is especially interesting from a comparative perspective. Figure 9 shows that countries such as Armenia and Belarus remain highly dependent on Russia. For half of the decade, Ukraine was number three on this list. Today the situation has changed. Ukraine’s dependence on Russia has gradually decreased and has become even smaller than Moldova’s, moving closer to the steadily low level of dependence of Georgia. This may signify a positive trend and a break of a decade-long relationship of dependence.

Figure 9. Dependence of post-Soviet countries on Russia, FBIC.

Conclusion

Consequently, Russia and Ukraine have become much more visible in the international academic and policy research efforts. This can be measured through a number of instruments, including a comprehensive mapping of the academic landscape itself with regard to salience and topics that are being studied, analysis of the word choice (that could be represented by the use of the terms to describe events in Ukraine by the government media and Google search users (“на Украине” versus “в Украине”); speeches of Russian presidents that use the same rhetoric of collaboration when talking about Ukraine despite the obvious change in relationships) and material coercion (significant increase in number of assertive conflictual Russia’s actions towards Ukraine). Some findings do give hope for change: the opinions of the Russian elite on recent Russian actions towards Ukraine while remaining generally unfavourable are not as cohesive as it might appear and Ukraine’s dependence on Russia has decreased significantly.

Disclaimer

This research is a part of a larger research effort titled RuBase funded by the Carnegie Foundation of New York and implemented jointly by The Hague Centre for Strategic Studies and Georgia Tech with the help of the Kyiv School of Economics StratBase team in Ukraine. The ‘Ru’ part of the title stands for Russia; and ‘base’ has a double meaning – both the knowledge base built during the project, and the (aspirationally) foundational nature of this effort. The project intends to look beyond the often-shallow traditional understanding of coercion and apply innovative tools and instruments to study coercion in its multifaceted form. This is only a small selection of the tools that have been successfully tested in the course of this (ongoing) research project and applied to the study of Russia’s coercion in different domains. The prospects of any progress in resolving the Russian-Ukrainian conflict are currently slim, thus further work that would allow identifying patterns and trends that the human eye may oversee to understand Russia better and develop an informed foreign policy strategy both for Ukraine and the West is crucially important.

References

• Boschee, Elizabeth et al. (2019). “ICEWS Automated Daily Event Data.” (November 12, 2019).
• Clarivate Analytics (2019). “Web of Science Core Collection.” Web of Science Group. (January 20, 2020).
• Dow Jones (2019). “Factiva – Global News Monitoring & Search Engine.” Dow Jones. (December 2, 2019).
• Elsevier (2019). “Scopus.” (December 3, 2019).
• Google (2018). “Google Trends – The Homepage Explained – Trends Help.” (January 20, 2020).
• Holynska, Khrystyna, Yevhen Sapolovych, Mikhail Akimov, and Stephan De Spiegeleire (2019). “Events Datasets and Strategic Monitoring: Method Piece” (Forthcoming). The Hague Centre For Strategic Studies.
• Levada-Center (2019). “Levada Center.” (December 3, 2019).
• Moyer, Jonathan D., Tim Sweijs, Mathew J. Burrows, and Hugo Van Manen (2018) “Power and Influence in a Globalized World.” Atlantic Council. (November 26, 2019).
• OU Event Data (2018). “Terrier (Temporally Extended, Regular, Reproducible International Event Records)”. (January 29, 2020).
• The GDELT Project (2015). “GDELT 2.0: Our Global World in Realtime.” GDELT Blog. (October 11, 2018).
• Zimmerman, William, Sharon Werning Rivera, and Kirill Kalinin. (2019). “Survey of Russian Elites, Moscow, Russia, 1993-2016”. Version 6.” (November 26, 2019).
• Президент России (2019). “Президент России.” Президент России. (November 26, 2019).