1. Introduction

A Nature commentary De Bruin, Braam & Moed (1991) analyzed the scientific literature from the Gulf region to show that the political developments of the 1980’s, culminating in Operation Dessert Storm, were clearly visible in the Web of Science. Similarly, Igić (2002) and Lukenda (2006) studied the influence of the civil war in Yugoslavia (1991-1995) on local scientific production. Ever since, bibliometricians have analyzed collaboration patterns as evidenced through co-authorship of scholarly publications to study their co-evolution with political and geopolitical developments. Recently these types of studies have attracted renewed interest as (1) geopolitical tensions have been rising, and (2) the Russia-Ukraine war has triggered a series of political reactions, including sanctions impacting the possibility of continuation of long­standing scientific collaborations. Indeed, a recent survey by Digital Science emphasizes that although collaboration is booming, so are concerns around research security and the possibility of damaging collaborations (Digital Science, 2024).

With regard to Russia, Makkonen & Mitze (2023) calculated a 15% reduction in scholarly collaboration between the EU and Russia as a result of the sanction regimes that kicked in after the start of the first Russia-Ukraine war phase in 2014. Shortly after the start of the second war phase in 2022, Plackett (2022) described the difficult considerations of individual researchers while ongoing funded collaborations had to wind down. Briefly afterwards, the European Commission suspended all Russian participation to the European framework programs for research and innovation, as did many national research foundations in the EU and beyond.

In their analysis of Russian research publications and collaboration, Zhang et al. (2024) conclude that since the start of the current phase of the Russia-Ukraine war in 2022, the Russian scholarly production has dramatically decreased. Moreover, they observe a decline in co-authorship in fields of research supported by large infrastructures established through state agreements. In terms of relative intensity of collaboration, they report a gradual decline of collaboration with Germany over the last 20 years, whereas collaboration with China and India has been on the rise, especially in recent years.

Using the relative intensity of collaboration, Smolinsky & Yang (2024) zoom in on Russia-Ukraine scholarly collaboration until 2020. They report a steep rise in collaboration until 2011, followed by a plateau at a very high intensity of collaboration for about 4 years, and a decline from 2016 onwards. Clearly, these developments can be linked to the first phase of the Russia-Ukraine war that started in 2014 with the invasion of Crimea and parts of the Donbas region in the east of Ukraine.

Smolensky & Yang (2024) deliberately did not analyze more recent publication years, as the effects of geopolitical developments, even if as impactful as war, may take several years to show up in bibliometric data, given the time needed for publications to go through the publication cycle. In a more recent analysis, however, Hladchenko (2025) shows that Ukrainian collaboration has been drastically shifting towards countries like Poland and away from Russia over the years 2020 to 2023.

A limitation of the aforementioned studies is that they consider all collaboration equal. However, whether collaborations are bilateral or multilateral can have significant implications in a context where geopolitics force a rethink of research collaborations. While all collaborations may be impacted by pressures from colleagues, institutions, politics and/or the public, as well as by practical issues such as sanction regimes, bilateral collaborations depend mostly on the individual researchers involved. Larger multilateral collaborations, however, depend more on political factors and may be more difficult for an individual researcher to abandon. Indeed, Zhang et al (2024) already pointed out that in the case of collaboration with Russia especially large infrastructure bound collaborations, that operate under formally concluded state agreements, seem to be impacted. Smaller scale bilateral collaborations may however be even more impacted in case the (potential) co-authors themselves feel that the context for collaboration has changed. As geopolitical tensions have been rising, these researchers may also ‘vote with their feet’ and decide to suspend collaboration, or may be forced to do so due to a variety of practical, institutional or community reasons.

Therefore, in this paper, we distinguish the relative intensity of collaboration (Fuchs et al., 2021) for two different segments of papers:

• Papers that involve co-authors from Russia and only one other country;
• Papers that involve co-authors from Russia and at least two other

Moreover, we distinguish collaborations that align with geopolitical fault lines in different subsets. Specifically, we analyze the evolution of Russian collaboration with

• Belarus, a major ally of Russia bordering the European Union. As Belarus has allied itself explicitly with Russia, is it also becoming a more important partner of Russia in research?
• How has the relative intensity of research collaboration evolved for bilateral and multilateral collaborations over the last 30 years?
• European countries that border Russia, Belarus, and/or the Black Sea, namely Norway, Finland, Estonia, Latvia, Lithuania, Poland, Slovakia, Hungary, Romania, and By and large, we expect the ongoing war in and with Ukraine and the sanction regimes that set in since to have harmed Russian collaboration with those countries. Note that we have not included Moldova in our study, because of the de-facto split between the main, Europe tilting part and the Russian-oriented province of Transnistria.
• Other large European countries, namely France, Germany, Italy, Spain and the For those countries too we expect a downward trend in Russian collaboration.
• As well as the US and China as the two largest producers of research papers in the world, with different approaches to Russia and the war in Ukraine until at least the end of

As such, we aim to present novel insights into the evolution of bilateral and multilateral collaboration with Russia since 1995, with a focus on the most recent years since the second phase of the war with Ukraine that started in February 2022.

2. Data and methods

We obtained 10,371,123 co-authored publications from OpenAlex (Priem et al., 2022), specifically the data dump of 22 January 2025. We selected publications that 1) appeared between 1995 and 2024 (inclusive), 2) have document type article, 3) are not ‘paratext’ (i.e. are not about the publication venue), and 4) have been cited at least once. Among all publications in our dataset, 7,973,372 are bilaterally collaborated papers (coauthored by two and only two countries), whereas 2,397,751 of them are multilateral collaborations (coauthored by more than two countries).

We show the temporal evolution of papers in the dataset in Figure 1. The decline in the number of papers per year for the most recent years is mostly due to the requirement that articles have been cited at least once. Even though this requirement clearly restricts the dataset and might impact different subsets differently (e.g. a bilateral paper with Belarus might be less likely to get cited within one or two years after publication than a bilateral paper with the US), we maintain it to ensure that our analysis includes only research articles.

Figure 1: Temporal evolution of the number of bilaterally coauthored papers and the number of multilaterally coauthored papers in the dataset, 1995-2024.

The number of bilaterally and multilaterally coauthored papers with Russia per partner country is presented in Table 1. The Baltic countries Estonia (n-bilat = 606), Latvia (n-bilat = 886), and Lithuania (n-bilat = 453), as well as Romania (n-bilat = 774) account for a relatively small number of bilaterally coauthored papers with Russia, while the numbers of papers coauthored with colleagues in the US (n-bilat = 41,855), Germany (n-bilat = 30,215), and France (n-bilat = 14,145) are substantial. For the multilaterally coauthored papers, the Baltic countries account again for the smallest numbers of papers (1,471 for Latvia, 2,370 for Lithuania, and 2,655 for Estonia), while the US (n-multilat = 53,546), Germany (n-multilat = 45,397), and France (n-multilat = 29,969) are the countries most involved in multilateral collaboration with Russia.

For each year from 1995 to 2024, we calculate Russia’s relative intensity of collaboration (RIC; Fuchs et al., 2021) with the countries in focus using both the subset of bilaterally coauthored papers and the subset of multilaterally coauthored papers. The RIC indicator compares the share of country Y within the collaboration profile of country X (Russia, in this paper) to the share of Y within all collaborations in the whole network. The RIC indicator is asymmetric (we here study it from the viewpoint of Russia only) and increases in value if collaboration between two countries increases. In this study, we adopt the publication-based interpretation of the relative intensity of collaboration (Fuchs & Rousseau, 2021), which focuses on assessing the contribution of each collaborative publication to a country’s total research output and implies that changes in these publication counts directly reflect shifts in the substantive strength of Russia’s international partnerships.

We here introduce a novel implementation of the RIC indicator, by distinguishing between bilateral and multilateral collaborations. In doing so, our results provide a more refined answer to the study of the evolution of Russian collaboration. For example, if the country Y is the UK:

• RICbila: A bilaterally coauthored paper with Russia is RICbila(Russia,UK) times more likely to include a coauthor from the UK as a bilaterally coauthored paper without an author from Russia;
• RICmulti: A multilaterally coauthored paper with Russia is RICmulti(Russia,UK) times more likely to include a coauthor from the UK as a multilaterally coauthored paper without an author from Russia.

Note that in this context ‘an author from’ means an author mentioning an affiliation in that country.

3. Results

We discuss the results presented in Figure 2 for each of the (group of) countries under focus.

3.1 Belarus

Over the last thirty years, the collaboration of Russia and Belarus has become more and more intense, to the current point of integration that implies that a bilaterally coauthored paper with Russia is over 30 times more likely to include a coauthor from Belarus than a non-Russian coauthored paper. Multilateral collaboration involving Russia and Belarus is also at a high level, much higher than with any of the other countries under study.

3.2 Ukraine

For Ukraine we find a very different pattern. Until 2014-2015 collaboration is increasing, with high probabilities of bilateral and even slightly higher probabilities of multilateral collaboration. Since then, the probability of collaboration has been in decline, very markedly for bilateral collaboration and somewhat more gradually for multilateral collaboration. Until 2020 this pattern is in line with what Smolinsky & Yang (2024) reported based on Scopus data. We add that, given the intensity of the war between Russia and Ukraine since early 2022, the decline in collaboration has continued ever since, and this is most clearly for bilateral collaborations.

3.3 European border countries

The European countries bordering Russia, Belarus, and/or the Black Sea (Norway, Finland, Estonia, Latvia, Lithuania, Poland, Slovakia, Hungary, Romania, and Bulgaria) have all been directly impacted by the war in Ukraine. Indeed, for some of those countries the threat of also being invaded by Russia looms large, while most have a pivotal role in connecting Ukraine to the rest of Europe, be it as a location for refugees, or as a source of humanitarian and military support. Overall, the probability of collaboration is clearly not as high as with Belarus, nor as it was with Ukraine. Still, some probabilities stick out, for example the fact that coauthors from Bulgaria or Latvia were up to four times as likely to be involved in a bilateral paper involving Russia, than in other bilaterally coauthored papers. It is also noticeable that, while for most of these border countries the probability of bilateral collaboration has been decreasing, the probability of collaboration with Hungary or Slovakia has steadily increased over the last 15 years. The pattern of the probabilities of multilateral collaboration is more diffuse, with a clear peak around 2012, followed by a gradual decline in multilateral collaboration involving European countries bordering Russia, Belarus, and/or the Black Sea.

3.4 Other large European countries

Upon inspection of the evolution of the probabilities of bilateral collaboration with other large European countries (France, Germany, Italy, Spain, and the UK), the most striking is the rather stable yet low probability of collaboration with Spain, the UK, Italy and France, while collaboration with Germany used to be above 1 until 2016. From 2019 onwards, and hence well before the official announcement of the German Zeitenwende regarding collaboration with Russia, the probability of bilateral collaboration with Germany has halved. The intensity of bilateral collaboration with the UK started to fall already from 2015 onwards. For multilateral collaboration we find higher probabilities of collaboration from 2012 onwards, followed by slight decreases in the most recent years.

3.5 US and China

In terms of collaboration with the US and China, the first observation is that the probability of a bilateral paper with a co-author or co-authors from Russia to involve a co-author from either the US or China is below 1. The evolution of the probabilities is, however, markedly different. Indeed, the gradual and recently very steep increase of the probability of the involvement of a co-author based in China contrasts with the slightly decreasing probability of the involvement of a co-author based in the US. In terms of multilateral collaboration, the pattern is different, illustrating until 2012 a gradually increasing integration of Russia in the global science system, followed by a more recent phase in which the probability of involvement of coauthors in China increases, while the probability of coauthors in the US slightly decreases.

Figure 2: Relative intensity of collaboration for five (groups of) countries: (a) Belarus, (b) Ukraine, (c) European countries bordering Russia, Belarus, and/or the Black Sea, (d) other large European countries, and (e ) China and the US. The vertical black dashed lines indicate the years 2014 (start of first war phase) and 2022 (start of second war phase).

4. Discussion and conclusions

This study traces both bilateral and multilateral research publications between Russia and key partner countries over the period 1995-2024 using the relative intensity of collaboration (RIC) indicator. The results reveal that collaboration with Belarus has become increasingly intense; a paper coauthored with Russia is now more than 30 times as likely to include a Belarusian coauthor compared to papers without Russian involvement. In contrast, Russian–Ukrainian collaboration increased until around 2014–2015 before dropping steeply, especially for bilateral papers. Other European countries bordering Russia, Belarus, and/or the Black Sea show mixed patterns; some, such as Bulgaria and Latvia, have high yet falling bilateral collaboration rates, while for Hungary and Slovakia collaboration seems on the increase. For other large European countries the intensity of collaboration is more variable, falling fast for Germany and more slowly for the UK, while remaining more stable for France, Italy and Spain. Meanwhile, Russia’s collaboration with the US and China exhibits a lower relative intensity (with the bilateral indicator remaining below 1), although the temporal evolution of these collaborations has diverged, particularly since 2022. While the probability of collaboration with the US shows a slight downward trend, the probability of collaboration with China has increased markedly.

So far, we have only analysed the relative intensity of collaboration from the perspective of Russia. In further analysis we will also study how the countries in focus have seen their probability of collaboration with Russia evolving. This is important as RIC is an asymmetric indicator.

Second, we position that a comparative perspective will enrich the analysis. One possibility is to add other Asian countries, from Turkey to Japan, to our analysis. India, for example, seems to be more and more involved in collaboration with Russia (Zhang et al, 2024). Another possibility might be to contrast the case of Russia with that of another recent important geopolitical event in Europe, e.g. Brexit, which included among other things an interruption of the participation of the UK in the European framework programmes for research and innovation.

Third, the constellation of countries in Europe is such that the notion of a national research system may hold to a limited extent only in most fields of research (Sørensen & Schneider, 2017). Hence bilateral and multilateral collaboration may reflect the necessity to collaborate with neighbouring countries in order to tackle certain research questions. Further refinement of the notion of multilateral collaboration might illuminate how research collaboration is evolving in the current context of geopolitical turmoil.

We conclude that distinguishing between bilateral and multilateral research collaboration helps in understanding the consequences of geopolitical developments. We find that in the case of Russia and in the wake of the Russia-Ukraine war so far especially Russian bilateral research collaboration has been be impacted.

Open science practices

This study uses openly available data from OpenAlex, specifically the 20250122 version of the data dump. The complete analysis code is shared on GitHub at https://github.com/ZHOUHONGY/ric_russia.

Acknowledgments

The authors thank the Flemish Government for its support of the Flemish Centre for R&D Monitoring (ECOOM).

Author contributions

TE: Conceptualization, methodology, writing – original draft; writing – review & editing; RR: formal analysis, methodology, writing – review & editing; HZ: data curation, formal analysis, software, visualization, writing – review & editing.

Competing interests

The authors have no competing interests.

References

De Bruin, R. E., Braam, R. R., & Moed, H. F. (1991). Bibliometric lines in the sand. Nature, 349(6310), 559–562. https://doi.org/10.1038/349559a0

Digital Science, Hahnel, M., Porter, S., & Delevante, R. (2024). Research transformation: Change in the era of open, AI and impact. Digital Science. https://doi.org/10.6084/M9.FIGSHARE.27193923

Fuchs, J. E., & Rousseau, R. (2021). How to Calculate the Relative Intensity of Collaboration (RIC) for Countries from Web of Science Data. ISSI Newsletter, 17(3), 55–60.

Fuchs, J. E., Sivertsen, G., & Rousseau, R. (2021). Measuring the relative intensity of collaboration within a network. Scientometrics, 126(10), 8673–8682. https://doi.org/10.1007/s11192-021-04110-x

Hladchenko, M. (2025). International collaboration of Ukrainian scholars: Effects of the Russian full-scale invasion of Ukraine. Submitted.

Igić, R. (2002). The influence of the civil war in Yugoslavia on publishing in peer-reviewed journals. Scientometrics, 53(3), 447–452. https://doi.org/10.1023/A:1014833315145

Lukenda, J. (2006). Influence of the 1991-1995 war on Croatian publications in the MEDLINE database. Scientometrics, 69(1), 21–36. https://doi.org/10.1007/s11192-006-0136-7

Makkonen, T., & Mitze, T. (2023). Geopolitical conflicts, sanctions and international knowledge flows: EU–Russia collaboration during the Ukraine crisis. The World Economy, 46(10), 2926–2949. https://doi.org/10.1111/twec.13421

Plackett, B. (2022). The future of research collaborations involving Russia. Nature. https://doi.org/10.1038/d41586-022-00761-9

Priem, J., Piwowar, H., & Orr, R. (2022). OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and concepts. https://doi.org/10.5281/ZENODO.6936227

Smolinsky, L., & Yang, S. (2024). Validity and bias of indicators of international collaboration: A theoretical analysis with an empirical study of Ukraine-Russia-United States and China-United States. Journal of Informetrics, 18(1), 101488. https://doi.org/10.1016/j.joi.2024.101488

Sørensen, M. P., & Schneider, J. W. (2017). Studies of national research performance: A case of ‘methodological nationalism’ and ‘zombie science’? Science and Public Policy, 44(1), 132–145. https://doi.org/10.1093/scipol/scw043

Zhang, L., Cao, Z., Sivertsen, G., & Kochetkov, D. (2024). The influence of geopolitics on research activity and international collaboration in science: The case of Russia. Scientometrics, 129(10), 6007–6021. https://doi.org/10.1007/s11192-024-04984-7