Introduction

There has been a growing emphasis among policymakers and research funders on how to accelerate and augment the societal impact of public research. Science policy has evolved from a science-push rationale of increasing basic research (Bush 1945) over to a systems-based rationale with additional focus on interactions (Lundvall 1992) to a missions-oriented rationale with focus on addressing societal challenges (Mazzucato 2018; Boon and Edler 2018).

Arguably, the current focus on societal challenges and directionality in funding policy increases the importance of understanding how funding influences research and its outcomes (Kattel and Mazzucato 2018; Aagaard et al. 2022). In order to understand this social phenomenon, it is important to identify the mechanisms that can lead to societal outcomes (Gläser and Velarde 2018, Gläser and Laudel 2019).

Existing literature has taken a number of strides in opening up the ‘black box’ behind the outcomes of research, for example identifying ‘productive interactions’ (Molas-Gallart and Tang 2011; D’Este et al. 2018), different stages of research outcomes (Donovan and Hanney 2011) and types of impact processes or pathways (Joly et al. 2015). However, while much recent work has addressed causes and outcomes, little work has been done to identify how linkages between them function, and to unpack these linkages in ways that are amenable to empirical research. Tracing the processes connecting research funding with societal outcomes can contribute to our understanding of how funding can guide research towards societal goals.

Drawing on two in-depth cases of research projects that have received societally targeted funding and appear to have involved highly intensive academic/non-academic engagements, this study seeks to investigate the following research question: How does societally targeted funding promote societally relevant outcomes? Societal targeting is broadly understood as the ‘levers’ funders may use to influence the direction of research (Norn et al. 2024). More specifically, targeting dimensions of funding instruments are defined as ‘tangible, objective funding instrument features’ used by funders to express expectations related to societal goals for funded research (Ramos-Vielba et al. 2022). For the two cases, we follow research processes from ideation to intermediate outcomes and decision-making on further development.

Societally relevant outcomes refer to results of societally targeted research that have potential to generate societal impact. Impact is typically based on the development and implementation of an innovation and hence takes place afterwards (though in some cases, research results can have direct societal impact). The two cases we examine are fairly recent to help ensure high recall of events, and hence have not reached this stage of development. We are instead examining outcomes that indicate development towards commercial or societal application and we also consider the potential of reaching final stated goals, which are to develop new technologies or innovations that will lead to societal impact. Examples of results can be innovations, uptake of results by non-academics for further development, patents, follow-up development projects, or assessment by multiple project participants that the project has succeeded in generating applicable results.

Process-tracing is a method for studying mechanisms linking causes with outcomes (Beach and Pedersen 2019). Through interviews and document analysis, we trace how the funding shaped the two studied research projects, and subsequently promoted the development of societally relevant research results. We model the research process through a series of stages, that together seek to explain the overall process from research funding to societally relevant outcomes. These stages run from funding to formulation of research projects, generation of data and results, evaluation of research results, utilisation in further work, and societally relevant outcomes. Beyond demonstrating the societally relevant outcomes of research, the process-tracing approach used in this study provides policy-relevant information on how and for whom the studied research funding program works, and it helps to assess whether funding is actually associated with societal outcomes in a causal sense (cf. Schmitt 2020).

The study should be seen as exploratory, with a dual objective of trying to establish a linkage between societally targeted funding and societal outcomes and of learning about the funding and research process itself. Throughout the analysis, we draw on the collected data both to make the best possible case for linkages and at the same time to discuss limitations (typically in the form of weaknesses in the evidence or the plausibility of alternative explanations). While the detailed case studies lend empirical support to the claim that funding is linked to outcomes through a collaboration mechanism, we cannot claim that the identified process is the only process linking funding to societal outcomes, nor that the funding was the sole factor involved in generating the research outcome. It is also important to note that this is a within-case approach, where we explore data and evidence for each individual case, as opposed to some form of cross-case analysis.

We believe that our study can make three main contributions. First, the relationship between research funding and research outcomes is still uncertain (Gläser and Velarde 2018; Ramos-Vielba et al. 2022). Previous studies of this relationship tend to focus on the effects of funding on research practices and rarely take a step further to study the societal outcomes generated from this research. Second, studies of the societal outcomes of research often miss a ‘detailed characterisation and understanding of the funding that was used to achieve research goals, as a prerequisite to understand whether and how funding has actually made a difference to the research’ (Ramos-Vielba et al. 2022: 203). We aim to fill this gap by analysing the specific characteristics of one selected funding program and tracing how it may be connected to generation of societal outcomes. As a third contribution, we believe that the process-tracing methodology makes an important contribution to identifying linkages between funding and research outcomes.

The next section reviews relevant literature on the societal outcomes of research and the role of funding. Section 3 describes the process-tracing methodology underlying our study and the two cases analysed in the paper. Section 4 presents the analysis, and section 5 the discussion and conclusion.

Literature on the Societal Outcomes of Research and the Role of Funding

Societal impact of research has been defined as ‘contributions to addressing current and/or future social, environmental, economic, and other needs outside academia’ (D’Este et al. 2018: 752). In line with this, Bornmann (2013: 218) concludes that most approaches to societal impact measurement ‘are concerned with the assessment of (a) social, (b) cultural, (c) environmental, and (d) economic returns (impacts and effects) from results (research output) or products (research outcome) of publicly funded research’.

The setup of the Payback Model has a strong focus on the benefits of research, with less focus on unpacking collaboration and knowledge production processes in greater detail (Donovan and Hanney 2011). The model includes seven stages of the research process and five categories of benefits: 1) Knowledge, 2) Benefits to future research and research use, 3) Benefits from informing policy and product development, 4) Health and health sector benefits, and 5) Broader economic benefits (Donovan and Hanney 2011).

Instead of assessing attribution and outcomes, the SIAMPI framework focuses on ‘productive interactions’ between researchers and other stakeholders under the assumption that such interactions function as pre­conditions for social outcome generation (Spaapen and Van Drooge 2011: 213). Productive interactions are formally defined as ‘exchanges between researchers and societal actors in collaborative settings (networks) in which knowledge is produced and valued that is at the same time scientifically and socially robust and relevant’ (ibid.: 212) and may potentially be linked with scientific and societal impacts (D’Este et al. 2018). However, it is unclear what makes them productive in a causal sense.

Focus can also be placed on the actors and their contribution to interactions (Kok and Schuit 2012: 2). This includes ‘engagement of potential key users in research formulation’ and ‘during the production phase’, as well as ‘utilization efforts by investigators’ whereby project participants also take on the role of key users (ibid.: 9). Our study draws in particular on this focus on the role of the user in the research process.

A central premise of the ASIRPA framework is that ‘knowledge as such is not immediately useful but is made so through a series of transformations performed by different actors’ (Matt et al. 2017: 209). Hence, the outcomes of research are based on the engagement of networks of actors involved at different stages and playing a variety of roles (Joly et al. 2015: 444). The ASIRPA approach uses a standardised case study methodology, which involves both within-case analysis of the chronology and impact pathway of each case, and quantitative cross-case comparisons used to derive ideal-type impact pathways (Joly et al. 2015; Matt et al. 2017). The method is mainly descriptive at the level of individual cases, prioritising cross-case comparisons over in-depth single-case studies. This differs from process-tracing, in which mechanisms are formulated based on detailed within-case studies with the ambition to gradually generalise to homogeneous cases.

Our analysis is also inspired by a framework for transdisciplinary co-production developed by Polk (2015) that was designed during the process of establishing a transdisciplinary research center, comprising both academic, public, and private stakeholders. The framework consists of three phases of co-production: formulation, generation, and evaluation. Unlike Polk, we incorporate the three phases into a larger model that explicitly links the collaboration process with funding conditions, and furthermore, we study successful cases of outcomes generation to identify the mechanisms that connect collaboration with outcomes.

Our formulation of these collaboration mechanisms is further inspired by a longitudinal, qualitative case study by Thune and Guldbrandsen (2014) who find that preexisting networks tend to ease recruitment of partners, and that partners who have previously collaborated find it easier to set up formal agreements (ibid.: 984). Examining conditions that are perceived to facilitate and support collaboration, Sjöö and Hellström (2021) finds that existing networks often play a role in project initiation. Another finding that resonates well with our analysis is that goal convergence and goal clarity, as well as a need for complementarity and prior experience with collaboration, function as conditions that support collaboration. In line with this, Stier and Smit (2021) find that joint problem formulation is seen as critically important, as well as early-stage communication about goals, underlying reasons for participation, and discussions on how partners view the outcomes, outputs, and impact both in the short and longer run. In another study of university-industry partnerships, Ankrah et al. (2013) find that collaboration may be challenged by conflicts relating to secrecy, intellectual property rights, or legal agreements, as the consideration for academic freedom and openness risks being incompatible with industry partners’ strategic interests.

Our model is developed both based on earlier work on research processes, notably on productive interactions (Spaapen and Van Drooge 2011), the role of users (Kok and Schuit 2012), impact pathways (Joly et al. 2015) and the co-production research process in Polk (2015), and on the cases themselves. Table 2 presents a model of the process derived from our analysis. As outlined in the model, the process is triggered by societally targeted funding and leads to societally relevant outcomes. The process is modelled in four parts. First, partners formulate goals and research problems for the project. The second and third parts focus on project collaboration: partners’ engagement in the research and sharing, discussion and decision-making based on the results. Arrows indicate feedback loops between these two parts. Part four is utilisation of research results to pursue further development towards an innovation. Finally, this process leads to societally relevant outcomes. In the subsequent paragraphs, we outline the analysis underlying this model.

Method, Case Selection and Data Collection

Process-tracing is a method for studying causal mechanisms based on within-case studies (Beach and Pedersen 2019; Beach 2022). A causal mechanism can be defined as ‘a theorised link between a cause […] and an outcome, where each part of the mechanism is clearly described in an ordered sequence, and, in particular, in terms of entities engaging in activities that transfer causal forces’ (Beach and Pedersen 2018: 841). As we aim to understand and formulate how societally targeted funding may lead to societally relevant outcomes, we use the process-tracing logic to explore how these are linked.

This analysis is part of a larger study that examines societally targeted funding, its design and influence on funded research (see e.g. Aagaard et al. 2021, Aagaard et al. 2022, Ramos-Vielba et al. 2022). The larger study focuses on funded research in three countries (Norway, Denmark and the Netherlands) within two areas (renewable energy and food science). Case studies were conducted around a series of funded projects, involving desk research and semi-structured interviews with PIs of each project and other academic and non-academic participants. The interviews inquired about motivations to apply for the grant and what researchers saw as the distinctive value of the selected societal grant, and about research networks and practices prior to the grant and how these developed afterwards. In addition, the interviews covered interdisciplinary collaboration, transdisciplinary collaboration, goals and prioritisation of research problems and the degree of focus of research towards the development of user-oriented outputs.

For the analysis in this paper, we have selected two of these cases that have received societally targeted funding from the same funder and appear to have been successful in creating societally relevant outcomes, which allows us to explore and trace the linkages in between funding and outcome within each of the cases (Beach and Pedersen 2018; Beach and Pedersen 2019). For these two cases, we have conducted additional interviews with project participants and additional desk research in order to trace in greater detail research activities and decision-making over time. Although the projects are within different areas (renewable energy and food science), the projects share comparable scope conditions that may influence whether and how the mechanism works. These include the involvement of industrial partners with considerable experience in and capacity for in-house R&D activities and in academic collaboration, and with many employees with an academic background. An overview of the cases (including fictive project names) is presented in Table 1. Identifiers of the nine interviewees are added in brackets.

In the ENERGY project, the most central non-academic partners were a research and technology organisation (RTO) and a large, research-intensive company. Crop development companies were the key non-academic partners in the CROP project whereas a larger company with a different expertise only played a minor role.

We collected our empirical data through semi-structured interviews with both academic and non-academic partners from the two cases to cover different perspectives on each project. For the interviews, we selected partners who had been closely involved in the collaboration as they were more likely to provide detailed insights into the project collaboration and key decisions. To ensure recall of events, we selected projects that were finalised within the past three years. As mentioned above, it is still soon to determine further/long-term societal outcomes of each of the projects, but we are, however, able to observe intermediate outputs and short-term outcome generation. In addition to the interviews, we conducted text analyses of relevant funding calls and articles about the projects. The funding calls formed the basis of our categorisation of the societally targeted funding and analysis of how it functioned as a trigger of the process, whereas the articles (academic publications and news articles) informed the assessment of project outputs and emerging outcomes. For reasons of anonymity, these documents are not referenced in this paper.

Table 1. Description of cases.

	The ENERGY project	The CROP project
Field	Renewable energy	Food science
Project duration	5 years	5 years
Principal investigator (PI)	Academic researcher (E-PI)	Academic researcher (C-PI)
Academic partners	Other national academic partners (E-A)	National and international academic partners (C-A)
Non-academic partners	Large and small companies and RTO (E-NA1) (E-NA2)	Large and small companies (C-NA1) (C-NA2) (C-NA3)

Our analysis relies on an abductive process, where we build on empirical material from the two cases as well as existing literature. Initially, we outlined an empirical, within-case narrative of each of the selected research projects with a focus on the project collaboration. We then compared these narratives to find critical and similar events across our two cases, which could indicate underlying mechanisms (Beach and Pedersen 2019). We also assessed the confirmatory power of our empirical observations. We further included existing literature to identify central concepts, actors, and stages of research collaboration which may be relevant. Drawing on the literature above and based on our analysis of the two cases, we have formulated a mechanism which explains how societally targeted funding contributes to societally relevant outcomes through collaboration (Table 2). However, we do not claim that the societally targeted funding nor the outlined mechanism are sufficient to produce societal outcomes (Beach and Pedersen 2018). Furthermore, we only make claims about the mechanism explored and not about what happens when the trigger is not present (Beach and Pedersen 2018; Beach and Pedersen 2019).

Analysis

The Trigger: Societally Targeted Research Funding

Societally targeted funding instruments are conceptualised here as instruments that encourage or require transdisciplinary collaboration between academics and non-academics, and explicitly aim at facilitating societal outcome generation from the funded research.

Both projects were funded by the same funding instrument, which according to the funding call and the assessment criteria was directed towards the funding research and development work that was aimed at innovative solutions with economic and societal impact. Although it did not target projects at a specific technology readiness level (TRL), the guidelines specified that projects should involve a high level of risk and ambition to produce commercialisable innovations within prioritised societal areas or research competences that could support further development. Furthermore, applications were assessed based on the project’s commercial potential. The funding instrument thus explicitly aimed to promote societal outcomes in terms of improved competencies within prioritised research areas, product commercialisation, and wider economic goals.

In addition, the assessment criteria encouraged inter- or transdisciplinary collaboration, for example, ensuring the inclusion of needed competences to reach project goals and division of roles that facilitates interaction among partners throughout the project. Both academic and non-academic partners were eligible for funding at varying rates depending on the organisation type (and for private companies, also on the type of activities they were involved in, research or development). All partners were required to provide some co-funding.

This was met in both cases. We found that academic and non-academic interviewees in both cases perceived transdisciplinarity to be informally required. ‘[…] you need industrial collaborators, otherwise they [applications] don’t really go through. I don’t think they [the funder] state specifically that you must have it, but I mean, in reality you need that’ (C-PI). Likewise, the non-academic partners perceived academic collaborators to be necessary: ‘I don’t think that there is a rule which is written down, but I would guess there is, I mean, an unwritten rule that if you want to do research, you need people with CVs which have shown that they are capable of doing research’ (E-NA1). In both cases, academic and non-academic interviewees did not see the funding call as initiating the project idea and consortium, but instead as providing an opportunity for and enabling collaborative research projects.

The interviews suggested two possible explanations of why this funding opportunity triggered the proposed mechanism. First, the opportunity to get external funding triggered the partners’ preexisting interests, particularly project ideas that the non-academic partners wished to pursue. As both projects were initiated by the key non-academic partners, their motivations and interests were essential. In the ENERGY case, the key non-academic partner was driven by a strong interest in developing a new technology that they could later commercialise. The non-academic partners had already demonstrated the potential of this technology, although still at an early stage, in a precursor project. The technology needed further development, and the two partners decided to move on to a more research-intensive project that would require academic partners. The CROP project was initiated by the crop development companies who sensed a growing opportunity for working with a new crop. By doing the early-stage research in collaboration with academic partners, the companies wanted to gain insights that would enable them to start separate crop development programs with commercial potential and that would address societal challenges. ‘We wouldn’t have, probably not been able to do it just with internal [funding]. It always works better for us to be brought into those projects where […] we combine academics and the practical because we wouldn’t be able to do those projects without the scientists. And I think that’s always those public projects that serve best for that’ (C-NA2). Likewise, the academic PI stated that ‘What we need is actually either the [crop development companies] or the food industry to be interested in setting this up, because we cannot go to [funder of basic research] and develop this. We need to go to [funder] to actually develop this. It’s not sufficiently cutting edge, so to speak, that the [funder of basic research] would actually fund it.’

As a second plausible linkage between the funding opportunity and project initiation, interviewees indicated that external funding was needed to reduce risk of the projects, thereby making the non-academic partners more willing to invest. In both cases, non-academic partners stated that they would not have been willing to invest in this type of early-stage research without external funding. The research grants provided financial resources for the project activities and enabling the ‘manpower’ that was needed.

Based on the empirical analysis of the two case narratives and their similarities, we can observe the following:

• A non-academic partner had a promising idea for an innovation but was unable to do it without an academic partner, or vice versa
• Non-academic partners were not willing to invest in early-stage research without external funding
• Project partners perceived the societally targeted funding to provide an opportunity for collaborative, societally oriented research projects:
  • The funding call and assessment criteria explicitly encouraged transdisciplinary collaboration and targeted generation of societal outcomes
  • The funding conditions specified that both academic and non-academic partners were eligible for funding

The finding that the proposed funding conditions were explicitly stated in the funding call, guidelines, and/or assessment criteria is strongly confirmatory of societal targeting being present. In addition, project partners recognised the funding as an opportunity and that they at the same time perceived a lack of alternatives to external funding. Interviewees gave detailed and plausible explanations of why they were not willing to invest without external funding, hence supporting that the funding functioned as a trigger of the process.

Part 1: Partners Identify Common Goals and Jointly Formulate a Research Problem and Design to Achieve Them

Before applying for funding, the partners engaged in negotiations where they identified common interests and defined common goals. In both cases, the common goal was a commercial innovation that would address challenges due to climate change. Based on initial discussions in a larger forum, the CROP partners decided to initiate a joint project. The crop development companies had the same goal going into the project, namely, to adapt and improve a specific crop for local use. All interviewees agreed that, overall, the project was oriented towards this goal. The academic PI was driven by dual motivations, namely to advance the development of the crop while at the same time strengthen the expertise of their research group and achieve scientific results within plant genetics. ‘[T]his is part of this learning that I say that we only go into this if we think we can get something out of it from the basic science in terms of competence development also […] [name of academic colleague] has used this as a platform for a couple of new projects. So, I mean, it definitely has an influence on our activities. There’s no doubt about that’ (C-PI).

In the ENERGY case, a similar process took place as the partners identified common goals during pre-project negotiations, and all partners recognised that they were mutually dependent on each other: ‘[…] everyone had this interest in fulfilling the overall goal, but no one could do it on their own. There was this real need for doing it together’ (E-PI). Despite not being identical, the interests were aligned in the sense that the planned research activities would benefit both academics and non-academics: ‘It was a clear end goal going into the project […] to establish the scientific basis of this […] We wanted to have a commercial product by the end of the project, something which could be scaled, but we wanted to do this by understanding […] We constantly had the detailed […] academic understanding of the choices we made’ (E-NA2). ‘[…] I think every partner saw an own interest in doing this, so the industries could see this could be something, what they develop their field and what they could produce and sell in the longer run. But it was worthwhile for them doing it and the same for the research, it was like even if it doesn’t become a project or a product, in the end, it will still be meaningful for the development of our research’ (E-PI). ‘[…] I think it’s important to know from the beginning of a project that like the motivation from the different partners will not be the same. Publishing is important for academia. Making money is important for industry. And if you can, you know, align those goals, you can make a good project. And then you can even make changes to that project as long as you are aware of the main goals’ (E-NA1).

Although only mentioned by one interviewee (NA1), it was revealed in the interview material that the funder required that partners formally agreed on the distribution of potential patents prior to the research project. According to the findings by Ankrah et al. (2013), intellectual property rights hold a potential for conflicts between collaborators. Thus, the funder also played a central role in the initial alignment amongst the collaborators and prevention of future disagreement.

The interview accounts pointed to two causal logics plausibly connecting the formulation (part 1) and generation (part 2) of the mechanism. Firstly, both academic and non-academic interviewees in the two cases substantiated how the project’s goals and activities were designed to be relevant for all partners. This likely affected their level of commitment and motivated them to engage actively in research activities. Secondly, the empirical analysis suggests that having clearly defined goals and being aware of each other’s interests helped foster effective collaboration and prevented tensions and conflicts between the partners.

Based on the analysis, we can again identify empirical observations that help formulate part 1 of the proposed mechanism (Table 2).

• Prior to the project, partners engaged in pre-project negotiations and identified common as well as aligned goals
• The research proposal and project design were jointly developed and involved active participation by both academic and non-academic partners

Finding that partners engaged in pre-project discussions, identified common goals, and designed and wrote the funding application together, suggest that the projects were in fact jointly developed and designed to achieve goals that were relevant to all the partners. Ideally, interview accounts would have been supplemented by other data sources such as meeting minutes from the pre-project negotiations and drafts of the project proposals. However, the fact that all interviewees had similar perceptions of the process, and also recognised that their interests differed to some degree, lends credibility to the interview accounts as confirming evidence.

Part 2: Partners are Actively Engaged in Production of Research Results and Contribute with Necessary Expertise

Following the formulation phase, this part of the analysis moves to the generation phase and explores how the project partners were engaged in the research activities, collaboration, and production of research results. As outlined above, both projects were designed to enable academic and non-academic partners to benefit from their complementary expertise. This was reflected in the division of tasks and responsibilities and hence influenced the way the partners collaborated. In the ENERGY case, the academic researchers (including PhD students) did smaller-scale experiments while the non-academic partners were responsible for producing materials and testing them at a larger scale. ‘There’s been a lot of flow of information. We at [xxx] made a lot of experiments ourselves, but [the university] had a lot of PhD students [who] came to our lab and then used our equipment’ (E-NA1). The PI experienced the collaboration and interaction as ‘[…] very, very merged together. We also had meetings together. Then, of course, we did some things here at the university, or the companies did things, but there was a lot of information going back and forth and also, you know, some measurements could be done in one place and the other. […] It was not like just getting a material from a company and then measuring it. […] There is a lot more interacting’ (E-PI).

A division of roles and responsibilities was also present in the CROP case where the academic researchers did the laboratory analysis and data processing whereas the crop development companies did the applied fieldwork. In this case, the mutual dependence between academics and non-academics was very evident. By combining the field and genetic data (produced by crop development companies and academics, respectively), the project aimed to identify genetic markers for specific traits that could later be used by the crop development companies to advance their product development. In both cases, academic and non-academic interviewees alike acknowledged that the contributions of their counterparts were necessary for the projects to succeed, and although the roles and responsibilities were divided, the research activities were still highly interconnected and dependent on each other.

The active engagement and interdependence were also reflected in the interaction between the academic and non-academic participants. In both cases, the partners communicated and interacted frequently to exchange knowledge, findings, materials, access equipment, etc. As an example, the university researchers in the CROP case occasionally came to visit the crop development companies’ fields to collect samples, and the crop development companies likewise went to the university to get data (C-NA3). The crop development companies also found it very useful to interact with the international academic partners who had experience with the development of the specific crop as they could provide answers to ‘[…] any random questions about the crop that we couldn’t get from the more lab-oriented people […] it’s been good for the project to be able to discuss things with them, discuss the results […]’ (C-NA2).

Some empirical observations for the second part of the mechanism are:

• Both academic and non-academic partners were actively engaged in performing project
• Tasks and responsibilities were divided between partners to make use of their complementary
• The different partners’ activities were connected and mutually dependent on each
• Project partners had a close collaboration and communication about project results and

Given their respective roles and responsibilities, and their intensive engagement in research activities, we found that almost all project partners were actively engaged in what we have called the ‘generation’ phase. We propose that this active engagement and joint generation of research results is connected to the next part of the mechanism, the ‘evaluation’ phase, by a feedback loop (as depicted in Table 2). Both case narratives indicated that the joint generation of results led to ongoing monitoring of progress, reflections, and discussions between the project partners, and that this information fed back into the research activities and generation of results. It seemed that the ‘generation’ phase was linked to the ‘evaluation’ phase through the sharing of knowledge and results across the partners. The feedback from the ‘evaluation’ to the ‘generation’ phase will be explained in the next section. In the CROP case, the interviewees explained convincingly why the academic and non-academic partners were mutually dependent on each other as they contributed with expertise and capabilities that other partners simply did not have. The evidence of mutual dependence is weaker in the ENERGY case because several interviewees indicated that the large company could have done a larger proportion of the work internally, but decided not to, most likely to minimise their costs in the earlier stages of development of the new technology.

Part 3: Partners Discuss and Make Decisions about the Research Process and Usability of Results

During the research process, the project partners in both cases had formal and informal interactions that allowed them to monitor progress and reflect upon the research process and results. The informal interactions took place

in different forms, either in person when partners visited each other to use laboratory facilities or do sampling, etc., or through emails and telephone calls. The formal interaction mainly took place at project meetings, but also at less frequent steering group or board meetings. In both cases, interviewees explained that these different forms of interactions were essential to monitor progression and make important decisions together.

In the ENERGY case, the project leader team met (either physically or in a teleconference) once or twice a month to discuss how the project progressed, as explained by one of the non-academics: ‘[…] then we would just talk about results we make, so alignment; when, where, who did what, and were we sort of progressing [in the same direction]?’ (E-NA1). The partners in the CROP case held formal meetings less frequently, but with the same purpose of discussing progress and making key decisions together: ‘[…] we also develop the [types] for these emerging populations for the project, and there we sit together all partners and then decide which [types] we should make for the project to do research’ (C-NA3).

The linkage from the ‘evaluation’ phase and back to the ‘generation’ phase was evidenced most clearly in the ENERGY case. All interviewees explained that it was originally planned to develop a new technology, but this turned out to be more complicated than expected, especially considering that the new technology should be suitable for upscaling and industrial application. The project leader at the large company explained how frequent interactions and discussions of preliminary results, especially in the beginning, allowed the project partners to change directions and pursue a new method in addition to the originally planned:

‘[to] drive towards a solution which could be scaled to commercial relevance […] that was the scope of the project […] We ended [up] with something which looked significantly different […] that’s not a bad thing. That’s just progressive learning […] That comes down to the frequency of talking together […] We work in [number] different physical locations, but by talking weekly in the beginning, having many meetings, also utilising that we’re not that far apart […] it’s easy to meet and discuss. So, communication was just central in the beginning until we […] had shaped the path’ (E-NA2).

The ENERGY case thus illustrates how frequent interactions and discussions allowed the partners to correct quickly and make a joint decision on pursuing a new method in addition to the originally proposed one. The existence of a feedback loop is also supported by evidence from the CROP case as joint monitoring and discussions secured that the partners would stay aligned during the research process, feeding back into the research activities and securing that the results would be relevant and applicable for all partners.

The discovery of the new method in the ENERGY case, and the decision to pursue it, also illustrates how the project partners’ preexisting interests influenced this stage of the process, and how the partners managed to change directions while preventing potential conflicts from evolving. The interviewees gave slightly different accounts of how the discovery and choice of the new method came about, but they all stated that the partners agreed on pursuing the new method as they assessed it to have a larger commercial potential.

Over time, the feedback between the ‘generation’ and ‘evaluation’ phases (Part 2 and 3) resulted in accumulated scientific advances, eventually leading towards the next part of the mechanism, namely the utilisation of results (Part 4). To round off the discussion of the ‘evaluation’ phase, we can again identify the following observations:

• Partners interacted regularly both formally and informally to share knowledge and discuss interim project results which allowed for short learning loops and adjustments or change of directions
• Close collaboration and discussion, combined with common goals, facilitated decision making on further development of results
• Partners made key decisions together to make sure they agreed on the direction, and that results would be relevant and usable

These observations in combination support the presence of Part 3. It is possible that alternative explanations could account for (some of) the observables in isolation. For instance, one alternative explanation could be that even though meetings were held, and decisions were supposed to be made jointly, some partners forced their preferences through while others were either passive or ignored. However, finding all empirical observations at the same time seems to support the ’evaluation’ part and the claim that the joint monitoring and decision-making had a real impact on the research process. A weakness here is that this part of the analysis only relies on interview accounts, and the interviewees could have strong incentives to claim that they made important decisions together and overstate the level of agreement.

Part 4: Partners Utilise Scientific Advances to Develop New Technologies or Products

As the final part of the mechanism leading toward societally relevant outcomes, this section focuses on the non-academic partners’ utilisation of scientific advances to further develop new technologies or products. This part of the mechanism can also be considered an intermediate outcome of the research process (e.g. Penfield et al. 2014).

The comparison of the within-case processes of the ENERGY and the CROP cases shows that both research projects resulted in scientific advances that were usable for the non-academics, but these advances took different forms across the cases. In the ENERGY case, the company’s project leader explained that the most important result from the project was a proof-of-concept experiment that consolidated the potential of the new technology:

‘[…] it was a [number] of years, at least [number], maybe [number] years into the project where we got the proof-of-concept experiment which was central to being where we are today. Because without this proof-of-concept experiment, nothing would have happened. Just having the pitch at the start of the [ENERGY] project wouldn’t have given enough incentive for doing the full project’ (E-NA2).

This view on the potential was confirmed by other non-academics: ‘[…] already when we were in the [ENERGY project], they showed that money-wise, this was not way off, like it was actually close to being viable’ (E-NA1).

Similarly, the CROP project resulted in advances for both academic and non-academic partners. Focusing on the non-academics, the most important outcomes were new knowledge, capacity-building in terms of trained employees, access to a broad range of genetic material, and (after the project ended) genetic markers that would benefit the companies’ ongoing development work. Non-academic interviewees assessed that their participation in the CROP project had benefited the development process significantly in ways that would not have been possible without the research collaboration: ‘I don’t think there will be any [new product coming out of the CROP project], but I don’t think we would be able to do the [development] that we’re doing now without the project’ (C-NA1).

In the ENERGY case, a non-academic partner took the initiative to further develop the new technology within a demonstration-scale project funded by a new grant. The follow-on project was characterised by the project leader from the large company as a ‘full value chain type of project’ (E-NA2) including both ‘upstream’ and ‘parallel’ partners, potential purchasers, and universities.

The investment in the follow-on project provides quite strong evidence that the ENERGY project led to results that carried commercial potential. Having these results increased the non-academic’s willingness to lead the follow-on project and tolerate the higher risks and costs of commercialising the new technology.

The CROP project led to many follow-on projects involving the same and new partners. One follow-on project was described as a direct result and continuation of the CROP project. This follow-on project involved the same core group of academic and non-academic partners and was funded by a new grant targeting development and demonstration projects. The new project was more applied than the CROP project, using the knowledge and tools developed in the initial project to conduct further development. Additional follow-on projects were initiated spanning both more applied projects, some with international partners, and a basic science project that also built on the results from the CROP project. According to the PI (and the co-PI), the CROP project functioned as a platform for the co-PI of the project to build up a larger research group and utilise the developed methods and approaches. This development even spilled over to building a European platform and establishing cross-European project collaborations. The CROP case thus provided evidence of a continuation of the collaboration between the academic and non-academic partners and sustained interdependence between the basic and applied aspects of the research and development processes.

By comparing the case narratives, we can again identify the following observations:

• Non-academic partners utilised the project
• Project results demonstrated that the new technology had commercial
• Partners decided to pursue further development of the new technology or product by follow-on

Taken together, the observations provide support for the fourth part of the formulated mechanism. The fact the non-academic partners were willing to invest in follow-on projects, evidences the potential of the new technologies and future products. As the follow-on projects were co-funded by public funders, information on these projects was available and it supported the interview accounts.

Societal Outcomes: Commercialisation and Contribution to Societal Goals

This last section of the analysis examines the empirical indications of outcome generation resulting from the two studied research projects. As mentioned, it is still too early to assess if the expected societal outcomes will be fully realised. However, in both cases we have evidence of the ongoing investments and follow-on projects leading to further enhancement of the technological readiness level of new technologies and products, thus moving closer to the commercialisation stage.

In the ENERGY case, the demonstration scale follow-on project was still underway at the time of the interviews. But already at that point, the company now felt it had technology that was ready for the market, and that they wanted to commercialise. If implemented successfully on a larger scale, the new technology could contribute to a significant environmental impact (based on published article by project group). The project also led to scientific outcomes, including an influential journal article and new research projects that built on or used similar approaches.

In the CROP case, the crop development companies expected their first new crops to be ready for market (or in testing) at the end of the immediate follow-on project. Both academic and non-academic interviewees assessed that the new crops looked promising. They expected to generate outcomes in the coming years by marketing the new and improved crops to [end users], and thereby contribute to the stated goal of increasing the local production, which would likely also have an environmental impact. Some interviewees mentioned a potential for additional outcomes in the longer term if similar approaches and methods were used to advance the development of the same and other crop species for feed as well as food production.

The CROP project also had important scientific outcomes as it contributed to an ongoing movement from model-based to field-based research approaches.

Discussion and Conclusion

Drawing on empirical accounts from two cases and existing models of the societal impact of research, this paper uses process-tracing which seeks to link funding to societally relevant outcomes through research collaboration. Our within-case analysis of the two cases has allowed us to explore the research process in each case. In this final section, we summarise the mechanism that we developed and discuss key insights from the analysis.

The analysis traces how societally targeted funding provided an opportunity for collaborative, societally oriented research projects. The funding triggered pre-existing interests amongst academic and non-academic partners and allowed to combine relevant areas of expertise, while at the same time de-risking project participation by partially funding all project partners, which enabled transdisciplinary research collaboration. The following mechanism can be divided into four parts. Firstly, academics and non-academics aligned their interests and jointly designed the research project accordingly. Secondly, based on these goals and project design, partners actively participated and contributed, each with their relevant expertise. This created a mutual dependence among project partners for the research to succeed. Thirdly, partners frequently shared findings internally throughout the project. Based on the project developments, they jointly made decisions about the further research process, and how to utilise the results. These interactions and decisions fed back into the research activities, until the scientific advances enabled partners to utilise the results to develop new technology, e.g., in follow-on projects, which is the fourth part of the mechanism. Although project partners emphasised that research results showed significant commercial potential, broader societal outcomes were expected to take several years of further work.

Our main contribution to existing models of societal impact of research such as Joly et al. (2015), Kok and Schuit (2012), Molas-Gallart and Tang (2011) and Donavan and Hanney (2011) is in tracing in detail the process that links funding with research process and societally relevant outcomes.

Existing work on user involvement has focused on their contribution to outcomes (Kok and Schuit 2012), the importance of prior relations (Thune and Guldbrandsen 2014) and active involvement during the entire research process (Stier and Smit 2021). Our analysis supports these findings and further unfolds the role of users or non-academics in many ways. A notable aspect of the two cases is the high level of engagement of non-academics in designing and conducting the research and in its further development. The studied cases illustrate the role of non-academics as an active, driving factor in the process from funding to (potential) societal outcomes. Non-academics were involved in the ideation of the projects, which ensured that they targeted a real need and had a strong (though not exclusive) societal orientation. Due to their research competences and involvement in the project, non-academics in both cases were well-positioned to move forward with the further development of research results.

For the studied funding instrument, private companies were able to cover some of their costs for research and development. In both cases, non-academics stressed that they would not have conducted the research without external funding. Although we are unable to fully confirm whether they would have participated without, their detailed explanations lend support to this. In a policy perspective, this aspect is very relevant as it is not always the case that private companies can receive partial funding for their project involvement.

To some extent, societally relevant outcomes can be viewed at different stages from research results to innovations with potential for societal outcomes, to benefits for users and societal impacts of innovations. Both our cases appear close to developing an innovation, with potential benefits that would take longer to achieve. Although not yet materialised, academic and non-academic partners had positive expectations on the commercialisability of project results in both cases, and all partners appeared confident that they had succeeded in developing an innovation that could be implemented in time, once further developed in follow-up projects. In both cases, we were able to identify a number of key outputs which led to new projects and new collaborations. Tracing research processes in these two cases thus demonstrates the length of research processes and how their innovative development and outcomes are a product of a series of grants. This emphasises the importance of viewing research and innovation funding from a systems perspective to understand the funding conditions that exist to enable research collaborations at different stages of development (Aagaard et al. 2022).

Other work, such as Donovan and Hanney (2011) have focused on more developed outputs over a lengthier time period. Our contribution here is to document how funding and subsequent research and collaboration processes are linked to intermediate outcomes that are oriented to societal outcomes. In the same way, this linkage allows us to first unfold the productive interactions (D’Este et al. 2018) that take place in these two projects and then how they lead to intermediate outcomes.

As a caveat, some partners collaborate in other projects and have a strong commercial interest in the results of the studied projects, thus they might have an incentive to downplay potential conflicts or overstate the innovative potential of the project results in the interviews. With this in mind, their uniform accounts of events and the fact that they have continued the collaboration on further developing the project outputs make their descriptions and accounts more convincing and trustworthy. Access to additional empirical material such as meeting minutes or email correspondence would have further strengthened the analysis.

Based on this within-case study of two comparable cases, we are unable to conclude to what extent the process would work similarly in other cases. Future studies could therefore trace and test our proposed mechanism in similar cases which share the same contextual conditions such as research field and type of non-academic collaborators, as these conditions may causally influence whether and how the mechanism works.

Declarations

All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript.

Ethical considerations

The study design that this paper is part of was developed in 2019. Aarhus University’s ethical review committee does not require ethical approval for studies of this nature, where the collected data is personal and non-sensitive. Given this, we did not apply for ethical approval, however we have following ethical guidelines in collecting and analysing the data. Written consent was obtained from all participants, who were informed of the purpose of the study, how their data would used and stored, that only the study team would have access to their data, and their rights concerning their data and withdrawal from the study.

Acknowledgements

This work was supported by the Novo Nordisk Foundation under the project ‘Promoting the socio-economic impact of research—the role of funding practices’ (PROSECON) (grant number NNF18OC0034422).

References

Aagaard, K., Mongeon, P., Ramos-Vielba, I., & Thomas, D. A. (2021). Getting to the bottom of research funding: Acknowledging the complexity of funding dynamics. Plos one, 16(5), e0251488.

Aagaard, K., Norn, M. T., & Stage, A. K. (2022). How mission-driven policies challenge traditional research funding systems. F1000Research, 11(949), 949.

Ankrah, S. N., Burgess, T. F., Grimshaw, P., & Shaw, N. E. (2013). Asking both university and industry actors about  their  engagement  in  knowledge  transfer:  What  single-group  studies  of  motives omit. Technovation, 33(2-3), 50-65.

Beach, D., and Pedersen, R. B. (2018). Selecting Appropriate Cases When Tracing Causal Mechanisms. Sociological Methods & Research. 2018; 47(4), 837-871.

Beach, D., and Pedersen, R. B. (2019). Process-tracing methods: Foundations and guidelines. University of Michigan Press.

Beach, D. (2022). Process Tracing Methods in the Social Sciences. Oxford Research Encyclopedias.

Boon W. and Edler, J. (2018). Demand, Challenges, and Innovation. Making Sense of New Trends in Innovation Policy. Science and Public Policy. 2018; 45(4): 435–447.

Bornmann, L. (2013). What is societal impact of research and how can it be assessed? A literature survey. Journal of the American Society for information science and technology, 64(2), 217-233.

Bush, V. (1945). Science: The Endless Frontier. Washington, D.C.: National Science Foundation.

D’Este, P., Ramos-Vielba, I., Woolley, R., & Amara, N. (2018). How do researchers generate scientific and societal impacts? Toward an analytical and operational framework. Science and Public Policy, 45(6), 752-763.

Donovan, C., & Hanney, S. (2011). The ‘payback framework’ explained. Research Evaluation, 20(3), 181-183.

Gläser, J., & Laudel, G. (2019). The discovery of causal mechanisms: Extractive qualitative content analysis as a tool for process tracing. SSOAR-Social Science Open Access Repository.

Gläser, J. and Velarde, K. S. (2018). “Changing Funding Arrangements and the Production of Scientific Knowledge: Introduction to the Special Issue.” Minerva 56 (1):1-10.

Haddad, C. R., Nakić, V., Bergek, A., & Hellsmark, H. (2022). Transformative innovation policy: A systematic review. Environmental Innovation and Societal Transitions, 43, 14-40.

Joly, P. B., Gaunand, A., Colinet, L., Larédo, P., Lemarié, S., & Matt, M. (2015). ASIRPA: A comprehensive theory-based approach to assessing the societal impacts of a research organization. Research Evaluation, 24(4), 440-453.

Kattel, R., & Mazzucato, M. (2018). Mission-oriented innovation policy and dynamic capabilities in the public sector. Industrial and corporate change, 27(5), 787-801.

Kok, M. O., & Schuit, A. J. (2012). Contribution mapping: a method for mapping the contribution of research to enhance its impact. Health research policy and systems, 10(1), 1-16.

Lam, A. (2011). What motivates academic scientists to engage in research commercialization:‘Gold’,‘ribbon’or ‘puzzle’?. Research policy, 40(10), 1354-1368.

Lundvall, B. A. (1992). National systems of innovation: towards a theory of innovation and interactive learning.

Matt, M., Gaunand, A., Joly, P. B., & Colinet, L. (2017). Opening the black box of impact–Ideal-type impact pathways in a public agricultural research organization. Research Policy, 46(1), 207-218.

Mazzucato, M. (2018). Mission-oriented innovation policies: challenges and opportunities. Industrial and corporate change, 27(5), 803-815.

Molas-Gallart, J., & Tang, P. (2011). Tracing ‘productive interactions’ to identify social impacts: an example from the social sciences. Research evaluation, 20(3), 219-226.

Norn, M.T., Aagaard, K., Bjørnholm, J., & Stage, A.K. (2024). Funder strategies for promoting research addressing societal challenges: Thematic, impact, and collaboration targeting. Science and Public Policy, 51(5), 910–922.

Penfield, T., Baker, M. J., Scoble, R., & Wykes, M. C. (2014). Assessment, evaluations, and definitions of research impact: A review. Research evaluation, 23(1), 21-32.

Perkmann, M., Salandra, R., Tartari, V., McKelvey, M., & Hughes, A. (2021). Academic engagement: A review of the literature 2011-2019. Research policy, 50(1), 104114.

Polk, M. (2015). Transdisciplinary co-production: Designing and testing a transdisciplinary research framework for societal problem solving. Futures, 65, 110-122.

Ramos-Vielba, I., Sánchez-Barrioluengo, M., & Woolley, R. (2016). Scientific research groups’ cooperation with firms and government agencies: motivations and barriers. The Journal of Technology Transfer, 41, 558-585.

Ramos-Vielba, I., Thomas, D. A., & Aagaard, K. (2022). Societal targeting in researcher funding: An exploratory approach. Research Evaluation, 31(2), 202-213.

Schmitt, J. (2020). The causal mechanism claim in evaluation: Does the prophecy fulfill? In J. Schmitt (Ed.),

Causal Mechanisms in Program Evaluation. New Directions for Evaluation, 167, 11–26.

Sjöö, K., & Hellström, T. (2021). The two sides of the coin: joint project leader interaction in university‐industry collaboration projects. R&D Management, 51(5), 484-493.

Spaapen, J., & Van Drooge, L. (2011). Introducing ‘productive interactions’ in social impact assessment. Research evaluation, 20(3), 211-218.

Stier, J., & Smit, S. E. (2021). Co-creation as an innovative setting to improve the uptake of scientific knowledge: overcoming obstacles, understanding considerations and applying enablers to improve scientific impact in society. Journal of innovation and entrepreneurship, 10(1), 1-14.

Tartari, V., & Breschi, S. (2012). Set them free: scientists’ evaluations of the benefits and costs of university–industry research collaboration. Industrial and Corporate Change, 21(5), 1117-1147.

Thune, T., & Gulbrandsen, M. (2014). Dynamics of collaboration in university–industry partnerships: Do initial conditions explain development patterns?. The Journal of Technology Transfer, 39, 977-993.