Published at MetaROR
April 21, 2026
Table of contents
More than ‘just’ Citizen Science: the need to define and categorise Open Engagement of societal actors for Open Science monitoring purposes
Maaike Duine1, Anastasiia Iarkaeva2 and Nadin Gaasch3
1 Open Research Office Berlin, Universitätsbibliothek der Freien Universität Berlin, Germany
2 Berlin Institute of Health at Charité (BIH), QUEST Center for Responsible Research, Germany
3 TD-Lab – Labor für transdisziplinäre Forschung, Technische Universität Berlin, Germany
Originally published on December 15, 2025 at:
This article was presented at the STI-ENID 2025 conference.
1. Introduction
1.1. Participatory Research and Open Science
Open engagement of societal actors in scientific processes is one of the key pillars in the UNESCO’s recommendation on Open Science (2021): “Open engagement of societal actors refers to extended collaboration between scientists and societal actors beyond the scientific community.” (UNESCO, 2021, p. 13). The UNESCO, in its toolkit “Engaging Societal Actors in Open Science” (UNESCO, 2023, p. 2), presents examples of various forms of opening science to society through the open engagement of societal actors, highlighting the diverse range of open engagement practices such as citizen science, crowdsourcing, participatory action research and living labs. However, this enumeration appears rather arbitrary and lacks clear differentiation.
The challenge of defining the diverse forms of Open Engagement of societal actors (Haklay et al., 2021; Hacking et al., 2024) complicates comprehensive monitoring. Rafols et al. (2024) also state that Open Science (OS) processes such as collaborative practices (including public engagement) are not yet part of OS monitoring frameworks. Monitoring Open Engagement of societal actors is further complicated by the diversity in uptake and implementation across scientific disciplines (Pettibone et al., 2017). Therefore, to enhance transparency and enable comprehensive monitoring, we need a better understanding and description of open engagement practices.
In this paper, we focus on participatory research (PR), which represents according to the UNESCO definition a segment of Open Engagement with society in addition to crowd funding/crowd sourcing and scientific volunteering (see Figure 1). In our view, Open Engagement and PR are distinguished by the role of societal actors in research projects, as well as by the level and intensity of their participation. PR emphasizes a ‘real’ involvement of non-scientific actors in research—characterized by co-design of research, co-creation of new knowledge, up to co-decision on outputs. Open engagement practices predominantly encompass forms of unilateral form of interaction, such as information dissemination or consultation.
Figure 1. Open Engagement of societal actors within the UNESCO Recommendation on Open Science – © 2021 by UNESCO
We propose a categorisation of PR approaches, which we have attempted to consolidate and expand through an online search strategy and an exploratory online survey of PR projects. This categorisation can contribute to the development of indicators for OS monitoring purposes. Additionally, we use the data we have collected to show preliminary findings on discipline-specific patterns of PR approaches and answer the following questions: Which PR approaches are most prevalent in the Berlin research area? Which PR approaches are more prevalent in which scientific discipline?
1.2. Participatory research in the Berlin University Alliance and PR Map project partners
To address these questions, we collected data from research projects conducted in collaboration with societal actors, undertaken by researchers affiliated with the Berlin University Alliance (BUA). In 2018, four institutions in Berlin, Germany—Freie Universität Berlin, Humboldt-Universität zu Berlin, Technische Universität Berlin, and Charité – Universitätsmedizin Berlin—signed a cooperation agreement. Since 2019, the BUA has been funded as a research network by the Federal Ministry of Education and Research (BMBF) and the State of Berlin, as part of the Excellence Strategy of the federal and state governments.
Preceded by the BUA OS Dashboards project (2021-2023), the BUA OS Magnifiers project started in 2024 (Bobrov et al., 2024), with project partners BIH@Charité, QUEST Center for Responsible Research and Open Research Office Berlin. One of the project’s aims is to collaborate with scientific communities to develop OS indicators and metrics, and visualise the collected metrics in dashboards to support OS monitoring.
The BUA Laboratory for Transdisciplinary Research (TD-Lab) is the focal point for transdisciplinary research in the BUA and was initiated in 2019. The TD-Lab aims to strengthen transdisciplinary and participatory research practices in the Alliance and in Berlin – for research with society. The TD-Lab refers to the definition of transdisciplinary research (TDR) used primarily in the European research area, but also endorsed by the Global Alliance for Inter- and Transdisciplinarity (ITD-Alliance1). Accordingly, TDR addresses the complexity of multifaceted real-world problems, integrating processes of co-design and co-production, and fostering collaboration between researchers from various disciplines and societal or practice-based actors. Through mutual learning, this approach enables the joint development of new knowledge and robust solutions for societal challenges (cf. Pohl, 2010; Jahn et al., 2012).
In 2023, the BUA Open Science Magnifiers project and TD-Lab joined forces to increase the visibility of BUA projects in which research is carried out with society. This collaboration enabled an interdisciplinary perspective on OS Monitoring generally, and on Open Engagement of societal actors in research specifically. With the specific expertise of both project partners, data on relevant PR projects could be collected and an online visualisation (PR Map) could be created.
2. Methods
2.1. Data collection
To create the PR Map and include all relevant BUA projects, a search strategy to identify projects was co-created in collaboration with the National Institute for Science Communication (Nationales Institut für Wissenschaftskommunikation (NaWik))2. Appendix A shows the keywords used for the search strategy, in both German and English. In total 255 projects were identified. An online survey was created at the same time in LimeSurvey. In a survey pretest we contacted 25 project coordinators and received 9 complete project entries. Based on feedback received, we adapted the introduction text, and added more options to enter specific research outputs (e.g. Master and PhD-Thesis). The final survey questions can be found in appendix B.
All identified project leaders were contacted by email and asked to complete the survey. Out of 255 projects, two contacted project leaders replied they would not classify their projects as a participatory research project. In addition to personally contacting researchers we already know, an announcement in the BUA newsletter and on TD-Lab’s website were also part of the communication strategy with the aim to include as many projects as possible.
At the time of writing this paper, data cleaning resulted in 91 complete project entries. Duplicate and incomplete survey entries were removed and additional measures were taken to improve data quality, e.g.: German project entries were translated into English to enable consistent search functionality, and specific entries, such as ORCID iDs, were standardized. The survey remains open to facilitate the continued collection of additional relevant projects.
The PR Map will be a user-friendly, interactively designed collection of projects with filter functions, dynamic search options to analyse collected project data, i.a. PR approaches, participating institutions, involved societal stakeholders and Sustainable Development Goals (SDGs). The four scientific discipline categories defined by the German Research Foundation (Deutsche Forschungsgemeinschaft, DFG3)—humanities and social sciences, life sciences, natural sciences and engineering sciences—are represented by a different colour, with the discipline-specific projects grouped around them (Figure 2). 46.2% of the projects are conducted interdisciplinary, i.e. multiple disciplines were involved in the PR project, which is visually represented by colored circles around the projects.
Figure 2. Screenshot of the Participatory Research Map (beta version).
2.2. Defining Participatory Research for the development of the BUA PR Map
“Participatory research is an umbrella term for research approaches that explore and influence social reality in a collaborative manner. The objective is both to understand and to transform social realities.” (Von Unger, 2014, p. 1). Von Unger emphasizes that participatory research is not a method in the sense of a standardized procedure but rather a research style. This approach is guided by a distinct self-conception of sciences, which includes the recognition that non-academic knowledge, such as everyday knowledge, experiential knowledge, or domain-specific expertise, is relevant for addressing scientific questions.
One of the challenges in defining PR and making it operational for OS monitoring lies in the immense diversity of participatory approaches and terminologies. This diversity arises from the epistemological and methodological principles of research, which differ across disciplines and national research traditions. Figure 3 presents a selection of different PR frameworks, orientations, and approaches (Vaughn & Jacquez, 2020). From this perspective, it becomes evident that citizen science, which is often used as an umbrella term for Open Engagement with societal actors, is just one of many PR approaches.
Figure 3. Frameworks and approaches of Participatory Research (based on Vaughn & Jacquez, 2020).
To develop the PR Map and to increase the visibility of the variety of PR approaches within the BUA, we were faced with the challenge of not knowing which of these approaches are used by BUA researchers from diverse discipline backgrounds. We therefore decided to categorise the approaches, based on the TD-Lab’s many years of experience in working with participatory researchers. The classification was co-developed by the project team in repeated iterations to further refine the categories, approaches and roles. This explorative approach enabled us to sketch an initial picture of PR approaches in the BUA research area.
The following PR approaches were defined:
- Citizen science: citizens are explicitly involved in the research process, e.g., in data collection, data analysis and/or discussion of results.
- Living lab: non-scientific and scientific actors collaborate in real-world settings, set up as platforms for local innovation.
- Patient engagement: patients as non-scientific actors are involved in the research
- Collaborative action research: members of the researched community are involved as non-scientific actors and are empowered to directly apply the research results.
- Transdisciplinary research: close collaboration with non-scientific actors during all project phases; from co-designing the research process, co-defining research questions, to joint development of solutions for societal issues.
Survey participants were asked to enter up to three approaches, with an option to enter an ‘other’ PR approach in a free text field.
Additionally, we wanted to find out to what extent these research projects are actually participatory. To describe PR projects in terms of their type of societal involvement, we used the concept of participation levels by Arnstein (1969), which defines different levels of power of societal actors in democratic decision-making. This concept has been taken up and further developed several times in recent years (cf. Schrögel & Kolleck, 2019). The decisive factor in these concepts is the normative claim to participation, whereby a distinction is made between levels of ‘non-participation’ or alibi participation (‘tokenism’, including levels 1-5) and levels of participation by societal actors, determined by a certain level of decision-making power (levels 6-8). ’Non-participation’ based on Arnstein (1969) are all activities that are more or less linear knowledge exchange, such as informing, consultation, placation or data mining. ‘Participation’ refers to activities that foster partnerships for co-producing new knowledge and joint learning.
A research project can therefore be considered participatory if it enables genuine participation of societal actors in parts or throughout the entire research process. Based on the levels of participation, we have decided on the following categorisation regarding the roles of non-scientific actors:
- Consultation: non-scientific actors have a consultative role in the research process; the decision-making power is exclusively in the hands of the scientists.
- Involvement: non-scientific actors are engaged in knowledge exchange with scientists in the sense of joint learning; the decision-making power is exclusively in the hands of the scientists.
- Collaboration: non-scientific actors co-create the research process with scientists; the decision-making power in the research process is shared between scientists and non-scientific actors.
Survey participants were asked to enter one role that dominated in the project. A free text entry could be used to describe an additional role.
3. Preliminary results
3.2. PR approaches and roles of non-scientific actors
Survey respondents were allowed to select up to three of five pre-defined PR approach categories. The results (see Appendix C, Table 1) show that all categories have been selected at least once which justifies their inclusion in the PR Map, and that 93.4% of the 91 project entries could be classified according to a predefined category. We acknowledge that researchers might have categorised their project differently, when they were not presented with this categorisation.
Most project leaders (40.7%) selected one research approach, with transdisciplinary research (17.6%, N=16) being most prevalent. When more than one approach was selected, a combination with transdisciplinary research was also most prevalent (see Appendix C, Table 2). When two research approaches were selected (33.0% of 91 projects), the most prevalent combination was transdisciplinary research & collaborative action research (N=7) and living lab & transdisciplinary research (N=6). When three PR approaches were selected (19.8% of 91 projects), the most prevalent combination was living lab & citizen science & transdisciplinary research (N=4).
Overall, transdisciplinary research was the most selected approach in both single- and multi-approach projects (N=50). Of these, participants entered the role of non-scientific actors as ‘collaboration’ 33 times (see Figure 4). The heatmap aggregates the number of projects for both single- and multi-approaches into a single value for each role-approach combination. A darker color on the heatmap represents a higher frequency of such combinations. In Section 2.2., we explained that measures solely aimed at informing and consulting societal actors do not qualify as ‘real’ participation. Conversely, this implies that a transdisciplinary approach should always involve collaboration. This discrepancy between the selected participatory approach and the actual roles of the stakeholders suggests that some respondents (namely all participants who work primarily in a consultative setting) may not have fully understood the meaning of different participatory approaches or roles of participants represented in our survey.
Figure 4. Frequency count of combinations between PR approaches and roles of non-scientific actors in BUA PR projects.
Patient engagement was the least commonly selected approach in our current data collection. Patient engagement is the only one of the selectable participatory approaches that can be assigned to a specific field of research, namely medicine, whereas the other approaches are applied across disciplines. This, together with the fact that medical research accounts for a comparatively small proportion of total research in the Berlin research area, explains the low value. However, this does not diminish the importance of patient engagement.
Additionally, we asked participants to enter other PR approaches in a free text field in case none of the mentioned approaches was applied or in case they wanted to extend the definition. This resulted in 12 entries: some of them elaborated rather on the stakeholder group involved than being an additional approach, e.g., student/school (pupil) involvement, industrial partner, actors, artists; or the method used in the project, e.g., co-speculation, deliberative democracy, design thinking. Other terms included user engagement, and educational design research. One project added the comment: ‘Transdisciplinarity “light” in the sense that non-academic actors are involved but not in a structured or comprehensive way.’ This elaboration points out the complication of classifying PR.
We also examined the most commonly attributed Role of non-scientific actors (see Appendix C, Table 3). The most prevalent here was ‘collaboration’. The free text fields for Role of non-scientific actors resulted in two entries: Deliberative Democracy (same project that added Deliberative Democracy in the free text field for PR approaches) and ‘The different actors contribute different skills and knowledge that go far beyond a purely discursive project and touch us in a deeper way than a purely informative discourse can.’ These entries were interpreted as additional information rather than a potential new category.
3.3. Discipline-specific PR approaches
More than half of the entered projects (53.9%) were attributed to one of the four main discipline categories (see Appendix C, Table 4). Of these projects, most are from Humanities and Social Sciences (33.0%) (see Appendix C, Table 5). 46.2% of the entered projects were attributed to more than one discipline category. The most frequent collaborations between disciplines were found between Humanities and Social Sciences & Life Sciences (14.3% of the total) and Humanities and Social Sciences & Engineering Sciences (12.1% of the total). Collaboration of all four disciplines was observed only in 1 project (1.1% of the total). We acknowledge that dividing research projects in four main discipline categories may generalize and oversimplify statements about the level of interdisciplinarity of these projects. Therefore, we also gathered information on the more precise level of the DFG-categorisation. Analysing these results on the research area level will be one of our next steps and is therefore not included in this paper.
To determine whether or not there is a correlation between discipline category and PR approach, we used Cramer’s V4. This statistical method measures the strength of association between two categorical variables on the range from 0 to 1. For the relationship between disciplines and approaches, Cramer’s V shows a weak association (V=0.134), which indicates no relevant patterns. This test is based on a small sample size of 61 variable combinations of disciplines and approaches, many of which are infrequent (14 of 61 combinations appear only one time), potentially reducing the test sensitivity. Fisher’s exact test5 supports the argument of no significant relationship between the variables with a p-value of 0.4546. Figure 5 shows the distribution of disciplines among approaches, showing a quite homogeneous use of PR approaches in all disciplines—which we did not anticipate, as the various participatory approaches often emerged from specific disciplines (transdisciplinary research from sustainability and transformation research, living labs from transformative sustainability and economic research or engineering, and citizen science from the natural sciences).
Figure 5. Discipline distribution among Participatory Research approaches.
Additionally, Figure 5 shows that the majority of projects included in the PR Map stem from the Humanities and Social Sciences. Participatory Research methods have a longer tradition in this discipline, addressing questions of societal change and transformation, specifically in sociology and ethnography. The relatively strong presence of Engineering Sciences projects within the PR Map can be attributed to the focus on technology transfer and assessment within this field, as well as the use of participatory processes in urban planning and development. It is not surprising that Life Sciences, particularly medicine, are strongly associated with patient engagement. However, other approaches are applied within this field as well: this is probably due to projects within psychology, applied biology, and agricultural sciences. Projects from the (often fundamental) Natural Sciences are least represented in the PR Map.
A detailed investigation of the dependencies between the research areas and PR approaches and the effect of interdisciplinarity on this correlation is the subject of further research.
4. Discussion
We acknowledge the limitations of the study, such as a small sample size, project information based on individual assessments (other project participants might assess the project differently), and a possible ambiguity in the interpretation of the multiple-choice options in our survey (e.g., when >1 response was selected, how were individual response weights allocated?). Nevertheless, the dataset created for the PR Map makes it possible to draw an initial picture of Open Engagement of societal stakeholders in the Berlin University Alliance and provide input for monitoring PR approaches.
4.1. OS monitoring requires a categorisation of PR approaches
OS Monitoring—and monitoring Open Engagement of societal actors specifically—requires an interdisciplinary discussion of terminology and an understanding of different research processes, practices, and outputs. In this paper we described how we contributed to this discussion by defining and categorising participatory research approaches, which is often insufficiently examined in the OS Monitoring debate.
Our proposed categorisation of the five PR approaches proved functional—with most participants being able to identify at least one category for their project or a combination of approaches. We cannot determine whether the assignment was made consciously or intuitively, as this would require consultation with the respondents.
In our data collection, only 7.7% of the projects were classified solely as ‘citizen science’ (26.4% of total for both single and multiple selection). This shows that ‘citizen science’ is only one among many PR approaches. We propose not to use citizen science as an umbrella term for monitoring Open Engagement of societal actors and advocate for a more nuanced approach. This is essential in creating a comprehensive and flexible OS Monitoring framework that includes a wide range of PR approaches and processes. The terminology could for example be further refined and implemented as part of case studies for OS monitoring (Bezuidenhout et al., 2024).
4.2. Discipline-specific monitoring of Open Engagement of societal actors requires further granulation
Our preliminary results indicate a slight preference for single-discipline attribution, with more than half of the projects falling into this category. Interdisciplinarity across all four major disciplinary fields (based on the DFG-classification) appears to be an exception, suggesting that participatory research is more commonly practiced within closely related disciplinary contexts. However, as noted before in this paper, further analyses of interdisciplinarity within participatory research is needed.
This also applies for analysing the different PR approaches used amongst disciplines. The results suggest that PR approaches can be intuitive, and be interpreted differently, not only by individual researchers, but also across disciplines. Preliminary analysis showed that the five participatory research approaches are homogeneously distributed among disciplines and cannot be contributed to specific fields (with the exception of Patient Engagement in Life Sciences). Further in-depth analyses of the data, such as finer discipline entries analysis or project stakeholder analysis and stakeholders’ roles in the research process, might help to identify possible patterns of discipline-specific prevalence for specific PR approaches and whether or not disciplinary affiliation shapes how PR is practiced.
4.3. Open engagement should be actively strengthened as a core element of open science
In a trend study on the changing scientific cultures in the Berlin research area (Ambrasat et.al., 2024), ‘citizen science’ was presumably used as an umbrella term for various PR approaches. The survey results show that citizen science, compared to other OS practices, is practiced less frequently by BUA researchers than other OS practices, such as data sharing and open peer review. When asked about the need for support in practicing OS, BUA researchers perceive applying citizen science practices as one of the biggest challenges. It is unclear whether this is related to lack of knowledge and failing resources, or if the term ‘citizen science’ was perceived differently by researchers.
Our preliminary results show a wide range of participatory research methods, not only citizen science. With our categorization of the different PR projects in the PR Map, we aim not only to increase the visibility of these different projects and approaches but also to raise awareness amongst BUA researchers to these approaches. The TD-Lab will continue to support BUA researchers engaging with non-scientific stakeholders and strengthen transdisciplinary and participatory research practices in the Alliance and Berlin.
5. Conclusion and next steps
In this paper we described how a categorisation of PR approaches could contribute to developing a more comprehensive OS Framework. By refining the categories, approaches and roles, and our preliminary analysis of the dataset, we gained insight into the importance of discussing these concepts on an interdisciplinary level. Not only to acknowledge the diversity of research practices, but also to contribute to the development of an inclusive OS monitoring framework that will include a wide range of participatory research approaches, going beyond ‘citizen science’. These insights will also support our work in the Open Science Monitoring initiative6 (OSMI). OSMI aims to develop indicators for standardised monitoring processes and underscores the importance of recognizing the diversity of research practices and processes in different disciplines and OS practices that go beyond quantifying OS.
The current data collection suggests that transdisciplinary research is the most prevalent PR approach in the BUA (both single- and multiple-choice). Furthermore, the data shows a noticeable prevalence only of Patient Engagement in Life Sciences, whereas other approaches are quite evenly distributed among all four (generic) discipline groups. We plan to further analyse the dataset, and further explore the discipline-specific approaches by examining the projects on the second level of the DFG categorisation. We will also assess which disciplines are collaborating with which societal stakeholders (Business & Industry; Politics; Media; Arts and Culture; Civil Society Organizations, Citizens, or Other).
In a recent scoping review on the societal impact of OS practices, Cole et al. (2024) found evidence that PR produces a wide variety of beneficial societal impact. We want to explore this further and look at the Sustainable Development Goals PR projects from the BUA are contributing to. Additionally, we will take a look at the outputs of the projects: What types of outputs are created and are these made openly available?
We will also continue to develop the visualisation and analyse if the PR Map is an effective tool for networking purposes and how we can promote good practices for Open Engagement of societal actors, in the Berlin research area, and beyond.
Open science practices
LimeSurvey software is open source. The data collection underlying the PR Map will be made openly available, as soon as the PR Map goes live.
Author contributions
Maaike Duine: Conceptualization, Methodology, Writing – original draft, Writing – review & editing; Anastasiia Iarkaeva: Conceptualization, Methodology, Formal Analysis, Visualization, Data curation, Writing – original draft, Writing – review & editing; Nadin Gaasch: Conceptualization, Methodology, Writing – original draft, Writing – review & editing
Acknowledgments
The authors would like to thank Dr. Nikola Nölle for her extensive comments to earlier versions of this paper.
Competing interests
The authors have no competing interests to declare.
Funding information
The BUA OS Magnifiers project and the BUA TD-Lab are funded by the Federal Ministry of Education and Research (BMBF) and the state of Berlin under the Excellence Strategy of the Federal Government and the Länder through the Berlin University Alliance.
References
Ambrasat, J., Lüdtke, D. & Yankova, Y. (2024). Forschungskulturen und Forschungsqualität im Berliner Forschungsraum. https://doi.org/10.18452/29605
Arnstein, S. (1969). A ladder of citizen participation. Journal of the American Planning Association, 35(4), 216–224. https://doi.org/10.1080/01944366908977225
Bezuidenhout, L., Castaño, P., Leonelli, S., Rafols, I. & Vargiu, A. (2024). Introducing Case Studies in Monitoring Open Science. Zenodo. https://doi.org/10.5281/zenodo.14174180
Bobrov, E., Duine, M., Kindling, M. & Iarkaeva, A. (2024). Das Projekt BUA Open Science Magnifiers: Weiterentwicklung des Open-Science-Monitorings in verschiedenen Disziplinen und Forschungsbereichen. Open Access Blog Berlin. https://doi.org/10.59350/yp5f9-wjn07
Cole, N.L., Kormann, E., Klebel, T., Apartis, S. & Ross-Hellauer, T. (2024). The societal impact of Open Science: a scoping review. Royal Society Open Science. 11:240286. http://doi.org/10.1098/rsos.240286
Hacking, N., Lewis, J. & Evans, R. (2024). Mapping Approaches to ‘Citizen Science’ and ‘Community Science’ and Everything In-between: The Evolution of New Epistemic Territory?. Minerva, 62, 549–572. https://doi.org/10.1007/s11024-024-09529-z
Haklay, M., Dörler, D., Heigl, F., Manzoni, M., Hecker, S., & Vohland, K. (2021). What is citizen science? The challenges of definition. In: Vohland, K., et al. The Science of Citizen Science. Springer, Cham, 13-33. https://doi.org/10.1007/978-3-030-58278-4_2
Jahn, T., Bergmann, M. & Keil, F. (2012). Transdisciplinarity: Between mainstreaming and marginalization. Ecological Economics, 79, 1-10. https://doi.org/10.1016/j.ecolecon.2012.04.017
Open Science Monitoring Initiative, Principles of Open Science Monitoring. (2024). https://doi.org/10.52949/49
Pettibone, L., Vohland, K. & Ziegler, D. (2017). Understanding the (inter)disciplinary and institutional diversity of citizen science: A survey of current practice in Germany and Austria. PLoS ONE, 12(6), e0178778. https://doi.org/10.1371/journal.pone.0178778
Pohl, C. (2010). From Transdisciplinarity to Transdisciplinary Research. Transdisciplinary: Journal of Engineering & Science, 1, 65-73. https://doi.org/10.22545/2010/0006
Rafols, I., Meijer, I. & Molas-Gallart J. (2024). Monitoring Open Science as transformative change: Towards a systemic framework [version 1; peer review: 2 approved]. F1000Research 13:320. https://doi.org/10.12688/f1000research.148290.1
Schrögel, P. & Kolleck, A. (2019). The Many Faces of Participation in Science: Literature Review and Proposal for a Three-Dimensional Framework. Science & Technology Studies 32(2), 77–99. https://doi.org/10.23987/sts.59519
UNESCO. (2021). UNESCO Recommendation on Open Science. https://doi.org/10.54677/MNMH8546
UNESCO. (2023). Engaging Societal Actors in Open Science. https://doi.org/10.54677/NIWD9521
Unger, H. von (2014). Partizipative Forschung. Einführung in die Forschungspraxis. Springer VS Wiesbaden. https://doi.org/10.1007/978-3-658-01290-8
Vaughn, L. M. & Jacquez, F. (2020). Participatory Research Methods – Choice Points in the Research Process. Journal of Participatory Research Methods, 1(1). https://doi.org/10.35844/001c.13244
Notes
- https://itd-alliance.org/
- https://www.nawik.de/
- https://www.dfg.de/en/research-funding/proposal-funding-process/interdisciplinarity/subject-area-structure
- https://www.ibm.com/docs/en/cognos-analytics/11.1.x?topic=terms-cramrs-v
- https://en.wikipedia.org/wiki/Fisher%27s_exact_test
- https://open-science-monitoring.org/
Appendix A: Search terms used for identifying PR projects.
| German | English |
|---|---|
| partizipativ*e forschen/ Forschung | participatory research |
| transdisziplinär*e forschen/ Forschung/ Transdisziplinarität | transdisciplinary research / approach/ transdisciplinarity |
| (kollaborativ*e …. (….. Arbeit, Methoden, Prozesse, Ansätze) | collaborative (action) research |
| Patient_innenbeteiligung | patient participation/involvement/engagement |
| Bürgerforschung (-beteiligung) | Citizen Science, Citizen participation, public engagement, societal engagement |
| Reallabor | (Urban) Living Lab (Real-Word-Laboratories) |
| Kokreation/ Ko-Kreation, Themenfindung(sprozess), Koproduktion/Ko-Produktion, gemeinsames Lernen Wissensintegration/ Integration von Wissen |
Co-Creation, Co-Exploration, Co-Production, Co-Design joint/ mutual learning knowledge integration/ integration of knowledge |
| Open Innovation | |
| Bürger*innen UND Projekt(e) Bürgerwissenschaften, Bürgerwissenschaftsprojekte | citizen AND project(s) |
| Zusammenarbeit/Beteiligung nicht-Wissenschaftler*innen | Collaboration/Involvement non-academic actors |
| Aktionsforschung | action research |
Appendix B: Survey questions
| Entry (*= obligated) | Type of entry (free text field or text field) | Description |
|---|---|---|
| Project title* | free text | Please enter the full project title. |
| Project affiliation | free text | This is a sub-project with affiliation to the superordinate project project: freetext |
| Project acronym (abbreviation) | free text | Please enter the project acronym, short title or key words (maximum number of characters = 50). |
| Project website* | free text | Please enter the links to the project website(s), or select ‘no website’. |
| Short project description or abstract* | free text (max. 500 words) | Please enter a short project description or project abstract (max. 500 words). |
| Participatory research approach* | Citizen science | Please select if citizens are explicitly involved in the research process, e.g. in data collection, data analysis and/or discussion of results. |
| Living lab | Please select if non-scientific and scientific actors collaborate in real-world settings, set up as platforms for local innovation. | |
| Patient engagement | Please select if patients as non-scientific actors are involved in the research process. | |
| Collaborative action research | Please select if members of the researched community are involved as non-scientific actors and are empowered to directly apply the research results. | |
| Transdisciplinary research | Please select if you closely collaborate with non-scientific actors during all project phases; from co-designing the research process, co-defining research questions, to joint development of solutions for societal issues. | |
| Other: (free text) | Please indicate any other participatory research approaches that have been applied in the project. | |
| Role of non-scientific actors in the research process | Consultation | Please select if non-scientific actors have a consultative role in the research process; the decision-making power is exclusively in the hands of the scientists. |
| Involvement | Please select if non-scientific actors are engaged in knowledge exchange with scientists in the sense of joint learning; the decision-making power is exclusively in the hands of the scientists. | |
| Collaboration | Please select if non-scientific actors co-create the research process with scientists; the decision-making power in the research process is shared between scientists and non-scientific actors. | |
| Other: (free text) | ||
| BUA Institution involved in project* | Charité – Universitätsmedizin Berlin (https://ror.org/001w7jn25) | Please enter involved BUA Institution. |
| Freie Universität Berlin (https://ror.org/046ak2485) |
||
| Humboldt-Universität zu Berlin (https://ror.org/02qt0xs84) | ||
| Technische Universität Berlin (https://ror.org/03v4gjf40) | ||
| Which other stakeholders are involved in the project? | (Other) scientific institutions; Business & Industry; Politics; Media; Arts and Culture; Civil Society organizations, Citizens, Other: free text | Please select the stakeholder type and enter the name in the following text field. |
| Academic Institutions | free text (Name) | Please enter institute’s name. |
| free text (Country) | Please select country from list. (https://en.wikipedia.org/wiki/ISO_3166-1) | |
| free text (ROR iD) | Please retrieve ROR iD from registry. (https://ror.org/) | |
| Non-scientific institutions | free text (Name) | Please enter stakeholder´s name. |
| free text (Country) | Please select country from list. (https://en.wikipedia.org/wiki/ISO_3166-1) | |
| free text (ROR iD) | Please retrieve ROR iD from registry. (https://ror.org/) | |
| Estimation of involved scientific actors | Scale (0-10; 11-50; 51-100; 101-1000; >1000) | Please estimate the number of all involved scientific actors (number of persons). |
| Estimation of involved non-scientific actors | Scale (0-10; 11-50; 51-100; 101-1000; >1000) | Please estimate the number of all involved non-scientific actors (number of persons) from all stakeholder groups (citizens, students, representatives of business & industry, politics, media, arts & culture, civil society organizations). |
| Researcher(s) involved in the project | first name last name; first name last name;
first name last name; …. |
Please enter the names of all researchers involved in the project. |
| Researcher affiliation (Institution, Country) | open; open; open; open
…. |
Please enter the affiliation for each researcher. Please add ‘formerly’ if the researcher is no longer affiliated with the BUA-Institution. Please select country from list. (https://en.wikipedia.org/wiki/ISO_3166-1) |
| Researcher ORCID iD | 16-digit ORCID iD | Please enter the researcher’s ORCID iD (https://orcid.org/) |
| Project start date* | month/year* | Please enter the project’s start date. |
| Project end date* | month/year; open | Please enter the project’s end date. When the end date has not been defined yet, please choose ‘open’. |
| Project funder(s) | BUA; DFG; BMBF; ERC; EU; no external funding; other: free text; | Please select all that apply. |
| Sustainable Development Goal* | List of 17 SDGs; unknown | Please select up to three SDGs. You may also use the following tool to assess to which SDG(s) the project is or could be contributing: https://osdg.ai/ When unknown, please select ‘unknown’. |
| Project output | Journal Articles, Books; Data; Code; Lab Notebooks; Blogposts; Podcasts; Apps; Platforms; Magazines; Teaching materials; Games; Bachelor-/Master-PhD-Thesis; General publication list (link); Other: free text | Please select all produced outputs (multiple entries possible). Add Persistent Identifiers (e.g. DOIs, handle or another PID) when applicable. Please use the ‘other’ category to add additional project outputs. When available, please include a link to the project’s publication list. |
| Subject Area DFG* | DFG list (level Fachkollegium); other | Please select all subject areas from the DFG-list (Fachkollegium) that are involved in the project: https://www.dfg.de/resource/blob/175334/89ba4a3464c99aaea40fdef47367e7b2/fachsystematik-2020-2024-de-grafik-data.pdf When unknown, please select ‘unknown’. |
Appendix C Preliminary results
| Approaches’ combinations (Nr) | Number of projects | % of projects of total (=91) |
|---|---|---|
| 0 | 6 | 6.6 |
| 1 | 37 | 40.7 |
| 2 | 30 | 33.0 |
| 3 | 18 | 19.8 |
| Approaches’ combinations | Number of projects | % of project s from total (=91) | Count overall |
|---|---|---|---|
| Transdisciplinary research | 16 | 17.6 | 47 |
| Collaborative action research | 5 | 5.5 | 31 |
| Citizen Science | 7 | 7.7 | 24 |
| Living lab | 5 | 5.5 | 24 |
| Patient engagement | 4 | 4.4 | 16 |
| Living lab & Citizen science | 1 | 1.1 | 1 |
| Living lab & Collaborative action research | 3 | 3.3 | 3 |
| Living lab & Transdisciplinary research | 6 | 6.6 | 6 |
| Citizen science & Collaborative action research | 2 | 2.2 | 2 |
| Citizen science & Transdisciplinary research | 2 | 2.2 | 2 |
| Patient engagement & Collaborative action research | 3 | 3.3 | 3 |
| Patient engagement & Transdisciplinary research | 3 | 3.3 | 3 |
| Collaborative action research & Transdisciplinary research | 7 | 7.7 | 7 |
| Living lab & Citizen science & Collaborative action research | 3 | 3.3 | 3 |
| Living lab & Citizen science & Transdisciplinary research | 4 | 4.4 | 4 |
| Living lab & Patient engagement & Collaborative action research | 1 | 1.1 | 1 |
| Living lab & Collaborative action research & Transdisciplinary research | 1 | 1.1 | 1 |
| Citizen science & Patient engagement & Transdisciplinary research | 2 | 2.2 | 2 |
| Citizen science & Collaborative action research & Transdisciplinary research | 3 | 3.3 | 3 |
| Patient engagement & Collaborative action research & Transdisciplinary research | 3 | 3.3 | 3 |
| No approach entry | 6 | 6.6 | 6 |
| SUM | 91 | 100 | 192 |
| Roles | Number of projects | % of projects total (=91) |
|---|---|---|
| Collaboration | 45 | 49.5 |
| Involvement | 29 | 31.9 |
| Consultation | 14 | 15.4 |
| No role entry | 3 | 3.3 |
| SUM | 91 | 100 |
| Disciplines’ combinations (Nr) | Number of projects | % of projects total (=91) |
|---|---|---|
| 1 | 49 | 53.9 |
| 2 | 32 | 35.2 |
| 3 | 9 | 9.9 |
| 4 | 1 | 1.1 |
| Disciplines’ combination (groups) | Number of projects | % of projects total (=91) |
|---|---|---|
| Humanities and Social Sciences | 30 | 33.0 |
| Life Sciences | 6 | 6.6 |
| Natural Sciences | 1 | 1.1 |
| Engineering Sciences | 12 | 13.2 |
| Humanities and Social Sciences & Life Sciences | 13 | 14.3 |
| Humanities and Social Sciences & Natural Sciences | 2 | 2.2 |
| Humanities and Social Sciences & Engineering Sciences | 11 | 12.1 |
| Life Sciences & Engineering Sciences | 4 | 4.4 |
| Natural Sciences & Engineering Sciences | 2 | 2.2 |
| Humanities and Social Sciences & Life Sciences & Natural Sciences | 2 | 2.2 |
| Humanities and Social Sciences & Life Sciences & Engineering Sciences | 5 | 5.5 |
| Life Sciences & Natural Sciences & Engineering Sciences | 2 | 2.2 |
| Humanities and Social Sciences & Life Sciences & Natural Sciences & Engineering Sciences | 1 | 1.1 |
| SUM | 91 | 100 |
Editors
Ludo Waltman
Ludo Waltman
Editorial assessment
by Ludo Waltman
In this article the authors study ‘open engagement’ of societal actors in scientific processes, arguing that open engagement is broader than citizen science and that there is a need for a systematic approach to classify different forms of open engagement. The article has been reviewed by two reviewers. Both reviewers recognize that the article makes a valuable contribution by showing that citizen science is just one of the many approaches to participatory research. However, the reviewers also point out important limitations of the article. Reviewer 2 cautions that the article uses “very coarse analytical categories” and therefore offers only limited insights into participatory research. Likewise, reviewer 1 expresses concerns about the robustness of the conceptual framing and the methodology. The reviewer therefore questions the contribution of the article to the goal of informing open science monitoring.
Recommendations for enhanced transparency
- The data underlying the research has not yet been made openly available.
Peer review 1
I reviewed an earlier version of this paper as a conference submission and it is good to see that many of the points I raised have been acknowledged in this updated version – particularly in relation to the limitations of the study (see p.10).
For me, the most valid and useful finding is “that ‘citizen science’ is only one among many PR approaches” (p.11). In the contexts I am familiar with (e.g. UK, Canada and across the Global South) ‘citizen science’ has not been commonly used as an umbrella term for PR approaches and the broader array of approaches is acknowledged. However, if this is not the case in the context of BUA (as the cited study suggests; Ambrasat et.al., 2024), then the authors’ proposal “not to use citizen science as an umbrella term for monitoring Open Engagement of societal actors and advocate for a more nuanced approach” has value.
I also think there is potential value in the authors’ plan to “assess which disciplines are collaborating with which societal stakeholders (Business & Industry; Politics; Media; Arts and Culture; Civil Society Organizations, Citizens, or Other).” (P. 12)
However, I am still not convinced that this paper provides a meaningful contribution to the broader goal of informing OS monitoring, given the conceptual and methodological confusion in the proposed framings, which are reflected back in respondents confusion in their answers to the survey alongside the lack of clarity in how ‘assignments’ to the different approaches were made.
For example:
P.6 – I am not sure that ‘transdisciplinary research’ constitutes a distinct approach, as it could apply to all of the other listed approaches and the fact that more than half of projects identified as being transdisciplinary indicates this as well. Other approaches that could be missing include deliberative approaches, organisational partnerships (e.g. between a research organisation and civil society/third sector organisation), community advisory boards, peer research, practitioner-inquiry and future-oriented design. I think a broader literature review could capture this diversity and strengthen the proposed framework.
P.7 in terms of roles and levels of participation, a fourth role might focus on ‘community-led’ initiatives (i.e. cases when non-scientific groups or organisations set research agendas and play a leadership role with scientists providing methodological expertise)
It is hard to imagine one ‘role’ applying consistently to a whole project. Perhaps it would be better to break down projects into stages or elements (e.g. agenda-setting, design, data collection/analysis, representation/communication, adaptation/use, evaluation/learning) and explore how roles (and potentially approaches) are distributed across these stages.
P.7 I concur with the acknowledgement that “researchers might have categorised their project differently, when they were not presented with this categorisation.”
P.7-8 What does the following acknowledgement mean for the data collected/analysis? “some respondents (namely all participants who work primarily in a consultative setting) may not have fully understood the meaning of different participatory approaches or roles of participants represented in our survey.” It would seem that this fundamentally destabilises the resulting analysis…
P.9 Regarding the free text additions to the ‘Role of non-scientific actors’, deliberative processes may constitute a new category if they place decision-making in the hands of the participants (with the scientists providing the evidence that forms the basis of the deliberation but not ultimately making the decisions). Similarly, the second entry: The different actors contribute different skills and knowledge that go far beyond a purely discursive project and touch us in a deeper way than a purely informative discourse can’ also comprises a distinct category in that it suggests a range of affects that cannot be reduced to the question of ‘who makes decisions’ (e.g. emotional connections, new representations etc.) I would suggest incorporating these into the framework – indeed that would make your research more participatory!
P.9 “- we did not anticipate, as the various participatory approaches often emerged from specific disciplines (transdisciplinary research from sustainability and transformation research, living labs from transformative sustainability and economic research or engineering, and citizen science from the natural sciences).” Is this perhaps revealing of the fact that PR approaches (like all methods) are not static but emergent with influences across disciplines over time, and that ‘transdisciplinarity’ like its predecessor ‘interdisciplinarity’ is increasingly being incorporated into many funding calls and research programmes? The authors may wish to reflect on the challenge here of pinning down fluid terminology for the purpose of monitoring.
P.10 I’d be interested to hear whether the study included any insight into the ‘purer’ humanities (e.g. philosophy, literature etc), which may also be more fundamental and involve less engagement?
P.11: Given the ambiguity in the framings of the study and the limited response rate, I’m not sure the following claim is warranted: “Our preliminary results indicate a slight preference for single-discipline attribution, with more than half of the projects falling into this category. Interdisciplinarity across all four major disciplinary fields (based on the DFG-classification) appears to be an exception, suggesting that participatory research is more commonly practiced within closely related disciplinary contexts.” Certainly in the UK context, Patient (Public) Involvement often incorporates the life sciences, social science (public policy) and increasingly the humanities (e.g. medical humanities). From what has been shared of the survey instrument, I am not sure that enough data was generated to make this claim.
While I am not convinced that the study’s conceptual framing and methodology are robust enough to fulfil the primary aim of the project (i.e. a contribution to OS monitoring), I think that if this paper was framed more reflexively, it is precisely the tensions and ambiguities in the framings that have the potential to raise interesting questions about the challenges of categorising PR approaches that are themselves continuously evolving (in terms of how they are defined, understood and applied).
Peer review 2
In my view, this paper makes an important contribution, which is to show that citizen science is just one of the many participatory research methods – and that participation as described in Open Science (UNESCO’s framework) needs to broaden to include various approaches.
This is clearly stated in this paragraph: “We propose not to use citizen science as an umbrella term for monitoring Open Engagement of societal actors and advocate for a more nuanced approach. This is essential in creating a comprehensive and flexible OS Monitoring framework that includes a wide range of PR approaches and processes.”
The data collected for this purpose is sufficient and convincing. Therefore, the study is successful and robust with respect to its central contribution.
It is also interesting to see that there is no main Participatory Research method across various disciplines – except prevalance of ‘patient experience’ in life sciences (as one would expect).
My main critique to the paper is that it uses very coarse analytical categories: five participatory research categories, only four disciplines, just three degrees of participation. My impression is that this coarse classification is useful to show some very simple patterns, but analytically is not a good tool to understand participatory research. This might be because the paper does not have a strong conceptual background.
In short, this is a valuable contribution for a very specific issue (a critique to citizen science), but in my view, one should be cautious about the robustness of the survey and its analytical framework for other purposes.
Specific comments:
1) What is ‘Open Engagement’?
In the sentence below, and in various places, you use the notion of ‘open engagement’. I am not aware of a special type of engagement – that is ‘open’. How is different from ‘engagement’? What would you say that ‘close engagement’ is?
“PR emphasizes a ‘real’ involvement of non-scientific actors in research—characterized by co-design of research, co-creation of new knowledge, up to co-decision on outputs. Open engagement practices predominantly encompass forms of unilateral form of interaction, such as information dissemination or consultation.”
2) What is justification of your classification of participatory research?
Your classification in terms of ‘citizen science’, ‘living labs’, ‘patient engagement’, ‘collaborative action research’ and ‘transdisciplinary research’ is not justified beyond saying that it is based ‘…based on the TD-Lab’s many years of experience in working with participatory researchers’. I am sorry, but this is asking the reader to trust you, which is not what a very rigorous research method.
My impression is that the categories that you used are research traditions related to specific social locations, not related to specific practices or methods. For example, ‘patient engagement’ can take a variety of forms, from participation priority setting, to commenting on how the patient will endure clinical experiments.
3) The degree of participation in terms of “consultation – involvement – collaboration” is portrays a very linear understanding of participation. In my view, issues such as the stage of the research (agenda setting, research, application of results), as shown by Michel Callon and colleagues can be equally relevant.
