Introduction

The modern landscape of medical research has undergone a profound transformation, marked by a shift towards industrial modes of knowledge production. As medical research became more institutionalized, collaborative, and resource-intensive, many of the hallmarks of industrialization such as efficiency, proceduralization and standardization, were adopted (Gaudillière and Löwy, 1998; Erlach, 2022). Throughout the 20^th century, various forms of standardizations accompanied the emergence of new research objects, methods, or research trends. The recent development and dissemination of reporting guidelines has led to the standardization and homogenization of research products, i.e. scientific papers, thus completing the picture of the industrialization of medical research. The improvements and changes promised by the widespread application of reporting guidelines are profound. Reporting guidelines are intended to impact the writing practices of almost all medical researchers worldwide and reshape the textual characteristics of medical studies on a macro scale.

Reporting guidelines suggest a fundamental transformation of formal scholarly communication in the medical sciences. This transformation bears substantial differences compared to other forms of standardizations in research. Firstly, reporting guidelines are more tangible than other standards. Core reporting guidelines such as CONSORT for randomized controlled trials or PRISMA for systematic reviews consist of short checklists that authors should follow. Codified and published as journal papers that can be downloaded, printed, or cited, the guidelines are comprehensible, acceptable and convenient to use (Schniedermann et al., 2021). In contrast, traditional forms of standardizations, such as standard humans (Epstein, 2007), theory-method packages or research bandwagons (Fujimura, 1988, 1992), involve several documents or even books that, taken together, compose a complex and interrelated set of regulations and specifications not reducible to a single document.

Secondly, the reporting guidelines claim to be applicable to all medical subfields. They address the forms and structures of scientific publications, irrespective of content. Unsurprisingly, some reporting guidelines have become widespread. Randomized-controlled trials cannot be published without compliance with CONSORT in core medical journals (Shamseer et al., 2016). PRISMA for systematic reviews have accumulated over eighty thousand citations since 2010. They are currently applied by over a third of all published systematic reviews, with an increasing trend (Schniedermann, 2023). Their focus on writing and their success suggests that reporting guidelines have authority and regulatory power across many domains. But such an authority is unusual to medical research which is composed of many subfields and practice communities that come with their own local authorities and forms of standardization (Bowker and Star, 1999; Galison, 2010; see also Whitley 2000/1984).

Thirdly, reporting guidelines have become politicized because, besides being given moral authority to justify their regulatory function, they have been found to be ignored, misunderstood or misused. The guidelines were originally developed in response to the high prevalence of selective outcome reporting in medical research since the 1990s (Schniedermann et al., 2021). As such, they were promoted with reference to the moral obligation of researchers, becoming instruments ‘mandating an occasion for truth-telling or lying’ (Freese and Peterson, 2018, p. 300). However, such high moral expectations are problematic given that standards in practice are subject to complex local contingencies and unforeseeable events — an ‘obvious and even banal’ observation in STS (Hedgecoe et al., 2023, p. 373). Scholars of science and technology studies have promoted more balanced and reflective understandings of standardization that acknowledge such complexities and uncertainties (Hedgecoe, 2023; Timmermans & Epstein, 2010; Van Drimmelen et al., 2024; Van Loon, 2015). Nevertheless, advocates of the guidelines criticize how they are ‘frequently used inappropriately’ (Caulhey et al., 2020, p. 87), despite them having a ‘clear and well-articulated intention’ (Sarkis-Onofre, 2021, p. 117).

Taken together, reporting guidelines seem to provide an unprecedented form of standardization in science. Analyzing their uses reveals how writing tasks are homogenized in practice and how local contexts influence deviations or modifications of the standard. Yet most research focuses on the production of reporting guidelines while their application and integration into everyday practices remain ambiguous (see also Fuller et al., 2015). Likewise, metascientific studies evaluate guideline compliance by counting what rules were followed or ignored most, despite calls for broader approaches (see Page and Moher, 2017).

Understanding the factors of success and resistance of reporting guideline implementation also provides a balanced perspective on standardization that is neutral to any positive or negative effects of standards and that was recently called for (Zuiderent-Jerak, 2021; Timmermans and Epstein, 2010).

Lastly, studying the standardization and homogenization of scientific articles reveals how the characteristics and social lives of textual types and literary genres in medical research emerge and unfold. Studying writing practices and products closes a gap in this regard, also because journal articles are not just outputs but also inputs to scientific procedures, i.e. research authors being readers too. Thus, studying reporting guidelines reconnects the sociology of scientific practice with literary studies which have experienced a revival of genre studies, albeit with a refocus on the context rather than the structure of genres (Bhatia 2008; Miller and Fahnestock, 2013; Swales, 2004).

To address these ambiguities and research gaps, this study shows how authors of systematic reviews have incorporated the ‘Preferred Reporting Items for Systematic Reviews and Meta-Analyses’ (PRISMA) guideline into their workflow. The following questions will be addressed in particular: Firstly, which systematic reviewing practices are affected, changed or left unchanged by the implementation of PRISMA? Secondly, which individual and contextual factors are related to the different types of guideline usage? For example, what role do professional backgrounds, attitudes towards standards, or collaborative and organizational settings play? Thirdly, how does the use of reporting guidelines affect the products and outcomes of research? In other words, which characteristics of systematic reviews are transformed by the PRISMA guidelines, and how does the standardization of scientific products affect formal scholarly communication in general?

The remainder of this article is structured as follows. The next section will briefly outline systematic reviews as both practices and products of research, as well as the characteristics and peculiarities of reporting guidelines. After that, some conceptual remarks on standardization are provided. Subsequent sections introduce the empirical approach and present and discuss results in relation to the research questions.

Systematic reviews and the PRISMA reporting guidelines

What is systematic reviewing?

During the postwar era, review articles and meta-analyses have become more and more important to cope with the abundant and often conflicting results of the nascent publish or perish culture in academia, as well as the ‘the clinical trial industry’ (Meldrum 2000, p. 755) which was fueled by regulatory requirements in the United States and private interests of the pharmaceutical industry (Sismondo, 2009). By selectively choosing favorable studies while ignoring those with conflicting results, review articles can be used strategically to redraw the picture of available evidence, so that it becomes publishable or even serves financial interests. In contrast, the systematization and standardization of synthesizing research promises to enable direct replications that uncover subjective or biased study selections. As a result, systematic reviews are seen as independent from private interests and more objective research syntheses overall (Chalmers et al., 2002; Hunt, 1999).

The core tasks in systematic reviewing are the search for relevant primary studies, their abstraction according to a predefined template, and the construction of new qualities which are re-aligned into a new story (Moreira, 2007). These processes involve agreed-upon techniques such as database search queries, formal inclusion and exclusion criteria, or so-called risk of bias assessments. Many of the relevant tasks have evolved into highly sophisticated methodologies, based on bodies of methodological or conceptual literature (Hunt, 1999). Properly applied, these techniques result in tables, graphs, or other codified items in the systematic review. This development not only standardized and professionalized the reviewing practice but also turned systematic reviews into its own genre of scientific publishing. As such, the genre was equipped with epistemic authority by putting it on top of the evidence hierarchy in medicine where it is the primary source of evidence for evidence-based medicine (Solomon, 2015; Moreira, 2005).

Reforming the reporting of research^[1]

Against the background of a growing dissatisfaction with published systematic reviews, reporting guidelines such as PRISMA were developed as interventions on improving medical writing. PRISMA consists of a checklist comprised of thirty-six different rules that cover each aspect of a systematic review article, from title to conclusions. It also provides templates for technical material and graphs, such as the PRISMA flowchart. This flowchart depicts the inclusion and exclusion of studies as a technical procedure, supports fast comprehension of methods, and increases the comparability. While PRISMA has been originally published in 1999, its 2009 version gained substantial attention and was evaluated and updated in 2020. The PRISMA guidelines have been published simultaneously in multiple journals, accompanied by explanatory documents and translations, as well as extensions and forks.[1]

The relation or distinction between conducting and reporting a systematic review is important to understand the current role of reporting guidelines such as PRISMA. Reporting guidelines are designed as interventions on practices of writing, rather than conduct, such as the aggregation and synthesis practices introduced above. They suggest leaving decisions of study design and methodology up to the authors and the disciplinary conventions (Freese and Peterson, 2018). For example, rather than setting up rules of how a database search should be performed, the guidelines focus on how to properly describe that procedure. In doing so, they promise to minimize the conflict with already established methodologies or unique types of systematic reviewing which may be different across medical subdisciplines. While the distinction between reporting and conduct can be considered crucial in that sense, it has been argued that the way how reporting rules are specified involves statements about practices of conduct, thus inherently makes valuations that go beyond mere reporting (Schniedermann 2022).

Two perspectives on the standardization of research

Standardized research practices

The incorporation of standards into research practices means to equip practitioners with accepted-upon, learnable and repeatable scripts to perform required tasks. This involves forms of proceduralization and routinization that promises replicability, rigor, and efficiency. Modern biomedical research settings are complex: distant and cross-disciplinary collaboration requires not only intellectual translations but also proper work organization and division of labor. In addition, unforeseeable problems can arise and require immediate improvisation. Thus, research projects come with substantial uncertainties and complexities that are addressed by standardizations and research protocols. In such protocols, experts preplan the overall process by predefining the ‘hypothetical sequence of events’ (Suchman, 1985, p. 124). Further, protocols make project plans transparent to all collaborators, supports the division of labor, and finally reduce costly and error-prone trial and error behavior (Fujimura, 1987; 1988; Whitley, 1984). In that sense, Richard Whitley conceptualized standardization as centralized form of reducing ‘task uncertainty’ (Whitley, 1984, p. 119).

Scholars in STS elucidate the role of standards in science from different perspectives. By shaping practices, standards also shape communities of practice, new trends or bandwagons, or whole new research fields (Fujimura, 1988, 1992; Galison, 2010). They conclude that standards in practice never live up to their ideal of homogenization and formalization of procedures (Moreira 2005; Timmermans and Berg, 2003; Solomon 2015). Local contingencies and complexities in the application of standards remain and must be muddled through, worked around, or tinkered with (Bowker and Star, 1999; Fujimura, 1987; Clarke and Fujimura, 1992; Sigl, 2016). Although no unform theory on standardization might be drawn, a more symmetrical and mundane approach towards standardization is required. Such an approach refrains from scandalizing moral claims as well as resistance and hostile attitudes. It takes diverse forms of using standards for granted and further attempts to understand how practices, standards and work contexts interrelate (Castel, 2009; Timmermans and Epstein 2010; Zuiderent-Jerak, 2021).

Standardized research products

Besides the guidance of scientific practice, modern medical research is permeated by standardized objects and products of research (Gaudillière and Löwy, 1998). Outputs of research, such as systematic review publications, are no exception in this regard, because they share certain textual forms, patterns in writing, or technical visualizations which, taken together, constitutes what we consider as the literary genres of science (Swales, 2004).

With regards to their modes of production, important analogies to the standardization of practices can be drawn. Textual standards became manifest with the more tacit proliferation of the experimental article (Shapin and Schaffer, 2011/1985). But scientific texts also became subject to more distinct formularizations such as the ‘Introduction-Methods-Results-Discussion’ (IMRaD) article format (Day, 1989), up to reporting guidelines such as PRISMA. Such standards promise to reduce the complexity of writing tasks, since there are many possible ways to describe research objects, methods and procedures, or draw and discuss conclusions. Subsequently, publications become authorized exemplars of the genre – standardized products, assets on scientific reputation markets (Gaudillière and Löwy, 1998; Erlach, 2022; see also Birch and Muniesa, 2020). However, similar to standardized practices, genres are dynamic, fluid and uncertain too.

Managing information and textual characteristics is a complex and uncertain balancing act and if we believe those who speak of a reporting crisis, many authors have been unable to hold this balance. Reporting requires authors to mediate between their own evaluations, disciplinary conventions, and what audiences might consider relevant (Bazerman, 1983). Although standardized genres suggest a template for such mediation and promise quality assured uniformity to their audiences, metascientists have criticized systematic review reporting as insufficient, as explained above. For this reason, not only the technical structures or semiotic forms themselves are important to understand genres, but also their contexts of production and social implications (Bhatia, 2008; Miller and Fahnestock, 2013). In this regard, we must consider that scientific publications are not only outputs or products of research but also inputs, and authors are readers too.

Together, reporting guidelines are situated at the intersection of practices and products of medical research. They guide writing practices and presumably shape the systematic review as a literary genre.

Data and methods

This study is based on the publications of 21 systematic review projects and semi-structured interviews with their authors. The selection of participants was based on different characteristics derived from bibliometric data, e.g., the review’s author count, author position, keywords, the author’s publication track, and country of affiliation.

The level of experience of the authors spanned all career stages from PhD students, postdocs to full professors. Participants came from the United Kingdom, European Union, Canada, Africa, and USA. The systematic review projects were of different sizes with most reviews having up to five authors (n=9), five to ten (n=8), or more than eleven authors (n=3). Medical specialties, as indicated by the keywords, involved epidemiology, psychology, oncology, surgery, or nursing among others. The participants described themselves as statisticians, economists, ethicists, or healthcare professionals and guideline developers. While all were researchers, roughly a third were also practicing doctors or psychiatrists, or had concrete aspirations to become one. The interviews covered individual background and project characteristics, as well as experiences with the PRISMA guideline and general perceptions about its role.

Informed consent was collected via email and digitally signed consent forms, and also orally during the interviews. The analysis employed a blended coding approach based on a set of deductive codes derived from the literature on standardization. The analytical focus was put on the correlation or co-occurrence of codes and involved MaxQDA and Microsoft Excel. All original names were removed and participants were pseudonymized. Further details on the sample of participants, the interview guideline, the codebook, can be found in the supplementary material. An earlier version of this manuscript can be found here https://osf.io/gs7e4/.

Results and Discussion

1. Contingencies and complexities in Writing Science

1.1. Extension of PRISMA’s scope

Authors and users of the PRISMA guideline offered a variety of ways how they interact with the guideline and incorporate it into their systematic reviewing workflows. In stark contrast to the suggested use ‘early in the writing process’ (Page et al., 2021a, p. 3 (BMJ version)), study participants started using PRISMA at an early stage of the entire project. As such, they explained how they used PRISMA for planning, designing, conceptualization of the whole systematic review, to write the protocol^[2], draft the method section and formulate a research plan, after writing the abstract and doing an initial screening, or in short, to ‘sort of frame everything’ (Louis). Some interviewees did not reconsider PRISMA during or after writing at all. Authors offered mixed opinions about these types of usage. They described it as ‘gold standard’ (Sabrina, Marco), or ‘non-standard’ but most beneficial use (Louis) that is either common, uncommon, or even a misuse.

Planning the review according to the PRISMA guideline helped to solve practical problems and manage current and future workloads. Understanding the written report as the endpoint of the process, interviewees explained how it is easier to finish the project if most of the steps and decisions can be anticipated already during the planning phase. If the guideline is only consulted as a checklist during or after the writing phase, users would have to go back in the process or even redo aspects of the analysis in case they missed something. This imposes additional efforts which should be avoided in any ideal case:

‘When it comes to reporting it’s just like if at the planning stage you forgot something it doesn’t matter how you report it. It is just wrong or probably would do a lot of damage to your work if you actually write, ‘oops I forgot the critical appraisal’. And then you have to go back and do it all over again. So, if you have the guidance at the beginning and you see oh yeah, the critical appraisal, I cannot forget about this. So that helps a lot more than just report it. I feel that reporting is like just to make happy the reviewer or just to make happy these people that set the guidelines’ (Alonso, postdoc in epidemiology).

Using PRISMA to guide planning and conduct of a systematic review means that it is used not only as a template that defines the structure and contents of the textual report but also as a tool that coordinates the reviewing procedure itself.

Assigning new functions turns PRISMA from a standard into local tool. Such conversions substantially expand the applicatory scope of the guideline but have not been anticipated or intended by the developers or are indicated anywhere in its codified rules. While such forms of extension might be considered as valuable practices, i.e. skillful tinkering (Fujimura, 1987), they go beyond what can be considered as ‘operationalizing underdetermined aspects in the research protocol’ (Van Drimmelen et al., 2024, p. 6). Further, they remain tacit and publicly unknown which makes their benefits and functions non-standard. Thus, even if such adaptations were beneficial, they can neither be quality-assessed, nor promoted for wider use.

1.2. Deviating from the script

Besides extending PRISMA’s scope to practices of conduct, authors can also decide to reduce its scope. There are different types of conducting systematic reviews, some which may differ substantially from what PRISMA’s developers had in mind. In such cases, authors modify some of PRISMA’s rules or write ‘not applicable’ in the checklist which is legitimate by the standard’s definition. While only a few interviewees did so, all found that their systematic review is somewhat different from the norm, even if they had no trouble in applying the guidelines. They understood the hypothetical procedure inscribed in PRISMA as ideal, rather than practical. Thus, deviation from the guideline should be considered the norm rather than the exception.

PRISMA was not understood as a limitation to creativity or communicational needs by the participants in this study. Besides one participant who generally rejected the idea of guidelines for writing, no interviewee provided fundamental issues with its application or criticized its overall epistemic role. Therefore, the guidelines do not seem to cause immediate friction with other methods and established practices even if embedded into a network of different standards. At this stage, critical aspects seem to be more tied to the realm of potential issues and theoretical considerations, rather than being observable phenomena. Consequently, the non-usage of PRISMA should not be interpreted as reluctance or stubbornness of individuals in complying with standards.

However, systematic reviewers choose different strategies in deviating from PRISMA and provide different justifications. These ranged from insecurity and self-criticism about what the correct application of the guideline might be, up to highly self-confident explanations of the reasons for deviation. In the latter case, these were contextualized with a general discussion about standards or transparency in research, showing a higher awareness for the social and epistemic contexts of knowledge production. In this regard, Sofia discussed the usage of PRISMA against fundamental aspects of research processes.

‘[T]here have to be standards, and you have to apply those standards. And you have to justify if you don’t do it, why you don’t do it. I think that’s the key to research. You have to be aware of your limitations. If you ARE aware of your limitations, then I think it’s all good, because then you can be transparent about it.’ (Sofia, PhD student).

By standardizing a particular set of tasks, the PRISMA guideline indirectly makes other, non-standardized tasks visible. Individuals who get acquainted with the level of uncertainty reduction provided by the guideline become aware of the uncertainty and complexity of adjacent tasks. This can create uncertainties and the desire for more standards and guidelines for these domains, which was frequently encountered during the interviews. Asked about their wish list, participants mentioned how they would welcome more guidance for other practices, for example for compiling supplementary materials or writing systematic reviews in non-traditional topic domains, such as AI-based research.

So far, this and the previous sections have focused on how actual usage of PRISMA differs from what might be considered as intended or ideal. Instead of plainly removing complexities and task uncertainties from the sequence of operations, standardizations shift those to adjacent tasks or create completely new managerial problems that must be solved. This has further implications.

Firstly, the separation between conduct and reporting, while being valuable for the dissemination and justification of the standard, is alien to the practical realities and problems of systematic reviewers. This goes even beyond traditional perspectives of how ‘emerging standards for the reporting of experiments create imperatives for experiments to be done in certain ways’ (Bazerman, 1988, p. 22). Rather, scientific writing is much more entangled with the overall process of scientific fact-making (see also Castañeda, 2019).

Secondly, experts should be more cautious with regards to how checklists fuel the impression that research practices can be reduced to a formulaic procedure. By defining ideals and norms, standards can change the fictional story about what research should be and eventually shape our expectations about it (see also Millerand et al., 2013). Notably, already the first experimentalists found how the persuasive power of replicability might be more important than actual replication (Shapin and Schaffer, 1985). In this regard, the perceived communicative functions of PRISMA played an important role for the systematic review authors which will be addressed in the next section.

2. Standardized systematic reviews and adjacent genres

2.1. Managing textual information

The wide application of the PRISMA guideline redefines systematic reviews as a genre. In general, the guidelines’ rules suggest authors where to put which information about the procedure. Specific information must be provided in the title, abstract, methods or result sections which not only reshuffles the distribution of information but also sets clear writing tasks for authors. PRISMA was developed against the background that necessary information is often missing. Thus, most of its rules require to add certain information about the project and its methodology.

The addition of information ranges from the usage of certain concepts or phrases in the text, over the reformulation of article titles, to the inclusion of technical visualizations that are highly visible and can be considered as formative for the genre. As such, PRISMA also provides a template to visualize the study selection process as a flowchart. The flowchart is commonly used and read so that it is a central aspect of PRISMA’s communicative function. Interviewees explained how the flowchart is the most distinct and visible sign of guideline compliance when they read or peer review other systematic reviews. The PRISMA checklist, in contrast, is only occasionally consulted. Notably, the most minimum interaction with PRISMA encountered in this study was when authors merely used the flowchart but did not consider the checklist and its reporting rules at all.

PRISMA leads to the removal of information from the main manuscript, yet in a more indirect way. Being a minimum standard, the guideline does not prohibit authors from providing more information than necessary. But by defining the required and relevant information for the review, PRISMA indirectly hints at what information might be considered irrelevant. Therefore, the guideline creates a boundary between important or necessary information, and unnecessary or additional information. This boundary is visible to authors and becomes practically relevant to them, as Adam noted:

‘(PRISMA) reduced my need to go hunting for other holes in my paper because it said, ‘These are the key things you need to say.’ As long as you’ve answered each of these things adequately (you are fine). That reduces effort as well.’ (Adam, postdoc from UK).

With the suggestion to add certain information and remove others, PRISMA contributes to the standardization of the systematic review genre. This might go as far as the guideline and the genre become ‘somehow synonymous’ (Thomas, postdoc). However, genres are situated within communication systems, and the reformulation of systematic review documents also impacts other forms of communication which will be addressed next.

2.2. Putting important information elsewhere

The regulation or removal of information does not necessarily lead to a net loss of information about the research projects. Sometimes, systematic reviewing projects can become very extensive or complex with regards to their study design or methods. In such cases, authors must provide additional details that go beyond what is required by reporting guidelines. Interviewees explained how information venues other than the main manuscript, such as protocols, preprints, supplementary material or even project websites became crucial for their reporting and communication activities. For instance, they mentioned how lengthy database search strings would break the flow of the manuscript and are thus shifted to the supplementary material.

In turn, the main manuscript contains references to these related sources but reports many aspects only in an abbreviated form to balance guideline compliance with readability and journal requirements. Notably, even the PRISMA guideline itself suggests to use of adjacent documents for this purpose (see Page et al., 2021a). Thus, by redefining the systematic review genre, i.e. specifying what belongs to the main manuscript, anything not covered by the guideline becomes external or additional information that must be reported elsewhere. As a consequence of this, systematic review articles become more similar to project maps that show where certain details can be found, rather than platforms that carry complete documentation.^[3]

PRISMA’s mentioning and strengthening of supplementary communication devices turns the distribution of information to multiple venues into an active task with additional uncertainties and responsibilities. Authors not only have to decide which information goes where, they might also face future contestation of these decisions and need to justify them. Consequently, users want guidelines that regulate these issues as well, turning them into the guideline’s ‘collateral reality’ (Law, 2011). Affected researchers such as Marco would welcome clear criteria what information has to go where or what the information thresholds for the different venues are:

I wonder if, uh, there could be a difference between what has to be reported in the, reported in the manuscript. And what could be just reported in the supplementary material. …. So, they say you present the full search strategy for all database registers and websites. So, for me, it’s like 15 pages of research equations. So, naturally, I thought, okay, I’ll put this as supplementary material, but I think some others would just think, ‘well, I cannot do this’. And so, they ended up reporting a very short research equation. (Marco, PhD student).

Making use of adjacent venues for reporting information creates new reporting tasks for authors. But these venues or tasks come with new informational uncertainties and complexities that must be solved by the authors, and for which no reporting guidelines might be available. Authors must invest additional time and attention to decide the proper quantities and qualities of information for adjacent documentations. In that sense, PRISMA reduces the informational uncertainty of systematic review manuscripts but fuels the creation of new documents, thus additional informational uncertainty. Therefore, PRISMA has a shifting effect on informational complexity, similar to how it affects practical task uncertainties as described above. This relation has also been illustrated by Law and Ruppert who wrote that ‘security cameras move theft around to places without such cameras’ (Law and Ruppert, 2013, p. 230).

2.3. Making compliance visible

The reformulation of the systematic review genre has important implications for its social functions and capabilities. This also involves what kind of information or reporting quality is expected by the wider community from certain textual genres. Formal reporting guidelines such as PRISMA provide a consensual agreement. This means that PRISMA’s rules do not only impact the manuscript itself, but also the informational expectations by readers and peer reviewers.

On the one hand, not complying with PRISMA can become an easy point of critique for reviewers, or a ‘low hanging fruit’ (Sofia). But even an allowed deviation from the guideline can draw critical attention. One interviewee suspected that if he writes ‘”doesn’t apply”, a reviewer may ask for that based on the checklist or the guidelines’ (Alonso). On the other hand, authors become able to judge if requests for more information are justified or exaggerated. But overall, interviewees remained ambiguous if PRISMA immunizes their systematic reviews against critique.

Irrespective of the textual changes induced by PRISMA, the mere visibility of guideline compliance has an impact on the critical assessment done by readers or peer reviewers. The guideline becomes visible when it is mentioned or cited in various places such as in the abstract, methods section, or supplementary material. While these occurrences are harder to spot, the characteristic PRISMA flowchart can be encountered while skim reading the manuscript. It can be used by readers as a shortcut to judge the reporting quality of the manuscript. This is based on the expectation that the mere interaction with a reporting guideline might be enough to trust the intentions of the author:

I think if you show this transparency, by showing ”here is the protocol“, ”here is the extracted data“, ”we have used these and these guidelines” and if it all comes from the beginning, then I think that inspires a certain amount of trust. And then most people think yes, it’s probably true that they followed these guidelines. (Bernd, translated).

In another interview, this form of trust became associated with the normal expectations of peer reviewers:

(One of the peer reviewers) said that he was very impressed with us that we decided to use the PRISMA guideline in Nigeria. … He said that if you know that a person used the PRISMA guideline, all reviewers will be quiet because it is now sure that I am a very thorough person. (Tangeni)

In such cases, PRISMA not only reshapes writing practices and resulting documents, but also the social life of systematic reviewing as a genre. This means that the expectations towards this genre by readers or peer reviewers are updated as well. If a systematic review bears textual elements that result from PRISMA compliance, consumers feel assured that the authors sufficiently interacted and complied with the guideline. The mere sign of interaction then makes further investigation superfluous. Thus, it is not the guideline itself that ensures transparency in practice, but rather the whole system comprised of the standard, its users, and their interaction. In that sense, several interviewees argued similar to (Lamin) that

As long as you are going by a system that is ensuring the quality of your work, then it should be considered for publication rather than a journal coming out to say you must use such and such a guideline. (Lamin).

On a first glimpse, shortcuts to critically assessing the reporting quality seem to be problematic. The mere reference to PRISMA does not imply or indicate the quality or depth of satisfying its individual rules, and some interviewees were more critical of the role of reporting guidelines in this regard. In fact, metascientists did not criticize the ignorance towards reporting guidelines but rather the quality of application, as explained above (see also Page and Moher, 2017). But since deviation from PRISMA or extending its applicatory scope was so common to the individual interaction with PRISMA, we might consider that the technical reference to the guideline is much more important in order to understand the standardization of scientific reporting.

While the practices remain heterogeneous, the visible impact of PRISMA on systematic review documents, i.e. its contribution to the typification of the genre, bears more homogeneity and similarity across projects. Instead of discussing whether standards fuel homogenization or not, the question is where instances of homogeneity and comparability are located in the overall research process. Technical references to reporting guidelines and visible literary technologies that emerge from using standards are then the culmination points of homogeneity with regards to standardization. In contrast, practices such as writing, reading, or peer reviewing remain more heterogeneous because they are situated around these culmination points and are more influenced by their respective contexts. Therefore, PRISMA’s role in the typification and co-creation of the systematic review genre has more profound implications for the social reality of standardization in biomedical research, than its impact on the practices of writers or readers.

3. PRISMA and its contexts of use

The previous sections have highlighted how PRISMA influences systematic reviewing by managing practical tasks and information flows. The types of interaction and deviation were not uniform, so that there is no standard use of the standard. But how do project contexts, collaborative configurations and researcher profiles define how PRISMA is used? To approach this question, first, a closer look is taken at the project size and its collaborative intensity. Second, the role of the researcher’s previous experience is illuminated.

3.1. Project size and collaboration

The intensity of interaction with the PRISMA guideline is shaped by the different contexts of systematic review projects. In short, in smaller, paper-centric project settings in which the published text is the central goal, PRISMA was more intensively used for planning and design. Interviewees reflected on PRISMA by recalling stories about crisis and epistemic reforms, expectation of reviewers and mistrust towards peers. In contrast, interviewees from bigger projects tended to be more critical and reflective on PRISMA’s actual scope. Emphasis was put on working culture and collaborative efforts. Responsibilities and epistemic requirements were defined with regards to colleagues who are the ‘first peer reviewers’ (Adam). The published review was more of an intermediate step, rather than final goal in such settings.

Standards are a central ordering tool for collaborative research projects and research consortia. Especially for the medical sciences, scholars have highlighted the role of research protocols for an overall project planning that is actionable and transparent for all project members. In such dense collaborative networks, standards and protocols reduce the managerial task uncertainty because they predefined the required tasks and show how these build on each other (Keating and Cambrosio, 2012). In addition, standards represent the required and involved professions, because they use technical jargon from those professions. In doing so, they set ex ante quality criteria, negotiate authority claims, and individual responsibilities. In this regard, Sofia who coordinated a systematic review for a research consortium explained how a specialist took over a central aspect of the review:

‘But we had an information scientist, which I only learned when I started working [on that project], that that’s a thing…. So, she runs a lot of the searches for [a research association] to make sure those are the correct searches. Because how I conduct searches, I’m just being a bit more open and using less search terms. And I basically have more articles and then drill it down. And she is a bit more specific, and I guess also knows it also better than I do. I think it’s really helpful, because it was such a big project. It was also good for my conscience to know every study which could be included, IS included, and if not, that’s not my fault, that’s HER fault (laughs).’ (Sofia, PhD student, epidemiologist).

At the same time, the interaction of different standards, professions, or even additional stakeholders such as patient organizations shape and stabilizes the managerial scope of involved standards such as the PRISMA guideline. The diverse interests require ‘opportunities for mentorship and growth for everybody’ (Sabrina, working with a team of 6-10). In such settings, PRISMA is more unlikely to be used beyond its intended scope because there are likely other standards and principles that shape those domains. Not surprisingly, interviewees in these settings have been more aware of the PRISMA’s actual scope and its limitations.

Although consortium based systematic reviews can be stricter in using PRISMA as reporting guideline, they also have the resources to amplify its definition as minimum standard. Repeatedly speaking of the review as ‘work package two’, Sofia explained how the huge team used the IT resources of an industry partner to turn the systematic review report into a separate website of its own. The review’s journal publication only became an abbreviated version of that website and did not report all the available codified information. The reporting needs of the project thus extended PRISMA’s scope by far.

Similarly, Marco mentioned how his team developed and published an extensive research protocol that served as background for multiple systematic reviews. The protocol was developed with the team, provided much more details on the processes and methods, and was used for templating the whole project beyond any single paper. In turn, the final manuscript featured only a shortened method section that referred to the protocol which was also handed out to the peer reviewers. Therefore, bigger project contexts can not only influence the level of task setting by PRISMA, but also fuel the genrefication of adjacent documents.

In contrast to research consortia, smaller research projects build their epistemic authority on their relation to disciplinary cultures and community-based standards instead of their collaborative mode of work. Maliciously speaking, getting published in a reputable journal and being cited are often the most distinct goals in those research projects. Here, journals and peer reviewers are the main factors to justify projects and legitimize knowledge claims. Compliance with PRISMA is seen as a necessary task to get published and interviewees repeated the prominent yet wrong perception that journals generally desk reject systematic reviews that do not report PRISMA compliance. On this way, the PRISMA guideline became an evaluative device similar to publication indicators and researchers start to ‘think with the guidelines’ similar to how they ‘think with indicators’ (Müller and de Rijcke, 2017, p. 160).

Seeing PRISMA as ex ante quality marker paves the way for using it during planning and conduct of the systematic review, as explained above. In addition, researchers in paper-centric projects must meet the complex tasks and requirements of a systematic review with more limited resources and available skillsets. Without access to an information specialist or librarian, they welcomed PRISMA as tool for planning and task scheduling.

Complying with PRISMA shows that the author can perform and report a valid systematic review article. The proper performance of the systematic review procedure and the technical proficiency as an individual asset becomes more important than the idea to solve a global scientific problem, similarly to how research bandwagons have been described (see also Fujimura, 1987). In this regard, interviewees framed their criticisms of the reporting quality in their field more similar to the metascientists, i.e. with emphasis on the official promises of standards and individual responsibilities or moral obligations (see also Van Drimmelen et al., 2024). Ironically, they offered much more dispassionate explanations for their own deviations. As such, Adam, a medical practitioner and clinical researcher, explained how he did not engage intensively with the epistemic role of the PRISMA guideline because getting published was not his major goal.

‘I expect that if you want to publish, if you’re really motivated by getting into top tier journals as well, that’s probably more important. If you are a professional academic, maybe it’s important. I’m a part-time academic, so it doesn’t matter too much.’ (Adam, postdoc).

The project contexts reveal that PRISMA’s impact on the complexity and uncertainty in systematic reviewing tasks differs by basic factors such as project size or collaborative configuration. This is because tasks are translated into responsibilities that can be distributed to collaborators or standards such as PRISMA. Involved actors must take the risks of potential errors and external critiques that may lurk in the remaining uncertainties. While bigger project contexts can divide these responsibilities among their collaborators, small projects don’t have such options. Instead, they shift as much task responsibility as possible to PRISMA. This could even lead to PRISMA being used excessively for planning and conducting practices that were not originally anticipated by its developers.

3.2. Prior research experience

Throughout the study, it became clear that Ph.D. students and early-career researchers frequently used PRISMA more intensively and even extended its scope. PhD students, in particular, were first introduced to PRISMA during workshops or seminars on systematic reviews or meta-analysis. Furthermore, some of their systematic review projects were part of their course assignments. However, early career researchers generally face higher levels of task uncertainty because they have less experience and need to develop their research habits. Interviewees mentioned that they stayed close to the guidance from supervisors, colleagues, or standards such as PRISMA. In that sense, Lamin had not encountered any reservations against PRISMA and explained that

It’s maybe because most of the people around me really are people who are looking to learn systematic reviews and have not conducted them before. So I guess I’m not in a circle where the skill level is high enough for people to have like different opinions and stuff like that. (Lamin, postdoc).

Traditionally, standardization has been characterized as a practice where ‘new ‘cooks’ tend to rely on it more than older ones’ (Fujimura 1987, p. 278; see also Zuckerman and Merton, 1981). In fact, bibliometric studies have indicated how higher PRISMA adoption rates correlate with lower academic age (Schniedermann, 2023). Besides its role in reducing the task uncertainty in dissertation projects, the PRISMA guideline also turns the proficiency in systematic reviewing into a tangible and communicable skill or even a fulltime profession (see also Sambunjak and Puljak, 2010; Garfield, 1977). Mastering this skill opens new career paths for early career researchers (Fujimura, 1988). In this sense, interviewed early career researchers were more aware of PRISMA’s official narrative and its links to issues such as reproducibility, accessibility, or the credibility of research.

In contrast to the in-depth usage of PRISMA by early career researchers, more senior researchers can face social and epistemic tensions that have a stronger influence on their practices and decisions than immediate task uncertainties. For example, project leaders are pressured to leverage their prior experience with PRISMA and cannot spend additional time on consulting the guideline during conduct. They ‘know most, maybe all, items by heart (Esther) or have the guideline ‘in the back of [their] mind’ (Marco) when planning and conducting the review. Although this behavior is an efficient form of standard usage, it can be prone to error and is slower in applying new items or versions. Not surprisingly, some interviewees did not use the most recent PRISMA version for their project.

Experienced researchers are also able to abstract and modify the epistemic function of PRISMA in order fit it better with other requirements or already established modes of work. Instead of seeking for the greatest possible reduction of task uncertainty, they translate moments of uncertainty into opportunities for creativity and niche building. Illustratively, Adam, a postdoc in psychiatry and practicing doctor compares his deviation from PRISMA with improvising over a standard set of chords as musician:

‘I was playing piano in my church last night, and it made me think about how I learned the chords and the chord structures. And now, when I play, I can break the chords and chord structures and play slightly more jazzily, you know, with the safety of I do know what to do, like, what the basic kind of chords are… So, I think that having those straight chord structures available is very, very important for people to be able to legitimately enter into producing systematic reviews. And then perhaps some people will try to do that differently.’ (Adam, postdoc from UK).

PRISMA users can choose different strategies to deal with the task uncertainty. Unexperienced researchers usually welcome as much uncertainty reduction as possible and want to display methodological proficiency which supports their career trajectories. In contrast, experienced researchers may choose to focus on their research contents and try to fit PRISMA into their project contexts.

Conclusions

The standardization of scientific reporting shapes the industrial production of biomedical knowledge. Although reporting guidelines turn transparency into a doable problem for authors, they do not reduce the complexity or task uncertainty of systematic reviewing in a uniform or mechanistic way. Instead, authors limit or extend the scope of the guideline and transform it into a research tool that serves their needs. Similarly, although PRISMA suggests a redefinition of the systematic reviewing genre by defining information as either required or additional, authors confidently deviate or make use of additional venues to serve their communicative needs, such as protocols, websites, or supplementary files. Taken together, the question of which systematic reviewing practices are affected, changed or left unchanged cannot be answered straightforwardly.

The complexities and contingencies involved in applying PRISMA pose a risk of misinterpreting attempts to reform science. PRISMA provides clear and distinct rules that suggest reporting a systematic review can be done in a uniform and procedural manner, or even mechanistic and ‘post-human’ task (Peterson and Panofsky, 2020, p. 20). By making the ideal picture of science more idealistic, it made actual research look worse. Subsequently, any deviations from the script may be perceived as inappropriate or erroneous. However, the interviews revealed that such deviations are modulations fuelled by the individual and contextual factors of systematic review projects. Furthermore, we can identify recurring patterns of these modulations when the contextual and individual factors are similar. This suggests that attempts to standardise scientific writing lack a thorough and systematic understanding of the tasks, practices and problems involved in writing. In other words, it is unclear what was actually standardised by reporting guidelines. The separation of conduct and reporting, which is important for promotion but irrelevant in practice, suggests a similar conclusion.

But then, what is PRISMA’s contribution to the progress of the biomedical research system, especially if the types of usage can differ so heavily? We have seen that the impact of PRISMA extends beyond any particular practice; rather, it aligns and connects different tasks, skills, and textual genres. In doing so, it establishes a socio-technical template for systematic reviewing as a profession within medical research. This profession legitimates new career opportunities and forms its own community that tells new epistemic stories, defines quality criteria, and develops further standardizations (Fujimura, 1988; Bowker and Star 1999; Millerand et al. 2013). Therefore, instead of deskilling research experts – a common criticism of industrial standardization – standards for systematic reviewing rather reskill researchers and allow them to chase new paths. Planning reviews and creating protocols, developing search strategies, or creating standardized reports become specialized tasks associated with systematic reviewing (see also Sambunjak and Puljak, 2010; see also Bennett et al., 2023).

In addition, standards such as reporting guidelines have important semiotic or signifying functions. Reporting guidelines are visible in the codified products of science and thus communicate compliance irrespective of their contexts of application. It is further their signalling function and the respective social expectations by researchers, readers, editors, or peer reviewers that becomes changed in a more straightforward or even homogeneous way. Thus, although standardization in science requires to perform bureaucratic acts (see Penders, 2022), the effectiveness and success of standardization on a macro scale depends much on communicating compliance. Standardization must manifest itself in the codified record of science. Thus, further attempts to understand standardization in science cannot rely on a systematization of how practices correlate with contexts alone. Rather, analyzing the published record reveals how new standards spread and become successful on a global level.

Acknowledgements

The author wants to thank Wolfgang Kaltenbrunner, Sven Arend Ulpts, and two anonymous reviewers for helpful suggestions and comments to earlier versions of this text. This study is part of a project about the role of review articles in science, funded by the German Federal Ministry of Education and Research (01PU17017). Sampling interview partners was supported by the German Kompetenznetzwerk Bibliometrie (16WIK2101B). Initial results have been presented at the conference panel “Contested Conduct”, organized by Melpomeni Antonakaki, held at the sts-hub conference in Aachen, March 2023.

References

Bazerman, C., 1988. Shaping Written Knowledge: The Genre and Activity of the Experimental Article in Science. University of Wisconsin Press, Madison, Wisconsin.

Bennett, A., Garside, D., Gould Van Praag, C., Hostler, T.J., Kherroubi Garcia, I., Plomp, E., Schettino, A., Teplitzky, S., Ye, H., 2023. A Manifesto for Rewarding and Recognizing Team Infrastructure Roles. JOTE 4, 60–72. https://doi.org/10.36850/mr8

Birch, K., Muniesa, F. (Eds.), 2020. Assetization: Turning Things into Assets in Technoscientific Capitalism. The MIT Press. https://doi.org/10.7551/mitpress/12075.001.0001

Bowker, G.C., Star, S.L., 1999. Sorting things out: classification and its consequences, Inside technology. MIT Press, Cambridge, Mass.

Castañeda, O.J.M., 2015. Making evidence, making legitimacy: Lancaster University.

Castel, P., 2009. What’s Behind a Guideline?: Authority, Competition and Collaboration in the French Oncology Sector. Soc Stud Sci 39, 743–764. https://doi.org/10.1177/0306312709104435

Caulley, L., Catalá-López, F., Whelan, J., Khoury, M., Ferraro, J., Cheng, W., Husereau, D., Altman, D.G., Moher, D., 2020. Reporting guidelines of health research studies are frequently used inappropriately. Journal of Clinical Epidemiology 122, 87–94. https://doi.org/10.1016/j.jclinepi.2020.03.006

Chalmers, I., Hedges, L.V., Cooper, H., 2002. A Brief History of Research Synthesis. Eval Health Prof 25, 12–37. https://doi.org/10.1177/0163278702025001003

Epstein, S., 2007. Beyond the Standard Human, in: Lampland, M., Star, S.L. (Eds.), Standards and Their Stories: How Quantifying, Classifying, and Formalizing Practices Shape Everyday Life. Cornell University Press, pp. 35–53.

Erlach, K., 2022. Die Industrialisierung des Wissenschaftsbetriebs, in: Mößner, N., Erlach, K. (Eds.), Kalibrierung der Wissenschaft: Auswirkungen der Digitalisierung auf die wissenschaftliche Erkenntnis, Science Studies. transcript Verlag, Bielefeld, Germany, pp. 183–206. https://doi.org/10.14361/9783839462102

Field, S.M., Derksen, M., 2021. Experimenter as automaton; experimenter as human: exploring the position of the researcher in scientific research. Euro Jnl Phil Sci 11, 11. https://doi.org/10.1007/s13194-020-00324-7

Freese, J., Peterson, D., 2018. The Emergence of Statistical Objectivity: Changing Ideas of Epistemic Vice and Virtue in Science. Sociological Theory 36, 289–313. https://doi.org/10.1177/0735275118794987

Fujimura, J.H., 1988. The Molecular Biological Bandwagon in Cancer Research: Where Social Worlds Meet. Social Problems 35, 261–283. https://doi.org/10.2307/800622

Fujimura, J.H., 1987. Constructing `Do-able’ Problems in Cancer Research: Articulating Alignment. Soc Stud Sci 17, 257–293. https://doi.org/10.1177/030631287017002003

Fuller, T., Pearson, M., Peters, J., Anderson, R., 2015. What Affects Authors’ and Editors’ Use of Reporting Guidelines? Findings from an Online Survey and Qualitative Interviews. PLoS ONE 10, e0121585. https://doi.org/10.1371/journal.pone.0121585

Garfield, E., 1977. Proposal For a New Profession: Scientific Reviewer. Essays of an Information Scientist, Current Contents 3, 84–87.

Gaudillière, J.-P., Löwy, I. (Eds.), 1998. The Invisible Industrialist: Manufactures and the Production of Scientific Knowledge. Palgrave Macmillan UK, London. https://doi.org/10.1007/978-1-349-26443-8

Harvey, P., Reeves, M., Ruppert, E., 2013. Anticipating Failure: Transparency devices and their effects. Journal of Cultural Economy 6, 294–312. https://doi.org/10.1080/17530350.2012.739973

Hedgecoe, A., Job, K., Clarke, A., 2023. Reflexive standardization and the resolution of uncertainty in the genomics clinic. Soc Stud Sci 53, 358–378. https://doi.org/10.1177/03063127231154863

Hunt, M., 1999. How science takes stock: the story of meta-analysis, 1. paperback cover ed. ed. Russell Sage Foundation, New York, NY.

Keating, P., Cambrosio, A., 2012. Cancer on trial: oncology as a new style of practice. The University of Chicago Press, Chicago.

Lampland, M., Star, S.L. (Eds.), 2009. Standards and their stories: how quantifying, classifying, and formalizing practices shape everyday life. Cornell University Press, Ithaca.

Law, J., 2011. Collateral realities, in: The Politics of Knowledge. Routledge, London.

Law, J., Ruppert, E., 2013. The Social Life of Methods: Devices. Journal of Cultural Economy 6, 229–240. https://doi.org/10.1080/17530350.2013.812042

Maldonado Castañeda, O.J., 2019. (Un)contested evidence: scientific literature, systematic reviews and the politics of evidence in the introduction of HPV vaccines in Colombia. Sociol Health Illn 41, 81–94. https://doi.org/10.1111/1467-9566.12808

Meldrum, M.L., 2000. A Brief History of the Randomized Controlled Trial. Hematology/Oncology Clinics of North America 14, 745–760. https://doi.org/10.1016/S0889-8588(05)70309-9

Millerand, F., Ribes, D., Baker, K.S., Bowker, G.C., 2013. Making an Issue out of a Standard: Storytelling Practices in a Scientific Community. Science, Technology, & Human Values 38, 7–43. https://doi.org/10.1177/0162243912437221

Moher, D., Liberati, A., Tetzlaff, J., Altman, D.G., The PRISMA Group, 2009. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ 339, b2535–b2535. https://doi.org/10.1136/bmj.b2535

Moreira, T., 2007. Entangled evidence: knowledge making in systematic reviews in healthcare. Sociology of Health & Illness 29, 180–197. https://doi.org/10.1111/j.1467-9566.2007.00531.x

Müller, R., de Rijcke, S., 2017. Thinking with indicators. Exploring the epistemic impacts of academic performance indicators in the life sciences. Research Evaluation 26, 157–168. https://doi.org/10.1093/reseval/rvx023

Page, M.J., McKenzie, J.E., Bossuyt, P.M., Boutron, I., Hoffmann, T.C., Mulrow, C.D., Shamseer, L., Tetzlaff, J.M., Akl, E.A., Brennan, S.E., Chou, R., Glanville, J., Grimshaw, J.M., Hróbjartsson, A., Lalu, M.M., Li, T., Loder, E.W., Mayo-Wilson, E., McDonald, S., McGuinness, L.A., Stewart, L.A., Thomas, J., Tricco, A.C., Welch, V.A., Whiting, P., Moher, D., 2021. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ n71. https://doi.org/10.1136/bmj.n71

Page, M.J., Moher, D., 2017. Evaluations of the uptake and impact of the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) Statement and extensions: a scoping review. Syst Rev 6, 263. https://doi.org/10.1186/s13643-017-0663-8

Page, M.J., Shamseer, L., Tricco, A.C., 2018. Registration of systematic reviews in PROSPERO: 30,000 records and counting. Syst Rev 7, 32. https://doi.org/10.1186/s13643-018-0699-4

Penders, B., 2022. Process and Bureaucracy: Scientific Reform as Civilisation. Bulletin of Science, Technology & Society 42, 107–116. https://doi.org/10.1177/02704676221126388

Penders, B., Lutz, P., Shaw, D.M., Townend, D.M.R., 2020. Allonymous science: the politics of placing and shifting credit in public-private nutrition research. Life Sci Soc Policy 16, 4. https://doi.org/10.1186/s40504-020-00099-y

Peterson, D., Panofsky, A., 2021. Self-correction in science: The diagnostic and integrative motives for replication. Social Studies of Science 23.

Peterson, D., Panofsky, A., 2020. Metascience as a scientific social movement. https://doi.org/10.31235/osf.io/4dsqa

Rennie, D., 1995. Reporting randomized controlled trials. An experiment and a call for responses from readers. JAMA: The Journal of the American Medical Association 273, 1054–1055. https://doi.org/10.1001/jama.273.13.1054

Sambunjak, D., Puljak, L., 2010. Cochrane systematic review as a PhD thesis: an alternative with numerous advantages. Biochem Med 319–326. https://doi.org/10.11613/BM.2010.041

Sarkis-Onofre, R., Catalá-López, F., Aromataris, E., Lockwood, C., 2021. How to properly use the PRISMA Statement. Syst Rev 10, 117, s13643-021-01671-z. https://doi.org/10.1186/s13643-021-01671-z

Schniedermann, A., 2023. Who writes what? The academic age patterns of review genres in biomedicine. https://doi.org/10.55835/6441b6d076bb0bb2c9ff4c15

Schniedermann, A., 2022. Shaping the Qualities, Values and Standards of Science. How Reporting Guidelines Improve the Transparency of Biomedical Research. Front. Res. Metr. Anal. 7, 846822. https://doi.org/10.3389/frma.2022.846822

Schniedermann, A., Blümel, C., Simons, A., 2022. On Top of the Hierarchy: How Guidelines Shape Systematic Reviewing in Biomedicine, in: Ehlers, S., Esselborn, S. (Eds.), Evidence in Action between Science and Society: Constructing, Validating and Contesting Knowledge, Routledge Studies in the History of Science, Technology and Medicine. Routledge, London/New York, pp. 102–126.

Shamseer, L., Hopewell, S., Altman, D.G., Moher, D., Schulz, K.F., 2016. Update on the endorsement of CONSORT by high impact factor journals: a survey of journal “Instructions to Authors” in 2014. Trials 17, 301. https://doi.org/10.1186/s13063-016-1408-z

Sigl, L., 2016. On the Tacit Governance of Research by Uncertainty: How Early Stage Researchers Contribute to the Governance of Life Science Research. Science, Technology, & Human Values 41, 347–374. https://doi.org/10.1177/0162243915599069

Sismondo, S., 2009. Ghosts in the Machine: Publication Planning in the Medical Sciences. Soc Stud Sci 39, 171–198. https://doi.org/10.1177/0306312708101047

Solomon, M., 2015. Making medical knowledge, First edition. ed. Oxford University Press, Oxford.

Suchman, L.A., 1985. Plans and situated action. Xerox Corporation, Palo Alto.

Timmermans, S., Berg, M., 2003. The gold standard: the challenge of evidence-based medicine and standardization in health care. Temple University Press, Philadelphia, Pa.

Timmermans, S., Epstein, S., 2010. A World of Standards but not a Standard World: Toward a Sociology of Standards and Standardization. Annu. Rev. Sociol. 36, 69–89. https://doi.org/10.1146/annurev.soc.012809.102629

Van Drimmelen, T., Slagboom, M.N., Reis, R., Bouter, L.M., Van Der Steen, J.T., 2024. Decisions, Decisions, Decisions: An Ethnographic Study of Researcher Discretion in Practice. Sci Eng Ethics 30, 59. https://doi.org/10.1007/s11948-024-00481-5

Vazire, S., 2018. Implications of the Credibility Revolution for Productivity, Creativity, and Progress. Perspectives on Psychological Science 13, 411–417.

Whitley, R., 1984. The Intellectual and Social Organization of the Sciences, 2. Edition (2000). ed. Oxford University Press, Oxford England ; New York.

Zuckerman, H., Merton, R.K., 1981. Age, Aging and Age Structure in Science, in: Riley, M.W., Johnson, M., Foner, A. (Eds.), A Theory of Age Stratification. Russell Sage Foundation, New York, pp. 292–356.

Zuiderent-Jerak, T., 2021. STS as a third Space between Evidence-based Medicine and the human Sciences, in: Zuiderent-Jerak, T., Downey, G.L. (Eds.), Making & Doing. Activating STS through Knowledge Expression and Travel. The MIT Press, pp. 197–218.

Notes

[1] An overview can be found in the supplementary material.

[2] Protocols are project descriptions which are publicly registered. This somehow established practice promises to make unpublishable results such as negative findings visible. PRISMA-P is specifically designed for writing protocols but the main version of PRISMA is used for this step too.

[3] An illustrative example which goes beyond the scope of this analysis can be fund in Cochrane Systematic Reviews. These can accumulate up to thousands of pages of documentation included in the Cochrane database. However, they are occasionally published as highly abbreviated version in common academic journals.