Published at MetaROR

November 19, 2026

Table of contents

Available versions
Cite this article as:

Barnett, A., & Spick, M. (2026, August 20). Who chooses open peer review and is it an indicator of article quality? An observational study of PLOS journals. Retrieved from osf.io/preprints/metaarxiv/2b7z5_v2

Who chooses open peer review and is it an indicator of article quality? An observational study of PLOS journals

Adrian Barnett1 EmailORCID, Matt Spick2 ORCID

1 Australian Centre for Health Services Innovation and Centre for Healthcare Transformation, School of Public Health & Social Work, Queensland University of Technology, Kelvin Grove, QLD, Australia
2 School of Health Sciences, Faculty of Health and Medical Sciences, University of Surrey, Guildford, United Kingdom

Originally published on August 20, 2026 at: 

Abstract

PLOS journals allow authors of accepted articles to choose whether the peer review will be openly published alongside the article. This creates an observational study to examine the characteristics of authors and articles that more often choose open peer review, and whether open review is associated with measures of article quality. We examined over 115,000 PLOS articles and estimated what characteristics of the articles were associated with open peer review. We also examined if open peer review was associated with the subsequent retraction of the article and the number of citations.

Forty percent of articles chose open peer review. Authors from the UK, France, the Netherlands, and Ethiopia were more likely to choose open peer review. In contrast, authors from Saudi Arabia, the Republic of Korea, Pakistan, Poland, and China were less likely to choose open review. Authors with an edu email were less likely to choose open review, whilst authors with a gmail were more likely. Articles with open peer review were less likely to be retracted (adjusted hazard ratio = 0.75, 95% CI 0.60 to 0.93) and had more citations on average (adjusted rate ratio = 1.07, 95% CI 1.05 to 1.09).

In this exploratory study, we found clear differences in participation in open reviews with strong differences between countries. Authors who have confidence in their article and who engage in other open science practices may be more likely to chose open peer review, making this choice an indicator of article quality.

Introduction

Open science practices make the workings and results of a scientific study openly accessible [1]. Open science includes making a version of the final article openly available and curating the data and code used in the analyses (see [2] for a more complete range of open science practices).

Openness has collective benefits for science, including facilitating future work, fixing software errors [3], and combating research fraud [4, 5].

Despite encouragement from some institutions, funders, and publishers, open science practices are still not the norm [6, 7]. Some scientists are more open, possibly due to personal experiences and expectations in their field [8, 9]. Practising open science often requires additional effort; for example, the time needed to curate data that are findable and reusable [10–12]; hence, open science practices may be neglected because of the highly competitive scientific world [13].

Figure 1. The key stages at PLOS journals that leads to the authors’ decision on open peer review. Only articles that are reviewed and then accepted are included. PLOS reviewers are informed that their reviews could be published openly and are given the option of whether they would prefer to have their identity declared on the open review or if they would like the open review to appear anonymously.

Here we examine the open science practice of open peer review. The term “open peer review” covers many options, including open identities of reviewers and open reports where all peer reviews are published with the article [14]. Open reports were first made available by the publisher BMC in 1999 [15] and have since been implemented by more than 600 journals [16–18]. Open peer review increases the transparency of the peer review process and may provide more constructive and informative feedback [19].

Most journals either use or do not use open reports for all articles. PLOS journals are different, as since May 2019 they have allowed authors to choose to publish the peer review history or not [20] (see Figure 1). This creates observational data on authors who have chosen open peer review or not, allowing the examination of differences in author behaviour. Peer reviewers can decide to sign their review or not, but we do not examine that decision here.

The open review choice that we examine for PLOS authors is a relatively simple decision of whether to engage with open science or not. This choice happens at the end of the publication process after their article has been accepted and requires no additional effort from the authors, except from potentially canvassing the input of their co-authors. This contrasts with other open science practices that require more technical knowledge and time; for example, openly sharing data.

The authors’ decision to choose open review is likely to have multiple drivers. Authors may be more inclined to choose open peer review when their reviews were generally complimentary. Some decisions will also likely be influenced by the authors’ attitude towards open science practices.

Choosing open review would likely be more common among researchers who use other open science practices and authors from countries where funders and institutions support open science.

A recent study uncovered serious problems with peer review at PLOS ONE, with some editors and authors colluding to corrupt the peer review process [21]. Authors using such practices are likely to choose closed peer review to help conceal the evidence of collusion. In light of this, we examine whether open peer review was associated with the article being retracted or having an expression of concern. Retractions are rare, but can indicate serious quality problems [22].

Choosing open peer review might also be a general marker of quality and research competence, for example, being practised by authors who use other good practices and are more confident in their results. As a proxy for the quality of the article, we examined whether the article’s number of citations was associated with choosing open review.

Our research aims were to: 1) examine which article characteristics are associated with open review; and 2) whether open review is associated with retractions and citations.

Methods

Data extraction

We downloaded the entire corpus of PLOS articles in XML format on 1 March 2026 [23]. We excluded articles that were not peer reviewed, such as editorials and obituaries. We excluded articles that were accepted before the policy change concerning open peer review on 22 May 2019 [20].

The analysis data were extracted from the XML files. We extracted the number of authors and all the authors’ countries from their affiliations. For example, an article with authors from Australia and the UK would have a binary indicator for both countries. We did not include relatively rare countries that were mentioned in fewer than 100 articles as the estimates for these countries are likely to be highly uncertain.

We extracted all available subjects per article, with more than 10,000 subjects available in PLOS journals [24]. The online article displays eight subjects, and the XML version includes the broader subjects related to these eight. We excluded relatively rare subjects that were used in fewer than 500 articles. Forty subjects were excluded as they were highly correlated with another subject (r > 0.98). This meant that we included 857 subjects out of the total of 9,369 (9.1% included; see Appendix A.12 for the complete list).

We extracted the funders mentioned in every article. The funding information was available in the XML in three forms: the Crossref funder ID number(s) [25], the funding statement, and the funders’ name(s). We created a list of potential funders by selecting all funder IDs mentioned five or more times (thus excluding rare funders). We extracted the preferred name for each funder ID from Crossref and then searched the funding statement and funders’ names for matches. The final funder variable combined the funder ID and any text matches.

We extracted the number of authors with an ORCID, which is a freely available unique identifier that helps to disambiguate authors with similar names and facilitates links between authors and their articles [26]. Having an ORCID could indicate a willingness to engage with open science and transparency. PLOS has mandated ORCIDs for corresponding authors since 2016 and strongly encourages all other authors to provide their ORCID [27]. However, despite this encouragement, there was still a wide variance in the number of authors with an ORCID.

We extracted the dates of submission and acceptance to calculate the peer review time. We considered that a relatively short or long peer review time could indicate peer review quality and/or extended peer review, which could influence the authors’ decision regarding open peer review.

We considered that the authors’ experience could influence their decision to choose open peer review. As a proxy for experience, we extracted the last author’s publication counts on the date the article was submitted. We chose the last author as they are generally the most senior and potentially the most influential in any decisions about open review [28]. For 3% of the articles, the last author’s articles were not available, and we instead used the first author. The authors’ article counts were extracted from the OpenAlex bibliometric database [29].

Variable selection and estimation

Our analysis was descriptive. We aimed to investigate which characteristics of the articles and authors were associated with open peer review. A list of the variables potentially associated with open peer review and the reason for including them or not is given in Table A.1.

Our large sample size increased the risk of finding associations that were statistically significant but of little practical importance. We therefore used a conservative approach to variable selection that might miss some true associations, with the aim of focusing on the more practically important associations. The three stages of variable selection and estimation were as follows.

Stage 1: Non-linearity. There were five continuous or integer variables: date published, number of authors, proportion of authors with an ORCID, peer review time, and last author’s publication count. All five could plausibly have a non-linear association with open peer review; therefore, we used the fractional polynomial approach to find the best fitting linear or non-linear association for each variable [30].

Stage 2: Variable selection. We selected the subset of variables associated with open review using stability selection [31]. This combines the LASSO method of regularisation with the uncertainty of the bootstrap to help select the ideal amount of regularisation. We used 500 bootstrap resamples which should be sufficient for an accurate estimation of the selection probabilities [32]. We selected the variables that were in at least 75% of the bootstrapped LASSO selections with the aim of finding variables where we are confident of an association. We set the expected number of falsely selected variables to one.

Stage 3: Final model. We fitted a final model using the variables selected at stage 2. We checked for collinearity using the variance inflation factor and checked for influential observations using Cook’s distance and df-betas [33].

We used a standard regression model without a link function rather than logistic regression because we prefer estimates on an absolute probability scale rather than odds ratios. Our large sample size meant there was no concern about the normal assumption [34]; however, we compared the estimates from a logistic model in a sensitivity analysis.

For categorical independent variables, the reference category in the regression model was the combination of all categories not selected by the stability selection.

Retraction and citation data

Retraction data were downloaded from the “Annulled” detector that includes retraction data from Retraction Watch, crossref, and PubMed [35]. Expressions of concern were downloaded from the “Problematic Paper Screener” [36]. We included retractions for all reasons, including errors, fraud, and ethical concerns. We modelled the time from publication to retraction and/or expression of concern. For articles that had both an expression of concern and subsequent retraction, we used the expression of concern as this came first. Articles that were not retracted and had no expression of concern were right-censored on the follow-up date (8 August 2026). We plotted the cumulative incidence of retractions and expressions of concern over time and estimated the hazard ratio associated with open peer review using a Cox model [33]. We also fitted a model that adjusted for country, subject, publication date, peer review time, and number of authors.

The citation data were taken from the OpenAlex database [29] on 2 March 2026. Article citation counts were modelled as an over-dispersed Poisson distribution with time since publication as an offset to control for varying follow-up times [33]. Open peer review (yes/no) was the only independent variable; however, we also fitted a model adjusted for country, subject, publication date, peer review time, and number of authors. Citation counts are an imperfect measure of quality [37]; however, we assumed that in such a large sample there would be some signal to determine if articles where the authors chose open peer review were associated with more citations on average.

Missing data and data quality

The analysis data were almost complete. There were 721 articles (0.6%) with no subjects, 191 articles (0.2%) without publication counts for the first or last author, 103 articles (0.1%) where all authors’ countries were missing, and 2 articles without citation data. These articles were excluded from the regression models. These very small exclusions are unlikely to cause any appreciable bias.

We manually checked our data extraction for the binary dependent variable of open peer review for 100 articles with closed peer review and 100 articles with open review. In a broader verification of data quality, we manually checked all the extracted data for another 100 articles. There were no errors in any of these checks.

Other indicators of open science

To help contextualise the differences between countries in the probability of choosing open review, we extracted three country-level indicators of support for open science. These were the number of DORA signatories, the percentage of open access articles, and the rate of preprints.

The number of individual DORA signatories per country was collected from the DORA website on 6 April 2026 [38]. As an approximate adjustment for the number of researchers per country, we extracted the number of researchers per country using the most recent affiliation in OpenAlex. To try to restrict the numbers to currently active researchers, we restricted the OpenAlex sample to researchers with a mean number of citations in the last two years of 1 or more. We used this citation metric as it is available in OpenAlex but acknowledge that these numbers will be imperfect and only offer an approximation of the total number of recently active researchers per country. We did not use the number of organisational signatories of DORA (which universities and journals) as we could not find a good estimate of the number of institutions per country.

We estimated the rate of preprints per 10,000 articles per country using PubMed searches [39]. The search was restricted to the year 2019 onwards to cover approximately the same period as the open review policy change at PLOS. We verified that our estimated number of preprints had a high correlation with a previous analysis of country-level preprints [40].

The percentage of open access articles per country was estimated using OpenAlex searches [29]. The search was restricted to articles from May 2019 to cover the same period as the open review policy change at PLOS.

Sentiment analysis

A sentiment analysis was used to examine whether the PLOS reviews were atypically positive because authors who received negative reviews would have declined open review. To assess sentiment scores from selective open review against mandatory open review, we compared PLOS Medicine reviews to BMC Medicine reviews where all reviews are open. Both journals have a similar scope and Scimago Journal Rank (2.9 and 3.6 respectively, as of Q1 2025). Full articles were randomly selected from the years 2019 to 2025, retractions or corrections were excluded, as were records where fewer than two reviewer reports contained extractable text, and the overall population was resampled until 50 manuscripts were obtained from each journal. Each manuscript record contained between two and five individual reviewer reports provided as free text. Sentiment scoring was performed at the individual review level using the Valence Aware Dictionary and sEntiment Reasoner (VADER) [41], implemented using the Natural Language Toolkit in Python. VADER produces a compound score normalised to the range [−1, 1], where values approaching +1 indicate strongly positive sentiment and values approaching –1 indicate strongly negative sentiment. A manuscript-level score was derived by averaging the compound scores across all eligible reviews for that manuscript. Differences in compound scores between the two journals were assessed at both the review and manuscript levels using an unpaired t-test.

Reproducibility

Data extraction and analyses were conducted using R version 4.5.2 [42]; sentiment analysis performed using Python version 3.13.9 using the nltk, pandas, scipy, matplotlib, and seaborn libraries. The data and R code are available on GitHub [43].

The study was not preregistered and all analyses are exploratory.

Results

The analysis used 116,359 articles. The flow chart of the included and excluded articles is in Figure A.1. The characteristics of the articles included by open peer review are summarised in Table 1. Forty percent of the articles chose open peer review. Most were research articles and the most common journal was PLOS ONE.

Table 1. Descriptive statistics for the included articles by open review (n = 116,359). Cells show either the number (column percent) or median [Q1, Q3].
Open peer review
Variable Level (if applicable) No Yes
Number (row percent) 69,439 (60.0) 46,920 (40.0)
Year published 2022 [2021, 2024] 2022 [2021, 2024]
Number of authors 6 [4, 8] 6 [4, 9]
Last author’s publication count 77 [28, 168] 82 [31, 176]
Peer review time (days from submission to acceptance) 154 [106, 223] 154 [107, 223]
Article type Discovery Report 20 (<0.1) 25 (0.1)
Lab Protocol 116 (0.2) 98 (0.2)
Meta-Research Article 12 (<0.1) 38 (0.1)
Methods 71 (0.1) 81 (0.2)
Methods and Resources 59 (0.1) 144 (0.3)
Preregistered Research Article 1 (<0.1) 7 (<0.1)
Registered Report Protocol 131 (0.2) 122 (0.3)
Research Article 67,752 (97.6) 45,079 (96.1)
Review 3 (<0.1) 1 (<0.1)
Short Reports 116 (0.2) 162 (0.3)
Software 29 (<0.1) 36 (0.1)
Study Protocol 1115 (1.6) 1115 (2.4)
Update Article 14 (<0.1) 12 (<0.1)
PLOS journal Biology 703 (1.0) 1194 (2.5)
Climate 185 (0.3) 144 (0.3)
Complex Systems 29 (<0.1) 35 (0.1)
Computational Biology 2050 (3.0) 2318 (4.9)
Digital Health 388 (0.6) 333 (0.7)
Genetics 1317 (1.9) 1239 (2.6)
Global Public Health 1725 (2.5) 1689 (3.6)
Medicine 474 (0.7) 679 (1.4)
Mental Health 148 (0.2) 109 (0.2)
Neglected Tropical Diseases 2760 (4.0) 1478 (3.2)
ONE 57,231 (82.4) 36,257 (77.3)
Pathogens 2217 (3.2) 1300 (2.8)
Sustainability and Transformation 48 (0.1) 55 (0.1)
Water 164 (0.2) 90 (0.2)
Citations 7 [2, 16] 7 [2, 17]
Retracted 456 (0.7) 188 (0.4)
Expression of concern 96 (0.1) 65 (0.1)

Variables associated with open review

The variable selection started with 1228 potential variables and 21 were selected using the stability selection approach (Figure A.3). The estimated differences in open review probabilities for the categorical variables are shown in Figure 2 and Table A.8.

Compared with all other article types, there was a reduced probability of open peer review for Research articles.

There were five countries where choosing openness was less common and four countries where choosing openness was more common. Ethiopia had a much higher probability than average, whereas Saudi Arabia had a much lower probability.

Email domains from education (.edu) were associated with a lower probability than average of open review, while a gmail domain had a higher probability of open review.

Only six of the 857 subjects were selected as associated with open peer review. Articles on “Materials science” or “Ecology and environmental sciences” had a lower probability of open peer review. Four subjects were associated with an increased probability of open peer review: Epidemiology, Africa, Health care and Health care facilities.

Figure 2. Estimated differences in the probability of choosing open peer review by article type, email domain, subject, country and funder. The dots are the mean difference from the average probability and the horizontal lines are 95% confidence intervals for the difference. The dotted vertical line shows no difference from the average probability.

Articles with funding from the European Research Council were associated with an increased probability of open peer review.

Two of the five continuous/integer variables were selected: the publication date and the proportion of authors with an ORCID (Figure 3). There was a sharp increase in the uptake of open review after the policy was introduced in May 2019. An increase in the proportion of authors with an ORCID was associated with a steady increase in the probability of choosing open review, with an estimated probability of 0.014 below the average when no authors have an ORCID and 0.047 above average when all authors have an ORCID.

Figure 3. Estimated non-linear differences in the probability of choosing open peer review by publication date and the proportion of authors with an ORCID. The plots shows the mean difference from the average probability and 95% confidence interval for the difference as shaded areas. (A) Date published. The policy was introduced in May 2019. The mean difference in July 2019 was –0.17 and in October 2025 was 0.01. The best polynomial transform was inverse. (B) ORCID. The mean difference at just 1% of authors with ORCIDs is –0.014 and the mean difference at a proportion of all authors with an ORCID is 0.047. The best polynomial transform was squared.

The model checks are in Section A.8 and there were no concerns about collinearity or influential observations.

Open peer review associated with article quality

Articles that chose open peer review were much less likely to be retracted (Figure 4). There were 188 retractions for articles that selected open peer review and 456 retractions in the closed group. The hazard ratio for open compared to closed review was 0.62 (95% CI 0.52 to 0.73), a greatly reduced hazard of retraction. Conversely, the cumulative incidences for expressions of concern were similar for articles that chose open or closed review and the hazard ratio was 1.02 (95% CI 0.75 to 1.40).

To investigate whether the reduced hazard of retraction from open review was driven by a few authors with a large number of retractions, we removed six authors with 10 or more retractions (71 retractions removed in total). The hazard ratio reduced from 0.62 to 0.71 (95% CI 0.60 to 0.85).

A test of the proportional hazards assumption for retracted articles showed non-proportionality (p = 0.012); hence, we used a time-dependent hazard which showed that open review had a stronger effect on relatively early retractions (see Appendix A.9).

After adjusting for country, subject, publication date, peer review time, and number of authors, the hazard ratio for open peer review reduced to 0.95 (95% CI 0.82 to 1.10) and using stability selection open review was selected in only 13% of the bootstrap samples with some countries and subjects being frequently selected (see Appendix A.10). The association between open review and retraction is potentially explained by country and subject.

Figure 4. Kaplan–Meier survival curves for the time to retraction and/or expression of concern by open peer review status. The numbers on the right show the total number of retractions or expressions of concern per group. The total follow-up time was 428,018 years with an average time per article of 3.7 years.

Articles with open peer review had more citations than articles without open review. The rate ratio was 1.07 (95% CI 1.04 to 1.10) which is an estimated 7% increase in citations per year for articles with open peer review. After adjusting for country, subject, publication date, peer review time, and number of authors, the mean rate ratio for open peer review was unchanged at 1.07 (95% CI 1.05 to 1.09).

Sentiment analysis

A sentiment analysis comparing the wording of 50 reviews from PLOS Medicine (reviews selectively open) and BMC Medicine (all reviews open) did not show clear differences between the two journals. A t-test for the difference in sentiment had a mean of 0.01 (95% confidence interval: –0.18 to 0.21) at the review level and a mean difference of 0.02 (95% confidence interval: –0.21 to 0.24) at the article level.

Other country-level indicators of open science

For comparison with our open review results, we extracted three other open science indicators for the eight countries that had a higher or lower probability than the average open review (Table 2). China had the lowest rates for all three indicators. France had the highest rate of DORA signatories, the UK had the highest rate of preprints, and Ethiopia had the highest percentage of open access. The rates of DORA signatories are only an approximation given the difficulty of estimating each country’s number of researchers.

Discussion

Uptake of open peer review

Forty percent of the authors in this sample chose open peer review, which means that most of the exchanges between the authors and reviewers are lost. This decreases transparency and limits opportunities for meta-research. Of course, all interactions would be lost in closed peer review, so 40% is still an overall gain. However, this relatively low engagement with open review is concerning and indicates that open science is still far from the norm. The trend of opting for open peer review has also remained flat after an initial increase shortly after the policy was introduced (Figure 3), indicating no recent improvement in attitudes. PLOS could change open review to an opt out rather than opt in choice, which would likely increase the number of reviews available and create a natural experiment on this change.

Table 2. Open science indicators for the eight countries with a lower or higher than average probability of open review. The DORA signatories are individuals and the estimated rate is per 100,000 current researchers per country. The preprint rate is per 10,000 articles per country. Open access relates to published articles. The time period is 2019 to 2026, except for DORA signatories which is all years up to 2026.
Probability of open review Country Number of DORA signatories (rate) Preprint rate Percentage open access
Lower than average China 145 (10) 6 44
Pakistan 102 (203) 6 70
Poland 419 (714) 11 80
Saudi Arabia 47 (92) 7 64
South Korea 239 (183) 14 58
Higher than average Ethiopia 25 (124) 12 82
France 1350 (858) 29 61
Netherlands 437 (538) 35 80
UK 1977 (747) 47 71

Compared with other cross-sectional studies that examined open science practices, 40% is higher than the percent of articles that provide open data and code [7] or use preregistration [44], but lower than the percent of articles that included statements on funding and conflict of interest [44, 45]. The degree of compliance with open science practices will be partly explained by the ease of compliance and hence the generally lower compliance with practices that are seen as difficult or time-consuming. Agreeing to open review is not difficult or time-consuming and therefore potentially more closely reflects the authors’ general attitudes to open science. However, we note that this is our hypothesis and more direct evidence is needed to confirm this link.

Variables associated with open review

The strong positive association between the proportion of authors with an ORCID and the probability of choosing open review (Figure 3) is likely because some authors support open science and will be open wherever possible [9].

There were nine countries that had a lower or higher than average probability of open review. Four of the five countries with a low probability also had relatively low support for three other indicators of open science, although Poland was an exception (Table 2). The reduced probability of open review in these five countries could reflect a broader reluctance to be open. Another exception to the trend was Ethiopia, which had a high probability of open review and a very high percentage of open access, but with a low rate of DORA signatories and a low rate of preprints. We discussed our results with researchers from Poland and Ethiopia, but found no clear explanation for Poland’s surprising lack of open review and Ethiopia’s surprising excess of open review.

An unexpected result was that edu emails had a below average probability of choosing open peer review. The edu domain is for educational institutions anywhere in the world, but is used primarily in the United States [46]. Academia has experienced a concerted push for greater openness and transparency from funders and governments [47]. However, some academic researchers may be unwilling to share reviews that contained negative comments due to concerns that it would create questions about the credibility of their article. Some academic researchers may not want to publish their responses to the reviewers if they used “sham responses” [48] or accepted the reviewers’ suggestions simply to get published [49].

The slightly higher probability of open peer review associated with a gmail could be from those authors who have considered how to remain contactable after leaving their institution. Authors who welcome email queries about their article are potentially more likely to choose open peer review. However, this is our speculation, and there may be other explanations.

The increased probabilities for subjects of “Health care” and “Health care facilities” could reflect the longer history of open peer review in medicine [50], which means that open peer review is more normalised for authors in this field.

The lower probability for “Ecology and environmental sciences” could be due to the negative attention from the “climategate” email and Nature peer review controversies [51, 52].

Researchers in these fields may have experienced (either directly or vicariously) climate sceptics picking holes in the peer review process; hence, these researchers may feel less inclined to open peer review. In addition, this is a subject area that has experienced epistemic trespassing by non-experts [53] and these researchers may prefer not to be transparent. However, we do not have direct evidence of these purported explanations for the reduced probability of open review.

The positive influence of the European Research Council indicates the potential of funders to increase participation in open science. Some funders have mandated that researchers publish their results in open access journals or institutional repositories [54, 55], openly share data [56], and use an ORCID [57]. Funders have a strong influence on researcher behaviour, and this influence could be used more often with new policies that aim to increase transparency and robustness. However, we could not find a direct mandate in the policies of the European Research Council on open review, although it requires that all articles be deposited in an institutional repository [58]. A commitment to open access may have had a spill-over effect on other open science practices. In addition, the funder may have encouraged open review outside of policy documents; for example, in meetings or emails.

Variables not associated with open review

Some variables that were not associated with open review were interesting. The number of authors had no association, but we suspected that larger teams could be associated with greater unwillingness to be open, as sometimes it takes only one person to express strong doubts about the “safe” decision of closed peer review. Closed review is also the norm at most journals and hence open reviews may be seen by some as unusual or risky. The lack of association could be because the submitting author often made the decision alone.

The time in peer review was not associated with the choice of open review. Short times could indicate a compromised peer review process [21] and therefore would be associated with a lower probability of open review. Long peer review times might indicate conflicting and potentially negative reviews with comments that the authors would rather not publicly share [59].

Given the expected differences between fields, it was surprising that only six subjects were associated with open peer review. In particular, we were surprised that subjects associated with meta-research, such as “Research integrity” and “Science policy”, were not associated with increased open reviews, as researchers in this area should be more aware of open science.

Open review and quality

A review of open peer review stated that a key question was whether open peer review increases trust in articles [60]. Our findings partly support this hypothesis: articles with open peer review had more citations on average, and this association was unchanged after adjusting for country, subject, publication date, peer review time, and number of authors. Authors who chose open peer review may be engaging in other good science practices and are therefore happy to invite greater scrutiny. In relation to this, a recent observational study found that papers that were first published as a preprint were retracted about half as often as non-preprinted papers [61].

The crude association between open review and retraction was similarly strong, but did not persist after adjustment: country and subject were stronger predictors of retraction than open review status itself, and open review was selected in only 13% of bootstrap samples when predicting retraction directly (Appendix A.10). This suggests the crude association is largely explained by uneven distribution of retraction risk across countries and research fields, rather than by the act of choosing open review itself (Figure A.11).

One plausible driver of this country- and subject-level variation is differing exposures to questionable research practices (QRPs), including the kind of peer-review collusion recently uncovered at PLOS [21]. Authors who engaged in malpractice in the review process will likely want reviewers’ comments to be closed, as they may be tokenistic; and QRPs more broadly – including paper-mill activity – are not evenly distributed, but have been previously described as clustering by country and by research field [62–64]. If country and subject act as proxies for the local prevalence of QRPs, this would explain why they predict retraction more strongly than an individual author’s choice about peer review transparency, without implying that researchers in any particular country are individually more prone to misconduct: the more plausible explanation is that research incentive structures and oversight vary systematically by setting.

This may also help explain our expression-of-concern result. We found no association between open peer review and expressions of concern, which is hard to reconcile with open review simply reflecting author confidence. One possibility is that the two outcomes reflect different kinds of problems: the more obvious errors, plagiarism, or misconduct that lead to rapid retraction may be precisely what an author engaging in QRPs would want to keep hidden by choosing closed review, whereas expressions of concern may mark the start of a longer, more uncertain investigation into problems – including honest statistical or methodological errors – that have little to do with an author’s transparency preferences. Consistent with this, open review’s protective association with retraction was strongest for early retractions and weakened over time (Appendix A.9). We cannot test this explanation directly with the data available, but it is consistent with both the timing pattern and the null expression-of-concern results that we found in this work.

Researchers could still consider open peer review a useful signal for citations, where the association held up under adjustment. For retraction, however, our results suggest that country and research subject (which are plausibly markers of local research incentives and/or topics that are more prone to QRPs) are the more informative signal, and open peer review status alone should not be read as a quality marker for retractions.

Related work

A scoping review on the impacts of open science found that open evaluation studies were relatively rare [65]. Most studies in open evaluation examined whether the reviewers’ identities were open or not, and the impact of openness on the quality of the review [65].

Two previous observational studies agreed with our finding that open review articles were associated with increased citations [66, 67]; however, we cannot compare the size of our estimated increase, as we estimated the change in citations as a yearly rate. A third study found the opposite, as open review was associated with fewer citations at the journal Nature Communications, with no difference in citations at PLOS ONE [68]. However, this study inexplicably excluded citation counts of 0 and 1, which in our data were 19% of the citations; hence, excluding these low counts could introduce a large bias.

A large observational study using PLOS and BMC articles found an increased citation rate for articles where data were declared to be available compared to articles without a data sharing statement [69]. This could be a causal link, via researchers re-using the data and therefore citing the original article, but it may also be partly due to higher quality articles being more likely to share their data and also accrue more citations.

The decisions we examined here are from the authors’ perspective, not the reviewers. Previous research has investigated the willingness of reviewers to participate in open peer review [14].

Limitations

We did not have data on whether all peer reviews were favourable or not. Most authors strongly prefer favourable reviews [70] and many authors may have chosen closed peer review when reviews were critical, even though their article was accepted. However, we did not find a clear difference between the sentiment of PLOS Medicine reviews, which may or may not be open, and BMC Medicine reviews that are all open.

Our statistical selection of what variables to include was somewhat arbitrary, including the 0.75 selection probability (Figure A.3). Choices other than 0.75 would have given different predictor variables. We chose this relatively high threshold because our approach was to focus on stronger associations that are likely to be more practically meaningful; however, we may have missed true associations with open peer review, although we suspect these associations will have a relatively small strength given our sample size and statistical power to detect small associations.

We included a binary variable for any country in the author list, but could have instead used a proportional weight according to the number of authors. This may have more accurately weighted the countries that had a greater influence in the choice of open review.

Our regression models assumed that all articles were independent and did not account for the correlation of articles by the same authors. To examine this, we randomly sampled 100 included articles which had 659 authors and found only one author on multiple articles. Even when articles share a common author, the average correlation in outcomes will not be 1 because of the influence of other authors and differences in the articles. Hence, any intraclass correlation from repeated authors would be small and unlikely to substantively change our estimates.

Our large sample size means that the confidence intervals around the estimates are narrow. However, there is model uncertainty and likely unmeasured confounding [71]. In particular, it was challenging to map the funder data and there will be some measurement error in this variable. Hence, all our estimates should be considered as indicators of the size and direction of associations.

The associations we show are likely not causal. Two examples: 1) a researcher choosing open peer review should not expect a 7% increase in their citations; 2) winning funding from the European Research Council (ERC) may not directly increase the probability of open review as researchers who win ERC funding may be more amenable to open science. The purpose of our analysis was to find associations to inform discussions about the potential associations between articles, authors, research quality, and open review.

Acknowledgements

Thanks to PLOS for making their data openly available for research. Thanks to Iain Hrynaszkiewicz from PLOS for providing helpful feedback on a draft. Thanks to Dalmeet Singh Chawla for suggesting an opt-out (rather than opt-in) approach to open peer review.

Thanks to MetaROR for arranging peer review. This version was updated in response to the reviewers’ comments that are available here.

Funding

There was no funding for this work.

Competing interests

The authors declare no competing interests.

Data availability statement

The data and R code are available on GitHub [43].

AI use declaration

We used Writefull to help with spelling and grammar. We used Paper-Wizard to peer review a draft of the article. Claude Code was used to help with text sentiment analysis.

Author contributions

Conceptualisation, Methods, Software, Validation, Data curation, Writing – Original Draft: AB Conceptualisation, Methods, Writing – Review & Editing: MS

Appendices

A.1 Variables potentially associated with the authors choosing open review.

Table A.1. Variables that could be associated with authors choosing open peer review and the reasons for including them or not in our statistical model. This is not a comprehensive list of all variables associated with open review, but covers those variables available in the PLOS data.
Variable (number of categories) Reason for inclusion or exclusion
Continuous/integer variables
Date published Expected a change in uptake over time after the policy change. An increase over time was found in an analysis by PLOS [72].
Number of authors Larger author teams may have a lower uptake as there are more people to convince.
Peer review time Short or long peer review times could influence the willingness to choose open review, as they may indicate an atypical peer review.
Authors experience Less experienced authors may be more unsure about open reviews; we used the last author’s publication counts as a proxy for experience.
ORCID Authors with an ORCID are likely more engaged with open science prac­tices.
Categorical variables
Country (n = 117) Expected differences between countries in open science practices [73]. Geographic region has been examined by PLOS and it was associated with open review [72].
Subject (n = 857) The authors’ scientific fields likely influences their attitudes and experiences with open science.
Article type (n = 13) The peer review process can differ by article type which could influence the decision to choose open review.
Email domain (n = 22) The authors’ given email domain(s), including gmail, gov, and edu, could be an indicator of attitudes towards open science. We did not include country domains, e.g., “.fr” as this information is already captured by the authors’ countries.
Funder (n = 212) Funders can strongly encourage or even mandate open science practices.
Journal (n = 14) We decided not to include journal as we expected subject to be the more proximal variable with no direct association from journal to open peer review. Journals were examined by PLOS and were found to be associated with open review [72].

A.2 Email processing.

Table A.2. Five examples of how emails were changed into domains
Example email Domain
a.barnett@qut.edu.au edu
matt.spick@surrey.ac.uk ac
Dispy@yahoo.com yahoo
Po@harvard.edu edu
Laa-Laa@up.lublin.pl Other

A.3 Included and excluded articles

Figure A.1. Flow chart of included and excluded articles.

The main reason why articles were excluded is that they were published before the policy change on open peer review. Other articles were excluded because they were not peer reviewed, e.g., Corrections and Perspectives.

A.4 Frequency tables for categorical variables

A.4.1 Top ten authors’ countries

Table A.3. Top ten authors’ country by open peer review. Cells show the number and row percentage.
Open peer review
Country No Yes
United States of America 20067 (61) 12853 (39)
China 12330 (68) 5822 (32)
United Kingdom 6930 (52) 6427 (48)
Germany 4652 (59) 3214 (41)
Canada 3834 (59) 2648 (41)
Japan 3276 (55) 2671 (45)
Australia 3249 (55) 2615 (45)
France 2488 (52) 2267 (48)
Brazil 2431 (61) 1549 (39)
Netherlands 1963 (50) 1939 (50)

Many articles had multiple authors from the same country, but each country was counted once per article. We only show the top ten countries and there were 221 countries in total. For statistical models, we used 117 countries, as those with less than 100 articles were not included.

A.4.2 Top ten subjects

Table A.4. Top ten most frequent subjects by open peer review. Cells show the number and row percentage.
Open peer review
Subject No Yes
Biology and life sciences 60078 (60) 40527 (40)
Medicine and health sciences 52342 (59) 36939 (41)
Research and analysis methods 34684 (60) 23336 (40)
Physical sciences 25925 (61) 16274 (39)
Social sciences 25123 (59) 17426 (41)
People and places 21530 (57) 15966 (43)
Organisms 21144 (61) 13644 (39)
Medical conditions 17953 (58) 13149 (42)
Physiology 15945 (59) 10922 (41)
Psychology 15666 (58) 11117 (42)

Each article had multiple subjects. For the statistical models, we used 857 subjects as we excluded those with under 500 articles or with a strong correlation with a related subject.

A.4.3 Email domains: top 10 for open peer review and by country

Table A.5. Top ten most frequent email domains by open peer review. Cells show the number and row percentage.

Open peer review

Domain No Yes
edu 23682 (64) 13561 (36)
Other 16021 (58) 11640 (42)
ac 10082 (55) 8091 (45)
gmail 8692 (54) 7309 (46)
163 3308 (68) 1592 (32)
com 2444 (63) 1413 (37)
org 2433 (57) 1834 (43)
yahoo 1732 (56) 1358 (44)
gov 1418 (65) 754 (35)
uni 1248 (62) 777 (38)

This is the email domain extracted from the authors’ given emails. There were 22 email domains in total. Some articles had multiple emails, but, in general, not all authors included their email.

Table A.6. Top ten countries using gmail and edu email domains by percentage. Cells show the number and row percentage.
gmail edu
Country N % Country N %
Ethiopia 2400 75 Kuwait 105 66
Nepal 339 58 Jordan 336 65
India 1089 36 Saudi Arabia 1402 64
Uganda 395 34 United States 20,456 62
Tanzania 304 33 Lebanon 192 59
Iran 490 32 Iraq 146 59
Nigeria 338 30 Malaysia 978 59
Kenya 415 30 Pakistan 1230 58
Bangladesh 439 29 Egypt 602 58
Brazil 1138 29 Colombia 401 56

A.4.4 Top ten funders

Table A.7. Top ten most frequent funders by open peer review
Open peer review
Funder No Yes
National Institutes of Health 4628 (60) 3126 (40)
National Natural Science Foundation of China 3623 (69) 1620 (31)
National Science Foundation 2256 (57) 1693 (43)
National Institute of Allergy and Infectious Diseases 1396 (62) 865 (38)
National Research Foundation, South Africa 1169 (65) 642 (35)
Japan Society for the Promotion of Science 1124 (56) 896 (44)
German Research Foundation 992 (58) 729 (42)
Horizon 2020 971 (52) 903 (48)
Medical Research Council 899 (47) 1017 (53)

A.5 Non-linearity

Figure A.2. Potential non-linearity in the association between article characteristics and open peer review. The plot shows the difference from the best model in the Akaike Information Criterion (AIC) for five continuous or integer variables across the eight fractional polynomials. The best model is that with the lowest AIC.

A difference in the AIC of 10 is considered large [74]. Of the five variables, only peer review time had no large AIC differences. The best fit for the other four variables is a non-linear association.

A.6 Stability selection

Figure A.3. Probability estimates from the stability selection used to select the variables associated with open peer review. To improve readability, only variables with a greater than 0.25 selection probability are shown (there were 1228 candidate variables). The threshold for inclusion in the final model was a probability of 0.75 and included variables are shown in green. The probability was estimated over 500 bootstrap resamples. There were nine variables that were selected in all 500 bootstrap samples.

A.7 Model estimates

Table A.8: Difference in the probability of choosing open peer review for the 19 selected categorical variables. The estimates are means and 95% confidence intervals. For each variable group the reference category is all other levels not included in the model; for example, all other countries apart from the nine selected.

 
Variable group Variable level (sample size) Mean (95% CI)
Article type Research Article (n=112,831) –0.10 (–0.12, –0.08)
Country Ethiopia (n=3,193) 0.10 (0.08, 0.12)
Country Netherlands (n=3,902) 0.07 (0.05, 0.08)
Country United Kingdom (n=13,357) 0.06 (0.05, 0.07)
Country France (n=4,755) 0.06 (0.04, 0.07)
Country Pakistan (n=2,106) –0.06 (–0.09, –0.04)
Country China (n=18,152) –0.07 (–0.07, –0.06)
Country Republic of Korea (n=3,053) –0.07 (–0.09, –0.05)
Country Poland (n=1,510) –0.09 (–0.12, –0.07)
Country Saudi Arabia (n=2,178) –0.10 (–0.12, –0.07)
Email domain gmail (n=16,001) 0.03 (0.02, 0.04)
Email domain edu (n=37,243) –0.03 (–0.03, –0.02)
Funder European Research Council (n=1,500) 0.09 (0.07, 0.12)
Subject Health care facilities (n=5,844) 0.03 (0.02, 0.05)
Subject Africa (n=7,140) 0.03 (0.02, 0.04)
Subject Health care (n=20,349) 0.03 (0.02, 0.04)
Subject Epidemiology (n=21,830) 0.02 (0.01, 0.03)
Subject Materials science (n=6,941) –0.03 (–0.05, –0.02)
Subject Ecology & environmental sciences (n=10,724) –0.03 (–0.04, –0.02)

A.8 Model checks

Model check 1: Variance inflation factor

Variable VIF
Date published 1.03
ORCID proportion 1.03
type = Research Article 1.01
domain = edu 1.13
domain = gmail 1.18
country = China 1.09
country = Ethiopia 1.24
country = France 1.02
country = Netherlands 1.02
country = Pakistan 1.08
country = Poland 1.01
country = Republic of Korea 1.02
country = Saudi Arabia 1.09
country = United Kingdom 1.06
subject = Africa 1.22
subject = Ecology and environmental sciences 1.02
subject = Epidemiology 1.05
subject = Health care 1.39
subject = Health care facilities 1.36
subject = Materials science 1.02
funder = European Research Council 1.03

All variance inflation factors are close to 1, indicating that there are no concerns about collinearity.

Model check 2: Cook’s distance

Figure A.4. Histogram of Cook’s distances

The largest Cook’s distance was 0.0003. Some of the largest Cook’s distances were for articles where the European Research Council was a funder.

The largest df-betas were for articles where the European Research Council was a funder. The largest change in probability was 0.0005, which is small and does not change our results or interpretations.

Given the potential influence of the European Research Council, we ran the model without this predictor and compared the estimates. A Bland–Altman plot of the differences in the 20 estimates (continuous/integer and categorical) is in Figure A.5. The largest differences were for three European countries where without the European Research Council the estimates for all three countries were larger. However, the differences in probabilities were small, around 0.003. Hence, we are comfortable presenting estimates from the model with the European Research Council.

Figure A.5. Bland–Altman plot comparing the parameter estimates for a model with and without the European Research Council (ERC). Both scales are the probability of open peer review.

Model check 3: COVRATIO

Figure A.6. Scatter plot of the COVRATIO against the index (dataset row number)

COVRATIO measures the overall change in the precision (covariance) of the estimated regression coefficients when the ith observation is removed [75]. Values close to 1 indicate little difference in the model’s precision. Values above 1 indicate increased precision and hence narrower confidence intervals after removing the observation.

There was a small but consistent difference in the COVRATIO for articles where the European Research Council was a funder. Removing these articles would improve the precision, but the change is small with a COVRATIO under 1.002. Because of this small change, we are not concerned with including these observations.

Model check 4: Model predictions vs observations

Figure A.7. Observed (yes/no) versus predicted open review probabilities. The smooth line shows a mostly linear increase in the proportion of open reviews for increased predicted probabilities. The smooth was a GAM using a cubic spline (the default option in ggplot); the grey area is a 95% confidence interval. The diagonal dotted line indicates perfect agreement between the predicted probabilities and proportion of observed outcomes. The top panel is a box plot of the predicted probabilities by open review.

The model did not cover the entire range of predictions, with a lowest predicted probability of 0.02 and highest of 0.77. The model struggled to create high probabilities where there was a high likelihood of open review. We examined articles with closed peer review but with a high predicted probability over 0.70 but found no consistent patterns.

Model check 5: Linear assumption

We used a linear model with no link function to model the binary outcome of open review (yes/no). As a sensitivity analysis, we fitted a model using a binomial family with a logistic link. We used the final set of variables from the stability selection.

Figure A.8. Bland–Altman plot comparing the fitted probabilities from the linear and logistic models. Each point is an individual article. The Bland–Altman limits of agreement were –0.004 to 0.005.

The Bland–Altman plot comparing the two models in Figure A.8 shows a strongly non-linear association due to the logit link function. The limits of agreement were relatively narrow, indicating little overall difference between linear and logistic. However, considering that a better model is one that covers the widest range of probabilities [76], the linear model is better at small probabilities, whilst the logistic model is slightly better at higher probabilities. The linear probabilities range from 0.02 to 0.77, whilst the logistic probabilities range from 0.11 to 0.75.

Overall, we prefer the linear model.

A.9 Time-dependent hazard for retraction

Figure A.9. A time-dependent effect of open review on retraction. The left panel shows the Schoenfeld residuals from a time-independent Cox model. Based on the residuals, we assumed a linear change over time and the estimated time-dependent hazard ratio is plotted in the right panel.

A.10 Stability selection for variables associated with retraction

To investigate potential independent predictors of retraction, we used a LASSO approach with the candidate variables of: open peer review, peer review time, number of authors, publication date, country and subject. The variables that were most commonly selected using a stability selection combined with a LASSO are in Figure A.10. Open peer review was only selected in 13% of the bootstrap samples.

Figure A.10. Probability estimates from the stability selection used to select the variables associated with time to retraction. To improve readability, only variables with a greater than 0.65 selection probability are shown (there were 978 total candidate variables). The threshold for inclusion in the final model was a probability of 0.75 and included variables are shown in green. The probability was estimated over 102 bootstrap resamples.

A.11 Causal diagram for the association between open peer review and retraction

Figure A.11. Simplified causal diagram showing a potential association between open review and retraction. Researchers who choose open peer review are likely engaging in other good research practices. Open peer review and retractions also vary by country and subject.

A.12 List of all subjects

A

Acids, Acoustics, Actinobacteria, Addiction, Adipose tissue, Adolescents, Adults, Adverse events, Africa, Age groups, Aging, Agricultural methods, Agricultural soil science, Agricultural workers, Agriculture, Agrochemicals, Agronomy, Air pollution, Alcohol consumption, Alcohols, Algebra, Algorithms, Allied health care professionals, Alternative energy, Alzheimer’s disease, Amino acids, Amniotes, Analgesics, Analysis of variance, Anatomy, Anemia, Animal behavior, Animal cells, Animal genomics, Animal management, Animal models, Animal products, Animal studies, Animals, Antenatal care, Anthropology, Antibiotic resistance, Antibiotics, Antibodies, Antimicrobial resistance, Antimicrobials, Antioxidants, Antiretroviral therapy, Antiviral therapy, Anxiety, Apoptosis, Applied mathematics, Aquatic environments, Archaeology, Arms, Arteries, Arthritis, Arthropoda, Artificial intelligence, Artificial neural networks, Asia, Asian people, Atmospheric science, Attention, Autoimmune diseases,

B

Bacteria, Bacterial diseases, Bacterial pathogens, Bacteriology, Bangladesh, Basic cancer research, Behavior, Behavioral and social aspects of health, Beverages, Bioassays and physiological analysis, Biochemical simulations, Biochemistry, Biodiversity, Bioenergetics, Bioengineering, Bioinformatics, Biological cultures, Biological databases, Biological locomotion, Biological tissue, Biology and life sciences, Biomarkers, Biomechanics, Biophysics, Biopsy, Biosynthesis, Biotechnology, Birds, Birth, Birth weight, Blood, Blood cells, Blood plasma, Blood pressure, Blood vessels, Bodies of water, Body fluids, Body limbs, Body mass index, Body weight, Bovines, Brain, Brain electrophysiology, Brain mapping, Brassica, Brazil, Breast tumors,

C

C-reactive proteins, Canada, Cancer detection and diagnosis, Cancer risk factors, Cancer treatment, Cancers and neoplasms, Carbohydrates, Carbon dioxide, Carcinoma, Cardiology, Cardiovascular anatomy, Cardiovascular disease risk, Cardiovascular diseases, Cardiovascular medicine, Cardiovascular procedures, Careers, Caregivers, Cell biology, Cell cultures, Cell cycle and cell division, Cell death, Cell differentiation, Cell lines, Cell membranes, Cell motility, Cell phones, Cell physiology, Cell processes, Cell signaling, Cell staining, Cellular structures and organelles, Cellular types, Census, Central nervous system, Centrality, Cerebrovascular diseases, Chemical characterization, Chemical compounds, Chemical elements, Chemical properties, Chemical reactions, Chemistry, Child health, Children, China, Cholesterol, Chromatin, Chromatin modification, Chromatographic techniques, Chromosome biology, Chronic kidney disease, Chronic obstructive pulmonary disease, Cities, Civil engineering, Classical mechanics, Climate change, Climatology, Clinical genetics, Clinical immunology, Clinical medicine, Clinical neurophysiology, Clinical oncology, Clinical psychology, Clinical research design, Clinical trials, Cloning, Cognition, Cognitive impairment, Cognitive neurology, Cognitive neuroscience, Cognitive psychology, Cognitive science, Cohort studies, Collagens, Collective human behavior, Colorectal cancer, Commerce, Communication equipment, Communications, Community ecology, Complementary and alternative medicine, Computational biology, Computational chemistry, Computational neuroscience, Computational techniques, Computed axial tomography, Computer and information sciences, Computer architecture, Computer networks, Computer software, Condensed matter physics, Connective tissue, Connective tissue cells, Conservation science, Convolution, Coronary heart disease, Coronaviruses, COVID 19, Criminology, Critical care and emergency medicine, Crop science, Crops, Crystal structure, Cultural anthropology, Culture, Cytophotometry, Cytoskeletal proteins,

D

Damage mechanics, Data management, Data visualization, Database and informatics methods, Database searching, Death rates, Decision making, Deep learning, Deformation, Dementia, Dengue fever, Depression, Dermatology, Development economics, Developmental biology, Developmental neuroscience, Diabetes diagnosis and management, Diabetes mellitus, Diagnostic medicine, Diet, Digestive system, Digestive system procedures, Disease surveillance, Disease vectors, DNA, DNA-binding proteins, DNA viruses, Dogs, Drosophila, Drug interactions, Drug research and development, Drug therapy, Drugs,

E

Earth sciences, Ecological metrics, Ecology, Ecology and environmental sciences, Economic analysis, Economic development, Economic geography, Economic growth, Economics, Ecosystems, Education, Educational attainment, Educational status, Eggs, Elderly, Electricity, Electroencephalography, Electromagnetic radiation, Electron microscopy, Electronic medical records, Electronics, Electronics engineering, Electrophysiological techniques, Electrophysiology, Embryology, Embryos, Emotions, Employment, Endocrine physiology, Endocrinology, Energy and power, Engineering and technology, Enterobacteriaceae, Entomology, Entropy, Environmental chemistry, Environmental health, Enzyme-linked immunoassays, Enzymes, Enzymology, Epidemiology, Epigenetics, Epithelial cells, Epithelium, Equipment, Escherichia, Ethiopia, Ethnicities, Eukaryota, Europe, European people, European Union, Evolutionary biology, Evolutionary processes, Extraction techniques, Eye diseases, Eyes,

F

Face, Factor analysis, Families, Fatigue, Fats, Fatty acids, Fear, Fevers, Fibrosis, Finance, Financial markets, Fish, Flaviviruses, Flowering plants, Fluid mechanics, Fluorescence imaging, Food, Forecasting, Forests, Freshwater environments, Fruits, Fuels, Fungal pathogens, Fungi,

G

Gastroenterology and hepatology, Gastrointestinal tract, Gastrointestinal tumors, Gene expression, Gene expression and vector techniques, Gene identification and analysis, Gene regulation, Genetic engineering, Genetic interference, Genetic loci, Genetic mapping, Genetically modified organisms, Genetics, Genitourinary infections, Genitourinary tract tumors, Genome-wide association studies, Genome analysis, Genomics, Geographic areas, Geographical locations, Geography, Geology, Geometry, Geomorphology, Geriatrics, Germ cells, Ghana, Global health, Glucose, Glycobiology, Governments, Grammar, Grasses, Gut bacteria, Gynecological tumors,

H

Habitats, Habits, Hands, Haplotypes, Head, Health care, Health care facilities, Health care policy, Health care providers, Health economics, Health education and awareness, Health information technology, Health insurance, Health systems strengthening, Heart, Heart failure, Heart rate, Helminth infections, Helminths, Hematologic cancers and related disorders, Hematology, Hemoglobin, Hemorrhage, Hepatitis viruses, Hepatocellular carcinoma, Heredity, Hip, Histology, HIV, HIV diagnosis and management, HIV epidemiology, HIV infections, HIV prevention, Hormones, Hospitals, Human capital, Human families, Human genetics, Human geography, Human learning, Hydrology, Hypertension,

I

Imaging techniques, Immune cells, Immune physiology, Immune response, Immune system, Immune system proteins, Immunoassays, Immunologic techniques, Immunology, India, Industrial engineering, Infants, Infectious disease control, Infectious disease epidemiology, Infectious diseases, Inflammation, Inflammatory diseases, Influenza, Infographics, Information technology, Information theory, Innate immune system, Inpatients, Insect vectors, Insects, Insulin, Intensive care units, Internet, Invertebrates, Ischemic stroke, Islands,

J

Japan, Jaw, Jobs,

K

Kenya, Kidneys, Knees,

L

Labor and delivery, Labor economics, Language, Larvae, Law and legal sciences, Learning, Learning and memory, Leaves, Legs, Lesions, Life cycles, Light, Linear algebra, Linear regression analysis, Linguistics, Lipid hormones, Lipids, Liver diseases, Livestock, Low and middle income countries, Lung and intrathoracic tumors,

M

Machine learning, Machine learning algorithms, Macromolecular structure analysis, Macromolecules, Macrophages, Magnetic resonance imaging, Maize, Malaria, Malignant tumors, Malnutrition, Mammals, Management engineering, Marine and aquatic sciences, Marine biology, Marine fish, Marketing, Material properties, Materials, Materials physics, Materials science, Maternal health, Mathematical and statistical techniques, Mathematical functions, Mathematical models, Mathematics, Measurement, Meat, Mechanical engineering, Mechanical properties, Mechanical stress, Medical conditions, Medical devices and equipment, Medical humanities, Medical implants, Medical microbiology, Medical personnel, Medical risk factors, Medicine and health sciences, Membrane potential, Membrane proteins, Memory, Mental health and psychiatry, Mental health therapies, Messenger RNA, Metaanalysis, Metabolic disorders, Metabolic pathways, Metabolism, Metabolites, Metastasis, Meteorology, Microbial control, Microbial genomics, Microbiology, Microbiome, MicroRNAs, Microscopy, Milk, Mitochondria, Mixtures, Model organisms, Molecular biology, Molecular biology assays and analysis techniques, Molecular biology techniques, Molecular development, Molecular probe techniques, Monosaccharides, Monte Carlo method, Morphogenesis, Mosquitoes, Mothers, Motion, Motivation, Mouse models, Movement disorders, Multivariate analysis, Muscle physiology, Muscles, Musculoskeletal mechanics, Musculoskeletal system, Mutation, Mycobacterium tuberculosis, Mycology, Myocardial infarction,

N

Nanotechnology, Natural resources, Neglected tropical diseases, Nematoda, Neonates, Neonatology, Nephrology, Nervous system, Network analysis, Neural networks, Neurochemistry, Neurodegenerative diseases, Neuroimaging, Neurology, Neurons, Neurophysiology, Neuropsychiatric disorders, Neuroscience, Neuroses, Neurotransmitters, Neutrophils, Nigeria, Non-coding RNA, Normal distribution, North America, Nucleic acids, Nucleotides, Nurses, Nutrition,

O

Obesity, Obstetrics and gynecology, Oceania, Ocular anatomy, Ocular system, Oncology, Ophthalmology, Opioids, Optical equipment, Optimization, Oral medicine, Organic compounds, Organism development, Organisms, Osteichthyes, Otorhinolaryngology, Outpatients, Oxidative stress, Oxygen,

P

Pain, Pain management, Paleontology, Pandemics, Parasite groups, Parasitic diseases, Parasitology, Parenting behavior, Pathogens, Pathology and laboratory medicine, Patients, Pediatrics, Pelvis, People and places, Peptide hormones, Perception, Personality, Personality traits, Petrology, Pharmaceutics, Pharmacology, Phenotypes, Phosphorylation, Phylogenetic analysis, Phylogenetics, Physical activity, Physical chemistry, Physical fitness, Physical sciences, Physicians, Physics, Physiological parameters, Physiological processes, Physiology, Pigments, Plant anatomy, Plant and algal models, Plant biochemistry, Plant cell biology, Plant ecology, Plant pathology, Plant physiology, Plant science, Plants, Plasmodium, Pneumonia, Political science, Pollution, Polymer chemistry, Polymerase chain reaction, Population biology, Population density, Population genetics, Population groupings, Population metrics, Post-translational modification, Poultry, Precipitation techniques, Pregnancy, Pregnancy complications, Preprocessing, Preventive medicine, Primary care, Primates, Principal component analysis, Probability distribution, Probability theory, Professions, Prognosis, Prokaryotic models, Prostate cancer, Proteases, Protein domains, Protein expression, Protein interactions, Protein structure, Proteins, Proteomics, Protozoan infections, Protozoans, Psychological attitudes, Psychological stress, Psychology, Psychometrics, Public and occupational health, Public policy, Pulmonology,

Q

Qualitative studies, Quality of life, Questionnaires,

R

Radiation therapy, Radiology and imaging, Random variables, Randomized controlled trials, Reaction time, Recreation, Regression analysis, Regulatory proteins, Religion, Renal cancer, Renal diseases, Renal system, Reproductive physiology, Reproductive system, Research and analysis methods, Research assessment, Research design, Research facilities, Research integrity, Respiration, Respiratory disorders, Respiratory infections, Reverse transcriptase-polymerase chain reaction, Rheumatology, Ribosomal RNA, Ribosomes, Rice, Rivers, RNA, RNA sequencing, RNA viruses, Roads, Rodents, Ruminants, Rural areas,

S

Safety, Schools, Science and technology workforce, Science policy, Seasons, Sedimentary geology, Seeds, Semantics, Sensory perception, Sensory physiology, Sensory systems, Sepsis, Sequence alignment, Sequencing techniques, Sexual and gender issues, Sexually transmitted diseases, Signal processing, Signal transduction, Signaling cascades, Signs and symptoms, Simulation and modeling, Single nucleotide polymorphisms, Skeletal joints, Skeleton, Sleep, Small interfering RNA, Smoking habits, Social communication, Social media, Social networks, Social psychology, Social sciences, Social systems, Socioeconomic aspects of health, Sociology, Software engineering, Soil science, Solid state physics, South America, Species diversity, Species interactions, Specimen preparation and treatment, Spectrophotometry, Spectrum analysis techniques, Sports, Sports and exercise medicine, Sports science, Staining, Staphylococcus, Staphylococcus aureus, Statistical data, Statistical distributions, Statistical methods, Statistics, Stem cells, Structural engineering, Substance-related disorders, Suicide, Support vector machines, Surface water, Surgical and invasive medical procedures, Surgical oncology, Survey research, Surveys, Sustainability science, Swine, Systematic reviews, Systems science,

T

T cells, Taxonomy, Teachers, Teeth, Terrestrial environments, Thermodynamics, Tomography, Topography, Toxic agents, Toxicity, Toxicology, Transcriptome analysis, Transfection, Transplantation, Transportation, Transportation infrastructure, Trauma medicine, Traumatic injury, Traumatic injury risk factors, Trees, Trophic interactions, Tropical diseases, Tuberculosis, Tuberculosis diagnosis and management, Type 2 diabetes,

U

Uganda, Ultrasound imaging, United States, Urban areas, Urban geography, Urine, Urology,

V

Vaccination and immunization, Vaccine development, Vaccines, Variant genotypes, Vascular medicine, Velocity, Vertebrates, Vesicles, Veterinary diseases, Veterinary science, Violent crime, Viral diseases, Viral load, Viral replication, Viral transmission and infection, Viral vaccines, Virology, Virulence factors, Virus testing, Viruses, Vision, Vitamins,

W

Walking, Water pollution, Water resources, White blood cells, Women’s health,

Y

Yeast, Yeast and fungal models, Young adults,

Z

Zoology, Zoonoses

References

  1. Munafò, M. R. et al. “A manifesto for reproducible science”. Nature Human Behaviour 1.1 (Jan. 2017). DOI: 10.1038/s41562-016-0021.
  2. Besançon, L. et al. “Open science saves lives: lessons from the COVID-19 pandemic”. BMC Medical Research Methodology 21.1 (June 2021). DOI: 10.1186/s12874-021-01304-y.
  3. Soergel, D. A. W. “Rampant software errors may undermine scientific results”. F1000Research 3 (July 2015), 303. DOI: 10.12688/f1000research.5930.2.
  4. Ince, D. “The Duke University scandal – what can be done?” Significance 8.3 (Aug. 2011), 113–115. DOI: 10.1111/j.1740-9713.2011.00505.x.
  5. Lumbard, H. and Routledge, D. “Open science and transparency are our strongest tools in the fight against fraudulent publishing activities”. PLOS Medicine 22.9 (Sept. 2025), 1–3. DOI: 10.1371/journal.pmed.1004774.
  6. Thibault, R. T. et al. “Open Science 2.0: Towards a truly collaborative research ecosystem”. PLOS Biology 21.10 (Oct. 2023), 1–22. DOI: 10.1371/journal.pbio.3002362.
  7. Hamilton, D. G. et al. “Prevalence and predictors of data and code sharing in the medical and health sciences: systematic review with meta-analysis of individual participant data”. BMJ 382 (2023). DOI: 10.1136/bmj-2023-075767.
  8. Hosseini, M. et al. “Messing with Merton: The intersection between open science practices and Mertonian values”. Accountability in Research 31.5 (Nov. 2022), 428–455. DOI: 10.1080/08989621.2022.2141625.
  9. Ferguson, J. et al. “Survey of open science practices and attitudes in the social sciences”. Nature Communications 14.1 (Sept. 2023). DOI: 10.1038/s41467-023-41111-1.
  10. Tenopir, C. et al. “Data sharing, management, use, and reuse: Practices and perceptions of scientists worldwide”. PLOS ONE 15.3 (Mar. 2020), 1–26. DOI: 10.1371/journal.pone.0229003.
  11. Armeni, K. et al. “Towards wide-scale adoption of open science practices: The role of open science communities”. Science and Public Policy 48.5 (July 2021), 605–611. DOI: 10.1093/scipol/scab039.
  12. Gomes, D. G. E. et al. “Why don’t we share data and code? Perceived barriers and benefits to public archiving practices”. Proceedings of the Royal Society B: Biological Sciences 289.1987 (Nov. 2022). DOI: 10.1098/rspb.2022.1113.
  13. Smaldino, P. E. and McElreath, R. “The natural selection of bad science”. Royal Society Open Science 3.9 (Sept. 2016), 160384. DOI: 10.1098/rsos.160384.
  14. Ross-Hellauer, T., Deppe, A., and Schmidt, B. “Survey on open peer review: Attitudes and experience amongst editors, authors and reviewers”. PLOS ONE 12.12 (Dec. 2017), 1–28. DOI: 10.1371/journal.pone.0189311.
  15. Godlee, F. “Making Reviewers Visible: Openness, Accountability, and Credit”. JAMA 287.21 (June 2002), 2762. DOI: 10.1001/jama.287.21.2762.
  16. Schmidt, B. et al. “Ten considerations for open peer review”. F1000Research 7 (June 2018), 969. DOI: 10.12688/f1000research.15334.1.
  17. Editorial. “Transparent peer review to be extended to all of Nature’s research papers”. Nature 642.8068 (June 2025), 542. DOI: 10.1038/d41586-025-01880-9.
  18. Wolfram, D. et al. “Open peer review: promoting transparency in open science”. Scientometrics 125.2 (May 2020), 1033–1051. DOI: 10.1007/s11192-020-03488-4.
  19. Álvarez-García, E. et al. “Published peer review reports have higher informative content than unpublished reports”. Journal of Informetrics 20.1 (2026), 101760. DOI: https://doi.org/10.1016/j.joi.2025.101760.
  20. PLOS. Open Peer Review. [accessed 12-September-2025]. 2020. URL: https://plos.org/resource/open-peer-review/.
  21. Richardson, R. A. K. et al. “The entities enabling scientific fraud at scale are large, resilient, and growing rapidly”. Proceedings of the National Academy of Sciences 122.32 (Aug. 2025). DOI: 10.1073/pnas.2420092122.
  22. Oransky, I. et al. “Retractions in medicine: the tip of the iceberg”. European Heart Journal 42.41 (June 2021), 4205–4206. DOI: 10.1093/eurheartj/ehab398.
  23. PLOS. Text & Data Mining – PLOS. [accessed 12-September-2025]. 2025. URL: https://plos.org/text-and-data-mining/.
  24. PLOS Subject Area Thesaurus. [accessed 14-September-2025]. 2021. URL: https://github.com/PLOS/plos-thesaurus/tree/master.
  25. Crossref. Open Funder Registry (OFR) v 1.62. [accessed 4-December-2025]. 2025. URL: https://www.crossref.org/services/funder-registry/.
  26. Akers, K. G. et al. “ORCID Author Identifiers: A Primer for Librarians”. Medical Reference Services Quarterly 35.2 (Apr. 2016), 135–144. DOI: 10.1080/02763869.2016.1152139.
  27. PLOS ONE. Submission Guidelines. [accessed 25-February-2026]. 2026. URL: https://journals.plos.org/plosone/s/submission-guidelines.
  28. Marušic, A., Bošnjak, L., and Jerončić, A. “A Systematic Review of Research on the Meaning, Ethics and Practices of Authorship across Scholarly Disciplines”. PLoS ONE 6.9 (Sept. 2011), e23477. DOI: 10.1371/journal.pone.0023477.
  29. Priem, J., Piwowar, H., and Orr, R. OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and concepts. 2022. arXiv: 2205.01833 [cs.DL].
  30. Royston, P., Ambler, G., and Sauerbrei, W. “The use of fractional polynomials to model continuous risk variables in epidemiology”. International Journal of Epidemiology 28.5 (Oct. 1999), 964–974. DOI: 10.1093/ije/28.5.964.
  31. Meinshausen, N. and Bühlmann, P. “Stability Selection”. Journal of the Royal Statistical Society Series B: Statistical Methodology 72.4 (Aug. 2010), 417–473. DOI: 10.1111/j.1467-9868.2010.00740.x.
  32. Hofner, B., Boccuto, L., and Göker, M. “Controlling false discoveries in high-dimensional situations: boosting with stability selection”. BMC Bioinformatics 16.1 (May 2015). DOI: 10.1186/s12859-015-0575-3.
  33. Dobson, A. J. and Barnett, A. G. An Introduction to Generalized Linear Models. 4th. Texts in Statistical Science. Boca Raton, FL: Chapman & Hall/CRC, 2018.
  34. Lumley, T. et al. “The Importance of the Normality Assumption in Large Public Health Data Sets”. Annual Review of Public Health 23.1 (May 2002), 151–169. DOI: 10.1146/annurev.publhealth.23.100901.140546.
  35. Cabanac, G. “Chain retraction: how to stop bad science propagating through the literature”. Nature 632.8027 (Aug. 2024), 977–979. DOI: 10.1038/d41586-024-02747-1.
  36. Cabanac, G., Labbé, C., and Magazinov, A. The ’Problematic Paper Screener’ automatically selects suspect publications for post-publication (re)assessment. 2022. DOI: 10.48550/ARXIV.2210.04895.
  37. Tahamtan, I. and Bornmann, L. “What do citation counts measure? An updated review of studies on citations in scientific documents published between 2006 and 2018”. Scientometrics 121.3 (Sept. 2019), 1635–1684. DOI: 10.1007/s11192-019-03243-4.
  38. Declaration on Research Assessment. Signers — DORA. 2025. URL: https://sfdora.org/signers/.
  39. Winter, D. J. “rentrez: an R package for the NCBI eUtils API”. The R Journal 9 (2 2017), 520–526.
  40. Eckmann, P. and Bandrowski, A. “PreprintMatch: A tool for preprint to publication detection shows global inequities in scientific publication”. PLOS ONE 18.3 (Mar. 2023), e0281659. DOI: 10.1371/journal.pone.0281659.
  41. Hutto, C. and Gilbert, E. “VADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text”. Proceedings of the International AAAI Conference on Web and Social Media 8.1 (May 2014), 216–225. DOI: 10.1609/icwsm.v8i1.14550.
  42. R Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing. Vienna, Austria, 2024. URL: https://www.R-project.org/.
  43. Barnett, A. Code and data for the analysis of who opts for open peer review. 2026. URL: https://github.com/agbarnett/open_peer_review.
  44. Hardwicke, T. E. et al. “Prevalence of Transparent Research Practices in Psychology: A Cross-Sectional Study of Empirical Articles Published in 2022”. Advances in Methods and Practices in Psychological Science 7.4 (2024), 25152459241283477. DOI: 10.1177/25152459241283477.
  45. Grundy, Q. et al. “Prevalence of Disclosed Conflicts of Interest in Biomedical Research and Associations With Journal Impact Factors and Altmetric Scores”. JAMA 319.4 (Jan. 2018), 408. DOI: 10.1001/jama.2017.20738.
  46. Wikipedia. .edu — Wikipedia, The Free Encyclopedia. http://en.wikipedia.org/w/index.php?title=.edu&oldid=1320638017. [accessed 11-November-2025]. 2025.
  47. Mahony, S. “Toward openness and transparency to better facilitate knowledge creation”. Journal of the Association for Information Science and Technology 73.10 (Apr. 2022), 1474–1488. DOI: 10.1002/asi.24652.
  48. Efron, N. “Ensure your paper gets published: ten tips for responding effectively to reviewer comments”. Clinical and Experimental Optometry (Nov. 2025), 1–5. DOI: 10.1080/08164622.2025.2589401.
  49. Frey, B. S., Eichenberger, R., and Frey, R. L. “Editorial Ruminations: Publishing Kyklos”. Kyklos 62.2 (Apr. 2009), 151–160. DOI: 10.1111/j.1467-6435.2009.00428.x.
  50. Smith, R. “Opening up BMJ peer review”. BMJ 318.7175 (Jan. 1999), 4–5. DOI: 10.1136/bmj.318.7175.4.
  51. Maibach, E. et al. “The legacy of climategate: undermining or revitalizing climate science and policy?” WIREs Climate Change 3.3 (2012), 289–295. DOI: https://doi.org/10.1002/wcc.168.
  52. McSweeney, R. and Tandon, A. Factcheck: Scientists pour cold water on claims of ‘journal bias’ by author of wildfires study. [accessed 11-November-2025]. 2023. URL: https://www.carbonbrief.org/factcheck-scientists-pour-cold-water-on-claims-of-journal-bias-by-author-of-wildfires-study/.
  53. Lewandowsky, S. Climate change denial and the abuse of peer review. 2011. URL: https://theconversation.com/climate-change-denial-and-the-abuse-of-peer-review-1552.
  54. Pinfield, S. “A mandate to self archive? The role of open access institutional repositories”. Serials 18 (1 2005). ISSN: 0953-0460.
  55. Registry of Open Access Mandatory Archiving Policies. [accessed 16-January-2026]. University of Southampton, 2026. URL: https://roarmap.eprints.org/.
  56. Kiley, R. et al. “Data Sharing from Clinical Trials — A Research Funder’s Perspective”. New England Journal of Medicine 377.20 (Nov. 2017), 1990–1992. DOI: 10.1056/nejmsb1708278.
  57. Haak, L. Research Funders and ORCID: New members, mandates, and platforms. [accessed 16-January-2026]. 2015. URL: https://info.orcid.org/research-funders-and-orcid-new-members-mandates-and-platforms/.
  58. European Research Council. Open Access Guidelines for research results funded by the ERC. [accessed 6-April-2026]. 2021. URL: https://erc.europa.eu/sites/default/ files/document/file/ERC_Open_Access_Guidelines-revised_Dec_2021.pdf.
  59. Falk Delgado, A., Garretson, G., and Falk Delgado, A. “The language of peer review reports on articles published in the BMJ, 2014–2017: an observational study”. Scientometrics 120.3 (June 2019), 1225–1235. DOI: 10.1007/s11192-019-03160-6.
  60. Ross-Hellauer, T. and Horbach, S. P. J. M. “Additional experiments required: A scoping review of recent evidence on key aspects of Open Peer Review”. Research Evaluation 33 (2024). DOI: 10.1093/reseval/rvae004.
  61. Yin, H. et al. “Tracking claim changes from preprint to publication across 72,644 biomedical studies using large language models” (July 2026). DOI: 10.64898/2026.06.30.735556.
  62. Abalkina, A. “Publication and collaboration anomalies in academic papers originating from a paper mill: Evidence from a Russia-based paper mill”. Learned Publishing 36.4 (Sept. 2023), 689–702. DOI: 10.1002/leap.1574.
  63. Sebo, P. and Sebo, M. “Geographical Disparities in Research Misconduct: Analyzing Retraction Patterns by Country”. Journal of Medical Internet Research 27 (Jan. 2025), e65775. DOI: 10.2196/65775.
  64. Spick, M. et al. “Quantifying new threats to health and biomedical literature integrity from rapidly scaled publications and problematic research”. Journal of Clinical Epidemiology 193 (May 2026), 112203. DOI: 10.1016/j.jclinepi.2026.112203.
  65. Klebel, T. et al. “The academic impact of Open Science: a scoping review”. Royal Society Open Science 12.3 (Mar. 2025). DOI: 10.1098/rsos.241248.
  66. Zong, Q., Xie, Y., and Liang, J. “Does open peer review improve citation count? Evidence from a propensity score matching analysis of PeerJ”. Scientometrics 125.1 (July 2020), 607–623. DOI: 10.1007/s11192-020-03545-y.
  67. Wei, C. et al. “What does open peer review bring to scientific articles? Evidence from PLoS journals”. Scientometrics 128.5 (Mar. 2023), 2763–2776. DOI: 10.1007/s11192-023-04683-9.
  68. Cheng, X. et al. “Open peer review correlates with altmetrics but not with citations: Evidence from Nature Communications and PLoS One”. Journal of Informetrics 18.3 (2024), 101540. DOI: https://doi.org/10.1016/j.joi.2024.101540.
  69. Colavizza, G. et al. “The citation advantage of linking publications to research data”. PLOS ONE 15.4 (Apr. 2020), 1–18. DOI: 10.1371/journal.pone.0230416.
  70. Goldberg, A. et al. “Peer reviews of peer reviews: A randomized controlled trial and other experiments”. PLOS ONE 20.4 (Apr. 2025), 1–18. DOI: 10.1371/journal.pone.0320444.
  71. Chatfield, C. “Model Uncertainty, Data Mining and Statistical Inference”. Journal of the Royal Statistical Society. Series A (Statistics in Society) 158.3 (1995), 419. DOI: 10.2307/2983440.
  72. PLOS. Looking back on four years of Published Peer Review at PLOS. [accessed 4-November-2025]. 2024. URL: https://theplosblog.plos.org/2024/01/four-years-of-published-peer-review-history/.
  73. Moskovkin, V. M. et al. “International movement of open access to scientific knowledge: A quantitative analysis of country involvement”. The Journal of Academic Librarianship 47.1 (Jan. 2021), 102296. DOI: 10.1016/j.acalib.2020.102296.
  74. Burnham, K. and Anderson, D. Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach. Springer New York, 2002. ISBN: 9780387953649.
  75. Belsley, D. A., Kuh, E., and Welsch, R. E. Regression Diagnostics: Identifying Influential Data and Sources of Collinearity. Wiley, June 1980. ISBN: 9780471725152. DOI: 10.1002/0471725153.
  76. Harrell, F. Statistically Efficient Ways to Quantify Added Predictive Value of New Measurements. [accessed 5-April-2026]. 2018. URL: https://www.fharrell.com/post/addvalue/.

Editors

Ludo Waltman
Editor-in-Chief

Ludo Waltman
Handling Editor

Editorial assessment

by Ludo Waltman

DOI: 10.70744/MetaROR.523.2.ea

In this article the authors present a comparison between articles with and without open peer review published in PLOS journals. The authors suggest that openness of peer review could be an indicator of article quality. The revised version of the article has been reviewed by the same three reviewers who also reviewed the original version. Reviewers 1 and 2 are satisfied with the revisions made by the authors. Reviewer 3 still has remaining concerns. These concerns relate to possible selection bias, the validity of open peer review as an indicator of article quality, and the interpretation given by the authors to the timing of retractions.

Competing interests: Ludo Waltman is co-Editor-in-Chief of MetaROR. Corresponding author Adrian Barnett is a MetaROR Editor.

Peer review 1

Reese AK Richardson

DOI: 10.70744/MetaROR.523.2.rv1

The authors have thoroughly and meaningfully engaged with all of the reviewers’ comments, which I believe has significantly improved the manuscript. In particular, I really appreciate the authors’ attention to sensitivity and stability selection analyses throughout the manuscript, which answered some outstanding questions I had during the first round of review. I have no major suggestions for improvements. As more journals adopt open peer review as an option for authors, I hope this is an analysis that can be revisited in the future.

Competing interests: I co-authored a short perspective piece with the second author of this article, published in September 2025: https://doi.org/10.3897/ese.2025.e165043

Peer review 2

Dietmar Wolfram

DOI: 10.70744/MetaROR.523.2.rv2

Thank you for the opportunity to provide feedback on the revised version of the manuscript. The authors have satisfactorily addressed my earlier comments and suggestions.

I see there have been additions to the findings and discussion (Table A.6, Figure A.9, Figure A.10, Figure A.11, additional discussion of open review and quality), which help to clarify and expand the findings and conclusions.

For Table A.7, there were 10 funders listed in the earlier version of this table (Table A.5). Now there are nine listed. Several of the funder names have changed from the previous version, along with the order and the values listed. I assume there were recalculations and tweaking of the table contents, which is okay. Either the title of the table should be changed or a tenth funder should be added.

Competing interests: None.

Peer review 3

Jiang Li

DOI: 10.70744/MetaROR.523.2.rv3

I thank the authors for their responses to my previous comments. However, I am not fully satisfied with their explanations, and I think my main concerns remain unresolved.

First, the authors have provided additional sentiment analysis and a causal diagram to explain the relationship between open peer review and article quality. However, these additions do not adequately address my concern about the possible selection bias. The observed associations may still be explained by unobserved factors related to authors and their research practices.

Second, the authors argue that the term “indicator” does not imply causality. I agree with this argument. However, my concern is not simply about causality, but about the validity of the indicator. An indicator does not need to be causal, but it needs to be valid. The associations between open peer review and citation counts or retractions do not necessarily establish open peer review as a valid indicator of article quality. This fundamental issue remains unresolved.

Third, the authors interpret early retractions as more likely to involve misconduct and later retractions as more likely to result from honest errors. Although this explanation is plausible, it is not directly supported by their empirical evidence. Retraction timing alone cannot reliably distinguish different reasons for retraction.

Competing interests: None.

Leave a comment