1. The article is assigned to one of MetaROR’s Editors-in-Chief (EIC).
2. The EIC recruits an Editor to handle the article. Together, the EIC and Editor assess the article’s suitability for peer review using the following criteria:
   • the article falls within MetaROR’s scope and meets all requirements,
   • the article is of sufficient quality to warrant peer review, and
   • the Editor likely will be able to identify appropriate reviewers who can deliver reviews within six weeks of the request.
3. The Editor selects reviewers based on a balance of factors, including expertise, scientific or scholarly reputation, and MetaROR’s previous experience with the reviewer. We strive for diversity in seniority, gender, and geographical location. We also aim to invite only reviewers without competing interests, including any role, relationship, or commitment that presents an actual or perceived threat to the integrity or independence of the review (e.g., current collaborator; co-author during previous five years unless the co-authoring group is unusually large; colleague working in the same department during the previous five years; close colleague including previous advisor, mentor, or student; one who might expect a direct financial interest or other professional benefit from the review; and any other relationship that might compromise the reviewer’s actual or perceived objectivity. Reviewers are required to disclose all actual and perceived competing interests.
4. Authors may suggest reviewers with relevant expertise. Authors should disclose any known competing interests when offering suggestions. Authors may also name anyone who should be avoided during the reviewer selection process. In this case, authors should briefly explain the lack of suitability.
5. Reviewers are strongly encouraged to submit non-anonymous reviews (by including identifying information in the review report), but they may choose to remain anonymous.
6. During the review process, reviewers are invited to discuss their reports with each other and the Editor. Informed by this discussion, the Editor, in consultation with the EIC, prepares an editorial assessment. The editorial assessment summarises the strengths and weaknesses of an article. Authors are given an opportunity to correct errors in the reviews and editorial assessment prior to posting. Unlike editors of traditional journals, MetaROR editors do not make accept/reject decisions. Review reports and editorial assessments are published on the MetaROR platform next to the full text of the article, along with a link to the preprint server.
7. Responses to the reviewers can be submitted via email to the editor. Responses are published on the MetaROR platform next to the review reports. If the authors upload a revised version of the article to a preprint server, a link to the revised version should be included in the responses. MetaROR will also post the link to the revision following the author responses. If the article is submitted for a subsequent round of review, we will post a link to the revised version when it is posted along with the assessment and reviews. 
8. Authors may request a review of the revised article by following the standard submission process. The same review and assessment processes used for original submissions will be followed. In some cases–e.g., when the revisions are such that another round of review is unnecesary–the editor may decide to publish an assessment of the revision without sending the revision to the reviewers. 
9. When MetaROR’s review process is complete for all submitted versions, authors may submit their article to a traditional journal. To reduce the reviewing burden on the metaresearch community, authors are strongly encouraged to include MetaROR reviews and editorial assessments when submitting to journals.

MetaROR requires authors to adopt open science practices in performing and reporting their research. We also value the broad epistemological and methodological diversity in metaresearch, and we acknowledge that certain open science practices are more relevant or more applicable in some areas of metaresearch than in others. In particular, sharing of data, software, and materials tends to be easier for quantitative research than for qualitative research. For qualitative research, sharing of raw research materials is encouraged but not mandatory. We recommend considering whether the materials can be redacted to allow sharing.

For quantitative research, sharing of the raw data used in the research is mandatory unless data sharing is not possible for legal or ethical reasons. The data must be made available in an appropriate repository, such as Open Science Framework, Zenodo, Dryad, or an institutional repository. In the latter case, the data must have a DOI or other persistent identifier as well as an open license. The data must be maximally reusable (e.g., the meta-data and accompanying text must describe the data carefully and accurately). Regardless of whether authors use original data or reuse data made available by others, they must provide sufficient information to allow an informed researcher to reproduce all reported results.

Authors must include in their article a data availability statement that reports whether the data and other materials used in the research are publicly available. If the data are publicly available, the data availability statement must indicate where the data can be found. If the data are not publicly available, the data availability statement must explain why the authors are unable to make the data publicly available (e.g., for legal or ethical reasons). In addition, the data availability statement must indicate exactly how others can access the data and use them to reproduce reported findings. MetaROR may refuse to review a preprint if it deems as spurious the reasons offered for not sharing data or if the article’s claims are such that readers will suspect bias given the data’s inaccessibility. If the research does not utilise any data, the authors must state this explicitly in the data availability statement.

Sharing the data only upon request is not acceptable.

Source code and scripts used in the research must be publicly available in an appropriate repository. Software packages used in the research, including version numbers, must be reported in the text of the article, in an appendix, in supplementary materials, or in the source code and scripts documentation. Software packages should also be cited in detail in the reference section (see Katz et al. 2021). Authors should be aware that Git URLs are not permanent. Authors should issue a DOI for a GitHub repository. Source code and scripts must be carefully and accurately annotated to maximise opportunities for reuse by other researchers. We encourage including a codebook.

Data citation

Authors must formally cite all publicly available datasets used in their article. For each dataset, include a citation in the text of the article and an entry in the reference list. Whenever possible, the reference must include a DOI or other persistent identifier. The use of DOIs or other persistent identifiers is preferable over URLs. At a minimum, include the author’s name, publication year, dataset name, version if applicable, and DOI.

Confirmatory vs. exploratory results and preregistration

In the case of quantitative research designed to test hypotheses, we strongly encourage explicitly categorising all results as confirmatory or exploratory. Confirmatory results relate to hypotheses that were generated prior to observing the data. Results arising from ad hoc data exploration should be categorised as exploratory (Schwab and Held, 2020).

We encourage authors to consider whether preregistration is appropriate (including hypotheses, data collection methods, and analysis methods). Research plans can be registered on any registration service (e.g., the Research on Research Registry or OSF). Authors that preregister should indicate in the article which of the study components were preregistered and which were not, and describe in the article all ways in which the article deviates from the preregistration and an explanation for each deviation. If the analyses do not deviate from the preregistration, this should be stated in the article.

If authors determine that they are not able to preregister their hypotheses and methods prior to observing the data, we encourage them to engage in other bias control procedures, such as perturbing the data before analysis (MacCoun and Perlmutter, 2015) or employing a hold-out dataset (Baldwin et. al, 2022).

Articles submitted to MetaROR must report the following information in the article:

• Author information. Authors must report their ORCID IDs and their affiliations. The corresponding author needs an ORCID ID to submit the article to MetaROR.
• Author contributions. Authors must report the contributions made by each author. The use of the CRediT taxonomy for reporting author contributions is encouraged. This webapp may be helpful.
• Competing interests. Authors must report all competing interests, including not only financial interests, but any role, relationship, or commitment of an author that presents an actual or perceived threat to the integrity or independence of the research presented in the article. If no competing interests exist, authors should explicitly state this.
• Funding information. Authors must report all funding in support of the research presented in the article. Grant reference numbers should be included. If no funding sources exist, explicitly state this in the article.
• Data availability. Authors must include a data availability statement. See the above section on data availability statements for details.
• Approval statement. If approval was required to perform the research (e.g., approval of an Institutional Review Board or a similar body because human participants were involved), authors must report the approval.

All authors of articles submitted to MetaROR agree to abide by MetaROR’s plagiarism policy. As defined by the Committee on Publication Ethics (COPE), plagiarism occurs “when somebody presents the works of others (data, words or theories) as if they were his/her own and without proper acknowledgment,” and self-plagiarism (also known as text recycling) is the “copying from author’s own work without attribution.” We will use plagiarism detection software to screen submitted articles. If potential plagiarism is detected during the review process, we will raise this with the authors, and, in some cases, may need to suspend or terminate the review process. If potential plagiarism is detected after the review process, we will investigate and may take action to ensure the integrity of the scientific record. We may flag the article with an expression of concern in cases where authors do not provide a satisfactory explanation. We may also contact the authors’ institutions to report concerns.

Use of Large Language Models (LLMs) and generative Artificial Intelligence (AI) tools in writing your submission

MetaROR recognises the value of LLMs (e.g., ChatGPT) and generative AI as aids in research and writing. All language models, however, have limitations and are unable to replicate human creative and critical thinking. To ensure content accuracy and appropriateness, authors must play an active role in overseeing and correcting LLM and generative AI outputs. MetaROR encourages authors who use these tools to be fully apprised of their limitations.

The Editor will check the content and wording of review reports before sending the reports to the authors. If authors believe that any report content is erroneous or inappropriate, they can formally request that the report be amended or withdrawn altogether before it is made public. The authors should address such a request to the Editor and should provide a detailed description of the reasons motivating the challenge.

The Editor will consider challenges only in cases involving clearly erroneous content or inappropriate wording. Scientific disagreements and controversies about the interpretation of the research or about its novelty or significance are not sufficient grounds for challenging a review report. Authors can instead address such issues in their public response to reviewers.

A decision on a challenge will be made by the Editor and EIC, possibly in consultation with the reviewers.