Evaluated works

Research Articles

  • Article

    Minhyuk Park, Haotian Yi, Tandy Warnow, George Chacko

    The global science literature, represented as a network with articles as nodes and citations as edges, is a rich artifact for scientometric studies. The structure of this network depends on its count of nodes and distribution of edges. We are interested in how the network has evolved from its origin to its present structure. Since extant theories of citation do not offer much in the way of quantitative explanation, we use a modeling approach that generates synthetic networks. Specifically, we have developed an idealized agent-based model of citations (SASCA-ReS) that can generate synthetic networks of size over 200 million nodes, which is comparable with the size of today’s science literature. This model allows us to reason in an artificial world and to identify patterns of citation that may explain real-world scenarios. We report results from simulations under this model with different parameter settings and at various scales to explore counterfactual and hypothetical scenarios.

    June 16, 2026

  • Article

    Juan Pablo Bascur, Rodrigo Costas, Suzan Verberne

    Traditional science maps visualize topics by clustering documents within a network, but they are inherently biased toward clustering certain topics over others. If these topics could be chosen, then the science maps could be tailored for different needs. In this paper, we explore the extent to which the topic bias of a science map can be changed by choosing different data sources to build the document network. We analyze this by evaluating the clustering effectiveness of several topic categories over two sources that are traditionally used for the creation of science maps (citations and text similarity) and six non-traditional data sources, which we found favor different kinds of topics: Health issues for Facebook users, biotechnology topics for patent families, government and social issues for policy documents, food topics for Twitter conversations, nursing topics for Twitter users, and geographical entities for document authors (the favoring in this latter source was particularly strong). Our results show that diverse data sources can be used to control topic bias, which opens up the possibility of creating science maps tailored for different needs.

    June 11, 2026

  • Article

    Hugh Shanahan, Louise Bezuidenhout

    Preprint services now play a key role in disseminating research across a wide range of domains. In this paper we examine where a set of 64 preprint services, collated by ASAPbio, are being physically hosted and the type of Internet Service Provider (ISP) they are using. In addition to this access to these services from 106 territories was simulated using Virtual Private Networks (VPNs). We find that the majority of services (47/64) are physically based in the USA, despite instances where the Top Level Domain (TLD) indicates a different country. In addition to this 56/64 of the services are being hosted by commercial ISPs with 43/64 being hosted by Cloudflare, Google LLC or Amazon. Poorer countries are more likely to encounter an error when attempting a URL corresponding to the landing page of one of the preprint services collated by ASAPbio than more wealthy countries. Sites that are physically located in Low or Middle Income Countries have similar accessibilities and may be better. We draw some overall conclusions about the potential frailty of these services with respect to the technical decisions made here.

    June 5, 2026

  • Article

    Wolfgang Kaltenbrunner, Andrea Chiarelli, Andrea Reyes Elizondo, Stephen Pinfield, Ludo Waltman, André Brasil

    We analyse focus group discussions and free-text survey responses from a multi-stakeholder consultation conducted after the October 2023 publication of the proposal Towards Responsible Publishing by cOAlition S. The proposal calls for a systemic reform of scholarly communication by reducing barriers to knowledge dissemination, promoting early sharing of outputs through preprints, and shifting peer review to an open, post-publication model. We focus on how different stakeholder groups –such as researchers, infrastructure providers, academic institutions, and publishers –perceive obstacles to the large-scale, coordinated reform envisioned in the proposal. We interpret these accounts as articulations of collective action problems, shaped by entrenchment of many actors in existing academic reward systems and established commercial revenue models that make transitions toward a more economically sustainable scholarly communication system difficult, even where many actors see the principal need for change. This approach highlights the extent to which stakeholder perspectives align or conflict. It also underscores the performative nature of discourse about collective action problems in scholarly communication: by articulating challenges to reform, participants simultaneously construct, reinforce, or contest their own roles within the system, which directly influences their collective capacity to act.

    June 3, 2026

  • Article

    Savannah C. Lewis, Alexa M. Tullett

    Scientific integrity depends on ethical and transparent research practices, yet surveys reveal that many researchers engage in questionable research practices (QRPs), ranging from minor issues (e.g., unclear preregistration) to major misconduct (e.g., data fabrication). This study investigates how people perceive researchers who commit QRPs (investigators) compared to those who report them (inspectors). Participants (N ≈ 566) will read three hypothetical vignettes describing an investigator engaging in a QRP and an inspector who reports it. Participants will then evaluate both researchers on trustworthiness and likeability. They will also rate the perceived trueness of the original finding, and compare both researchers on a range of positive attributes. Thus, the overall design is a 2 (role: investigator vs. inspector) x 3 (QRP severity: minor vs. moderate vs. major) fully within-subjects design. Multilevel models will test whether perceptions vary by role and QRP severity. This research will deepen our understanding of how accountability in science is socially evaluated, and how the severity of misconduct shapes views of both those who commit and those who call out QRPs.

    May 28, 2026

  • Article

    Alex Hulkes

    All other things being equal, two-stage research funding processes that involve the initial submission of a relatively short proposal containing only minimal information (typically referred to as an outline proposal) followed by a preparation and submission of a full proposal will on average require less effort from their applicants than will single-stage processes. But it is likely that a funder operating a process which begins with a lower-effort outline stage will receive more applications than they might have expected to see had applicants been required initially to prepare a full proposal in a single-stage process. The net effect of these interacting and competing influences on the overall effort required in, and efficiency of, a funding process is not currently known and so is investigated in this work using an Agent-Based Modelling approach. The results of this model suggest that while the number of applications submitted will indeed increase, perhaps by as much as 40%, if two-stage processes are used, the level of applicant effort per unit output (that is, unit of funding awarded or number of awards made) may be reduced by around 15% to 20%. A weaker but more general interpretation, that does not rely so much on the specifics of the model, is that substantial increases in demand arising from use of outline processes might still come with an overall decrease in applicant effort. A reasonable conclusion is that more extensive use of two-stage research funding processes may lead to significant cost savings.

    May 12, 2026

  • Article

    Zhicheng Lin

    Scientific communication faces a dual crisis: exponential publication growth overwhelms human readers, and fragmented research practices block automated synthesis. AI-assisted writing exacerbates the volume problem, producing papers faster than they can be read. Behavioral and social sciences in particular suffer from incomparable stimulus databases, jingle–jangle measurement fallacies, and demographic blindness that conceals effect heterogeneity. Current AI tools aid comprehension and summarization yet cannot aggregate findings from incommensurable studies and risk amplifying biases when trained on unstructured, unverified text. We propose restructuring scientific papers for dual audiences: front-loaded narratives for time-pressed human readers, paired with machine-readable appendices containing executable code, standardized metadata, and ontologically mapped constructs. This design turns papers into queryable research environments where readers can interrogate data and rerun analyses, and where structured appendices enable automated verification of statistical methods and AI-assisted peer review grounded in executable rather than narrative claims. Such papers become nodes in continuously updated evidence networks: each publication automatically contributes effect sizes to real-time meta-analyses, with corrections and retractions propagating through dependent analyses. Widespread adoption will require institutional recognition of structured documentation as essential scholarly output and computational infrastructure that serves both human comprehension and machine analysis.

    May 6, 2026

  • Article

    Moin Syed, Caroline H Armstrong, Emily J Chan, Abby Person

    A persistent concern about implementation of preregistration and Registered Reports in psychology is that doing so would reduce the frequency and value of exploratory research, and therefore restrict creativity, serendipity, and discovery. As we are nearly 15 years on from the initial proposal to adopt registration in psychology, it seems time to formally examine whether these concerns have any merit. The purpose of the present study is to find out. In this proposed project, we will examine a matched set of Registered Reports, preregistered articles, and traditional articles (total N = 300) for the frequency of reported exploratory research, defined as any analysis that is indicated to not be a planned test of a specific hypothesis. This project will provide strong data that should provide an empirical basis to be used in future discussion about the impact of registration on exploratory research.

    May 5, 2026

  • Article

    Lukas Röseler, Lukas Wallrich, Helena Hartmann, Luisa Altegoer, Veronica Boyce, Sarahanne Field, Janik Goltermann, Joachim Hüffmeier, Charlotte Pennington, Merle-Marie Pittelkow, Priya Silverstein, Don van Ravenzwaaij, Flavio Azevedo

    The practice of repeatedly testing published results with the same data (reproduction) or new data (replication) is currently gaining traction in the social sciences, owing to multiple failures to reproduce and replicate published findings. Along with increased skepticism have come guidelines for the repeated testing of hypotheses from various disciplines and fields. This guide aims to enable researchers to conduct high-quality reproductions and replications across social science disciplines. First we summarize recent developments, then provide a comprehensive guide to carrying out reproductions and replications, and finally present an example for how guidance needs to be tailored for specific fields. Our guide covers the entire research process: choosing a target study, deciding between different types of reproductions and replications, planning and running the new study, analyzing the results, discussing outcomes in the light of potential differences, and publishing a report.

    May 4, 2026

  • Article

    Steffen Lemke, Isabella Peters

    The term Bronze open access (Bronze OA) describes research articles that are free to read on publisher-hosted sites, but lack concrete license information that would allow any form of reuse beyond reading. Bronze OA is an under-researched facet of open access surrounded by many uncertainties regarding its causes and their individual significance. This study aims to reduce these uncertainties by providing an overview over the state of Bronze OA within Web of Science, and by investigating relationships between research fields, publishers, and their rates of Bronze OA. We analyze 3,943,511 Bronze OA publications registered in Web of Science since 2000, applying statistical analyses and heat maps. Our results show high occurrences of Bronze OA in biologic and medical fields, while arts & humanities, engineering sciences, physics, and chemistry comprise the lower end of the spectrum. Regarding publishers, large university presses and the BMJ Group stand out as heavy users of Bronze OA. Continuations of this study will semi-manually investigate article pages of Bronze OA publications to provide empirical evidence for the significance of different speculated reasons for the Bronze OA label.

    May 4, 2026