Published at MetaROR
July 22, 2026
Table of contents
Matching data references and institutional output to map the reuse of biomedical research data
Originally published on April 17, 2026 at:
Editors
Ludo Waltman
Ludo Waltman
Editorial assessment
by Ludo Waltman
In this article the authors study the reuse of data sets shared by researchers from a biomedical research institution. The article has been reviewed by three reviewers. The reviewers are generally positive. Reviewer 1 for example praises the ‘very thorough’ methods description and considers the article ‘a very thorough and nice example of practicing open science’. Reviewer 3 considers the research to be ‘very exciting’. The reviewers also provide detailed suggestions for improving the article. Reviewers 2 and 3 for example suggest being more explicit about the research questions or the purpose of the research. Somewhat related to this, reviewer 1 suggests adding a discussion of use cases. Furthermore, reviewers 1 and 2 recommend expanding the discussion of the Data Citation Corpus, and reviewer 2 suggests some conceptual clarifications that the authors could make. The reviewers also provide other smaller suggestions for improving the article.
Recommendations for enhanced transparency
- Add an email address for the corresponding author.
For more information on these recommendations, please refer to our author guidelines.
Peer review 1
Summary/strengths
This paper presents a very thorough description of the methods used to trace the reuse of datasets published by authors at the Charité University Medical Hospital between 2020 and 2023. The study used two methods, both focusing on detecting reuse via references to data in the published literature: first via the Data Citation Corpus and second, using an analysis of datasets described in a sample of data papers. The level of detail in the methods is a strength, and this can be a useful starting point for others struggling with the sticky problem of how to trace dataset reuse. This is further augmented by the availability of the data, code, and text, which has been published in both Github and Zenodo – a very thorough and nice example of practicing open science.
The paper also presents some interesting findings about the nature of datasets which have been reused. Some of these ideas are not new, but this study provides empirical evidence to support them. Findings that I found particularly interesting or surprising included evidence that:
-
Many of the analyzed datasets are referenced only once, if at all
-
Human data had a high level of reuse (which the authors propose could be due to increased curation)
-
There is a negative relationship between DOIs and CC licenses and reuse. (The authors explain this surprising finding further in the Results section as being due to the disciplinary nature of the shared data and the preference for using accession numbers in those fields. I would encourage authors to give more nuance to this finding in the Abstract to avoid misinterpretation by people who only skim the paper).
-
Disciplinary metadata and/or data papers (particularly for data deposited in a generalist repository) can lead to a higher level of reuse.
Questions/suggestions
I also have a few points for clarity or suggestions for improvement. Most of these are related to style and structure or methodology.
Terminology.
I suggest that authors clearly define “data articles” much earlier in the paper. There is a good definition in the Discussion section, which could come earlier. This is important as much literature refers to these works as “data papers;” the authors could also add that these two terms are synonyms. The paper spends quite some space defining data reuse, which given the scope of the study is a positive thing. One article which the authors may want to review on this subject is recent work from Borgman and Groth [1], as well as the resulting commentaries which were published in the same journal in response to this article.
Methods.
I suggest renaming this section to “Methods and data sources” and to dedicate a bit more space to describing the Data Citation Corpus (in this section, rather than in the Introduction), as well as reflecting more on its limitations. How was this corpus created in particular? To my recollection this corpus did not originally contain any data citations (meaning citations in the reference list of publications) but only data mentions. Did the authors also encounter that in their work? The documentation on the DCC’s website is unclear on this point. As the authors did not include an examination of the ratio of data mentions to data citations in their work (as stated in the Discussion), I was left wondering if this was due to a limitation in the DCC itself. Similarly, it would help to have more reflection on DataStet, how it has been used in other studies, and any limitations with this tool.
I believe it would help understanding of the methods (which are complex) to bring Figure 1 earlier in the narrative, perhaps even at the beginning of the methods section. Including the data preparation step in the Figure would also help to make the entire process very clear. Similarly, a Figure in 3.8, describing the method for detecting indirect data citations, would help to clarify the entire process.
In section 3.3.1, the paper states that the team chose the “better identifier,” e.g. the DOI. Was this also true for “omics” data, where accession codes are the preferred identifier in practice? The text suggests this is the case, but I was left wondering what makes a DOI better in this case, where practice in the community suggests otherwise.
The seeding process described in 3.3.2 was not completely clear to me. It would be helpful to have a specific example here. Similarly, in 3.8, the authors describe that they differentiated between indirect data citations (indicating data reuse) and no reuse; adding a table with a few examples of both cases would make it very transparent how reuse was determined.
I am also struggling a bit with the extrapolation step described in 3.8. Was this a method that was used elsewhere? Can we assume that the ratio used here was appropriate, especially because this extrapolation led to such a large number of assumed references? Is any type of error analysis possible here?
More basicly, the paper assumes that a data reference is an indication of reuse (or sharing), and the authors justify that assumption well. I wonder if they encountered any false positives in their analysis, i.e. where data were referenced but were not actually used. For example, we know from work on literature citation practices that there are many reasons for citing something; not all of these indicate using a particular work. People may cite an article to position themselves within a field or for more social reasons. This is something that my colleagues and I looked at a bit for data referencing, using interviews [2]. It would be very interesting to know if this was detected at all in, e.g., the analysis of data articles.
Implications/contributions.
I believe that the case of broader applicability for this work could be made earlier on in the article. In the discussion, the authors highlight that this is one of the first studies to focus on tracking data reuse at the institutional level. I also see the method as being a core contribution that others could learn from.
What was missing for me was a discussion about the use of this type of tracking. Do the authors envision it being used to evaluate the institution as a whole? To inform individual researcher evaluations? To feed into a data sharing policy? Could there be any negative repercussions from tracing dataset reuse?
Structure/style.
Introduction/Background: I found this section to be a bit disjointed. If the authors are going to make this a submission to a journal (or even if they are not), they may want to consider using the Introduction to more broadly set out the problem, and to use the Background for the literature review. A small thing: it seemed strange to have the first section in the paper (aside from the Abstract) be labelled with the number 2.
Methods: I like the numbered list at the beginning of the methods section. I would use the same names in this list as in the subsequent section headings.
Possible futures for this preprint. The article is quite long. I do not know if the authors are planning to submit this preprint to a journal (and it is fine if they aren’t!), but in a journal it is quite likely that it would be impossible to include all of the details included here due to length restrictions. This is a strength of a preprint. If the authors do go on to submit this to a journal, they can reference this preprint for further methodological details. Another option (perhaps a very meta one) could be to turn the methods section into a data paper to describe the openly published dataset which is published in the reproducibility package.
[1] Borgman, C. L., & Groth, P. (2025). From Data Creator to Data Reuser: Distance Matters. Harvard Data Science Review, 7(2). https://doi.org/10.1162/99608f92.35d32cfc
[2] Gregory, K., Haustein, S., Poitras, C., Roblin, E., Ninkov, A., Ripp, C., & Peters, I. (2026). Digging deeper into data citations: recognizing and rewarding data work. Research Evaluation, 35, https://doi.org/10.1093/reseval/rvag008
Peer review 2
- Description of the article’s distinctive contribution to the existing literature
- This article provides a novel, large-scale empirical analysis of dataset reuse at the institutional level by linking institutional dataset outputs to references identified in the Data Citation Corpus (DCC). It operationalizes a method for detecting dataset reuse via bibliographic traces and complements this approach with an analysis of indirect data citations through data articles. The study is distinctive in combining multiple detection strategies (DCC matching and forward citation analysis of data articles) and in focusing on reuse patterns within a single biomedical institution. It advances understanding of dataset reuse patterns, determinants of reuse (e.g., repository type, dataset characteristics), and highlights the limitations of current bibliometric approaches to measuring data reuse.
- List of the major strengths and weaknesses of the conceptual framework, methods, and interpretations and discussions of results
- Strengths
- Clear and well-motivated focus on dataset reuse as a proxy for the value of data sharing.
- Innovative methodological design combining DCC-based detection and indirect citation analysis.
- Well-documented and transparent data preprocessing, cleaning, and matching procedures.
- Strong justification for focusing on repository-based data and on citations rather than mentions of data articles.
- Inclusion of both direct and indirect data references provides a more comprehensive view of reuse.
- Use of multiple analytical approaches (descriptive statistics, regression, temporal analysis).
- Engagement with relevant literature on data citation and reuse, with appropriate and up-to-date references.
- Weaknesses
- Conceptual framing of “data reuse” is somewhat narrow and underdeveloped, with limited discussion of different types or functions of reuse (e.g., replication, reanalysis, rhetorical uses).
- The definition and operationalization of “data reference” lacks clarity early in the manuscript, particularly regarding formal vs. informal references and ambiguity handling.
- Determinants of reuse analyzed in the regression model appear somewhat ad hoc and insufficiently grounded in prior literature.
- Limited temporal scope (primarily ~3 years) constrains the ability to detect meaningful trends in reuse over time.
- Strict reliance on quantitative measures of reuse neglects qualitative dimensions (e.g., depth, significance, or purpose of reuse).
- Some methodological choices (e.g., exclusion of preprints, removal of version information) are not sufficiently justified.
- Discussion of limitations is present but could more explicitly connect to implications for interpretation of findings.
- Are the ideas clearly communicated?
- Overall, the manuscript is clearly written and logically structured, and most methods are presented in a transparent and reproducible way. However, several key concepts (e.g., “data reference,” “Data Citation Corpus,” and distinctions between citation, mention, and reuse) require clearer definition earlier in the text. Introducing explicit research questions at the outset and revisiting them in the results would improve clarity and coherence.
- Are the ideas clearly communicated? Is the relevant existing literature described and discussed? Are the references appropriate?
- The manuscript cites relevant and appropriate literature covering data citation, reuse, and open science. However, the conceptual framing could be strengthened by deeper engagement with literature on types and functions of data use (e.g., rhetorical roles of data references) and by more explicitly linking prior findings to the selection of explanatory variables used in the analysis.
- Are all research methods appropriate, rigorous, and clearly described?
- Methods are generally appropriate, rigorous, and well documented, particularly the data preprocessing and matching steps. The dual approach (DCC + data article citations) is a strength. However:
- Some decisions (e.g., excluding preprints, removing version information, handling multiple identifiers) require clearer justification.
- The lack of systematic disambiguation in some cases and reliance on partial validation should be acknowledged more explicitly.
- More detail on the scope, coverage, and definition of the Data Citation Corpus would strengthen methodological clarity.
- Are the results clearly presented? Are graphs easily readable and clearly labelled? Do they contain descriptive captions?
- Results are generally clearly presented, and figures are helpful (e.g., flow diagrams and Sankey diagram). Suggestions for improvement include:
- Introduce key figures (e.g., the workflow diagram) earlier in the manuscript to orient readers.
- Improve readability of figures (e.g., scaling nodes in flow diagrams, clarifying ordering in Sankey diagrams).
- Ensure captions clarify design choices (e.g., orientation, ordering of variables).
- Move method-like details currently in the results section back into the methods for clarity.
- Are the results discussed in the context of previous findings? Are differences from previously reported findings clearly described?
- Yes, the discussion appropriately situates findings within the existing literature and highlights consistencies (e.g., uneven distribution of reuse, importance of disciplinary repositories). Some findings (e.g., lower reuse associated with DOIs) are discussed thoughtfully, though additional supporting evidence or triangulation would strengthen these interpretations.
- Are interpretations and conclusions consistent with the empirical materials and data?
- Largely yes. Interpretations are generally well aligned with the data and appropriately cautious. However, some conclusions about determinants of reuse would benefit from stronger theoretical grounding and additional supporting evidence.
- Are the limitations of the study and their implications for interpreting the results clearly described where applicable?
- Yes, the manuscript includes a thorough limitations section. However, the implications of these limitations (e.g., incomplete detection of reuse, reliance on bibliographic proxies, short time horizon) could be more explicitly linked to how results should be interpreted. Additional emphasis on potential biases in the DCC and dataset coverage would be beneficial.
- Suggestions for improvements
- Clarify and strengthen the conceptualization of “data reuse,” including distinctions among types and functions of use.
- Define “data reference” more clearly at first mention, including handling of ambiguous or informal references.
- Introduce explicit research questions early and revisit them in the results and discussion.
- Provide more detail on the Data Citation Corpus (scope, coverage, limitations) and dataset inclusion criteria.
- Justify key methodological decisions (e.g., exclusion of preprints, removal of version information).
- Expand discussion of determinants of reuse and more clearly link them to prior literature (e.g., via a summary table).
- Improve organization by moving method-like content out of the results section.
- Enhance figures (placement, scaling, captions) to improve readability and interpretability.
- Expand discussion of the qualitative dimensions of reuse (e.g., depth, significance, purpose of use).
- Acknowledge limitations of the short temporal window more prominently and consider extending the analysis if possible.
- Provide more discussion of generalizability of the methods to other institutions or domains.
- Include potential policy or practical implications for data sharing and repository practices.
- Consider future integration of alternative data sources (e.g., OpenAlex, Internet Archive) for detecting informal references.
- Issues related to data and code availability, research ethics, or other issues related to lack of adherence to MetaROR policies (see Author Guidelines)
- The manuscript provides a clear data and code availability statement, including links to GitHub and Zenodo, which supports reproducibility.
- No major concerns related to research ethics or conflicts of interest are evident.
- A minor concern relates to transparency and usability of the Data Citation Corpus: additional discussion of its limitations, data quality issues, and implications for reproducibility would strengthen compliance with MetaROR expectations on transparency.
- The manuscript could further improve adherence to open research best practices by providing clearer documentation of data processing workflows and identifier standardization procedures for reuse by other researchers.
- Results are generally clearly presented, and figures are helpful (e.g., flow diagrams and Sankey diagram). Suggestions for improvement include:
- Methods are generally appropriate, rigorous, and well documented, particularly the data preprocessing and matching steps. The dual approach (DCC + data article citations) is a strength. However:
- Strengths
Peer review 3
I want to congratulate the authors on this very exciting piece of research. The manuscript covers two highly relevant topics: institutional monitoring of Open Science practices and data reuse.
In my opinion, the major strength of the manuscript is the thorough analysis of research data output and reuse for one institution. As the authors point out on p. 22: “This is, to our knowledge, the first analysis to quantitatively show at the scale of a research institution, how the datasets produced are reused and enter the published literature.” I think this statement would fit well in the introduction.
When revising the manuscript, I recommend the authors put a stronger focus on the institutional use case and use that to weave together the different analysis steps.
The description of the methods is very detailed and sound, which is extremely useful for people who want to use the Data Citation Corpus or are looking for examples of institutional monitoring of data output and reuse. Therefore, I want to focus my review on the structure of the manuscript, which I think could be revised for more clarity.
2 Introduction
The discussion of literature in the introduction section is very short, and I recognize that this might be prescribed by the journal the authors chose for this manuscript. The literature is described in detail in the discussion section, which is useful to some extent for contextualizing the authors’ own findings. However, I think readers might need more context early on to be able to follow along. Relevant definitions and findings (for example, what is data reuse, how is it operationalized in studies and how prevalent is it) should be moved from the discussion to the introduction.
I suggest that the authors strenghten the institutional perspective and reframe the analysis as an investigation of institutional data output and reuse, using Charité as a use case. This would make the purpose of the manuscript clearer to readers and explain how the individual steps of the analysis tie in together.
I recommend moving section 2.3 to the top and extending the brief description of Charité and, then follow with the other concepts relevant to the analysis.
4 Results
The authors should check the results section and move interpretations to the discussion section, especially those that are repeated in both the results and the discussion section. One example for this is this paragraph on p. 16: “However, while the data show reuse to be shaped by data content, the impact of repository practices on reuse is difficult to interpret against the backdrop of the specific practices of a set of highly-used disciplinary repositories.” – in my opinion, statements like that better fit in the discussion section.
Figure 5: The sankey plot only shows intersections between two variables, maybe a venn diagram that shows intersections between multiple variables might be more useful here?
5 Discussion
I recommend the authors add a section about institutional monitoring, which would be a nice way to wrap up the Charité perspective: What do the results reveal about monitoring the data output of an institution and how it is reused? What would be needed for more reliable processes and results (improvements of data sources, wider agreement on metrics…)?
I think the manuscript also raises some very interesting questions regarding gaps between policy and practice. I know that the manuscript is already quite long, but maybe the authors could consider discussing some of these points. For example, how can we reconcile that whereas the value of data is primarily communicated in relation to reuse, only a small fraction of datasets are actually reused?
Author response
We would like to thank all reviewers for their very helpful and thorough comments. Below, we describe how we addressed each of the reviewers’ suggestions.
In addition to changes prompted by reviewer comments, we have also made a number of further improvements to the reviewed version. Changes included the correction of the number of overall dataset references, which was not correctly summed up before, and aligning wording around citation vs. reference with the definitions applied. We also improved the precision and clarity of figure labels and captions, as well as of methodological explanations.
Reviewer #1
There is a negative relationship between DOIs and CC licenses and reuse. (The authors explain this surprising finding further in the Results section as being due to the disciplinary nature of the shared data and the preference for using accession numbers in those fields. I would encourage authors to give more nuance to this finding in the Abstract to avoid misinterpretation by people who only skim the paper.
We have added a statement to the abstract to avoid misinterpretation: “Dataset properties indicative of technical reusability as DOIs and CC licenses had no positive influence on reuse probability, presumably because these are not used by the most common omics repositories.”
I suggest that authors clearly define “data articles” much earlier in the paper. There is a good definition in the Discussion section, which could come earlier. This is important as much literature refers to these works as “data papers;” the authors could also add that these two terms are synonyms.
We have shifted the definition to the introduction and have stated explicitly that we understand the terms to be synonymous: “’Data articles’ are a type of article which explains in detail the provenance and properties of a dataset, but does not report results derived from its analysis. Other terms are ‘data descriptor’ or ‘data paper’, and we understand these terms to be synonymous.”
One article which the authors may want to review on [the subject of data reuse] is recent work from Borgman and Groth [1], as well as the resulting commentaries which were published in the same journal in response to this article.
We now make reference to the concept of data distance, which we find very helpful as well:_ “This is complemented by “closed communal sharing, based on a feeling of belonging together” and “closed associative sharing, in which the participants act on the basis of an agreement”. A framework for data reuse which describes the effort needed to reuse data as “data distance” and lists different dimensions which influence this distance is provided by Borgman C Groth (2025). “Communal sharing” is characterized by a short data distance and is thus potentially very effective, but cannot be assessed using bibliographic methods.”
I suggest […] to dedicate a bit more space to describing the Data Citation Corpus (in this section, rather than in the Introduction), as well as reflecting more on its limitations. How was this corpus created in particular? To my recollection this corpus did not originally contain any data citations.
We added a detailed description of why we think the DCC also contains a small fraction of data citations. We decided not to discuss the limitations of the DCC there, but we now explicitly refer to the DCC documentation, which contains a limitations section. As we do not discuss this in detail, we also decided not to shift this to the methods.
Similarly, it would help to have more reflection on DataStet, how it has been used in other studies, and any limitations with this tool.
We now cite Bassinet et al. as the study which describes DataStet, and this article provides the information requested with regard to limitations. DataStet has, to our knowledge, not been used in other published studies, although the related DataSeer tool has.
I believe it would help understanding of the methods (which are complex) to bring Figure 1 earlier in the narrative, perhaps even at the beginning of the methods section. Including the data preparation step in the Figure would also help to make the entire process very clear. Similarly, a Figure in 3.8, describing the method for detecting indirect data citations, would help to clarify the entire process.
We appreciate the value this figure could have for understanding the methods, but this shift would remove the figure with the overall number of detected references, which is arguably the central finding, from the results. This might create new problems with following the article. Given that we refer to figure 1 in the methods, we decided not to shift this figure, and similarly also nor figure 8a which shows the process for detecting indirect data citations.
In section 3.3.1, the paper states that the team chose the “better identifier,” e.g. the DOI. Was this also true for “omics” data, where accession codes are the preferred identifier in practice? The text suggests this is the case, but I was left wondering what makes a DOI better in this case, where practice in the community suggests otherwise.
In retrospect, our choice is not supported by the findings, and we have rephrased this section to reflect this: “As part of this verification process, we documented the most persistent identifier of a dataset, e.g. preferring a DOI over an accession code, where both were available. The decision which identifier to document was based on the assumption of higher usability of data with persistent identifiers. However, this assumption and thus the subsequent methodological choice were not supported by the findings.”
The seeding process described in 3.3.2 was not completely clear to me. It would be helpful to have a specific example here.
We have substantially expanded this section, which now includes an example and reads: “In the detection procedure described in Iarkaeva, Nachev, and Bobrov (2024), as well as briefly in Section 3.3.1, we extracted one dataset identifier per repository. However, we had observed that in many cases multiple datasets had been deposed in the same repository, in some cases up to several dozen. Thus, we used the already documented identifiers as “seeds” to detect further identifiers in the same repository. To do so, the previously extracted identifier of a dataset was searched for in the article text and any additional identifiers from the same repository were also documented. These additional identifiers were only searched for in the text surrounding the seed identifier, as well as any additional sections clearly indicated there as containing further identifiers. Importantly, we only used identifiers as seeds if the dataset had been referenced in the literature, and thus some cases of dataset reuse were probably missed. Zhang et al. (2020) can serve as an example for the procedure. This article contains the statement (shortened from the original): “Crystallographic coordinates are available from the PDB under accession codes 6Y2E, 6Y2F, and 6Y2G.” Initially, accession code 6Y2E had been documented. In a more recent open data detection procedure, starting from 2025, we immediately extract all identifiers mentioned in an article.”
Similarly, in 3.8, the authors describe that they differentiated between indirect data citations (indicating data reuse) and no reuse; adding a table with a few examples of both cases would make it very transparent how reuse was determined.
We have added to section 3.8 “Indirect citation detection via data articles” a table with examples for both cases, as well as examples for direct references to datasets.
I am also struggling a bit with the extrapolation step described in 3.8. Was this a method that was used elsewhere? Can we assume that the ratio used here was appropriate, especially because this extrapolation led to such a large number of assumed references? Is any type of error analysis possible here?
Amongst different extrapolation scenarios, it was difficult to find relevant examples, but there is one precedent with a very similar extrapolation procedure. We have added the statement “A similar extrapolation procedure has been applied by Hardwicke et al. (2021) for an article in their sample which had too many citations to check all manually.”. Regarding error analysis, we do see that a bootstrapping of this outcome using subsampling could be used to determine the variance. However, we decided not to do this for time reasons. Certainly, knowing the variance would help in interpreting the result, but sixty manually checked references per data article seems to us a solid basis for a sufficiently small variance. Also, we explicitly point to the fact, that one outlier contributes a large number of indirect data citations.
More basicly, the paper assumes that a data reference is an indication of reuse (or sharing), and the authors justify that assumption well. I wonder if they encountered any false positives in their analysis, i.e. where data were referenced but were not actually used. For example, we know from work on literature citation practices that there are many reasons for citing something; not all of these indicate using a particular work. People may cite an article to position themselves within a field or for more social reasons. This is something that my colleagues and I looked at a bit for data referencing, using interviews [2]. It would be very interesting to know if this was detected at all in, e.g., the analysis of data articles.
We have rephrased the respective paragraph in the methods to make it clearer that all indirect data citations were cases of data reuse: “We distinguished between cases of dataset reuse (i.e., indirect data citations), and cases where no data reuse took place. These second cases correspond to references to the data article text proper. While we attempted to detect all indirect data citations, we note that references to the text proper could also concur with indirect data citations. All cases classified as indirect data citations referred to some use of the data.
However, we did not distinguish between different purposes of data reuse (e.g., re-analysis, meta-analysis, comparison). For data article references proper, we also did not distinguish between different citation contexts (e.g., supportive, contradictory, neutral). A full mapping of the distribution and purposes of data article citations was not in scope for this study.“
In addition, we have adjusted the corresponding text in the methods, included an additional reference and adding that the literature strongly suggests a broad range of uses also in our sample: “The observations by Lafia et al. (2023) that only a subset of references indicates “interactions” with the dataset and by Gregory et al. (2026) that data is referenced for many reasons strongly suggest that not all references to datasets we observed are indicative of active engagement with the data.“
What was missing for me was a discussion about the use of this type of tracking. Do the authors envision it being used to evaluate the institution as a whole? To inform individual researcher evaluations? To feed into a data sharing policy? Could there be any negative repercussions from tracing dataset reuse?
A discussion of uses is present in the Outlook section, but it does indeed not address the points mentioned by the reviewer. In the article, we highlight the possibility to inform individual researchers about data reuse to motivate them for future data sharing. This is in our opinion the most important immediate application. We also mention analyses which could guide institutional action. While it is conceivable to use data reuse numbers for researcher evaluations on the institutional level, for our institutional use case this would currently not be advisable, unless very well embedded and communicated. One important reason for this reservation is the very skewed distribution of both data sharing and data reuse between research fields. Thus, we abstain from making this recommendation in the Outlook. However, such evaluations would work substantially better when comparing researchers from the same field, as e.g. in hiring processes. Also, we do not list negative repercussions, as we do not see any immediately negative consequences, although they are conceivable downstream if data reuse numbers are taken up as metrics for evaluation.
[…] it seemed strange to have the first section in the paper (aside from the Abstract) be labelled with the number 2.
We have adjusted this.
I like the numbered list at the beginning of the methods section. I would use the same names in this list as in the subsequent section headings.
We have harmonized the items of the numbered list and the first section headings.
The article is quite long. I do not know if the authors are planning to submit this preprint to a journal (and it is fine if they aren’t!), but in a journal it is quite likely that it would be impossible to include all of the details included here due to length restrictions. This is a strength of a preprint. If the authors do go on to submit this to a journal, they can reference this preprint for further methodological details. Another option (perhaps a very meta one) could be to turn the methods section into a data paper to describe the openly published dataset which is published in the reproducibility package.
We thank the reviewer’s for these suggestions. We decided not to submit the article to a journal, but rather to pursue a reviewer version in MetaROR. We will not pursue a data article at this point, but with more data collected it might be a strategy for the future.
Reviewer #2
Conceptual framing of “data reuse” is somewhat narrow and underdeveloped, with limited discussion of different types or functions of reuse (e.g., replication, reanalysis, rhetorical uses).
We have addressed this, as well as a comment by reviewer #1, by adding a reference to a recent article and by mentioning that the data interaction category again includes different subtypes. The section now reads: “The observations by Lafia et al. (2023) that only a subset of references indicates “interactions” with the dataset and by Gregory et al. (2026) that data is referenced for many reasons strongly suggest that not all references to datasets we observed are indicative of active engagement with the data. As we did not restrict our analysis to instances of proven interaction with datasets, we could also not investigate specific types of interaction, as e.g. replication and re-analysis. An additional consideration given our inclusion of all types of data reference, regardless of purpose, is to assess which kind of value is created by other types of data reuse than “interaction”, and whether these other uses are then still in scope for the monitoring exercise conducted.”
The definition and operationalization of “data reference” lacks clarity early in the manuscript, particularly regarding formal vs. informal references and ambiguity handling.
We have added a text box within the introduction with definitions of the following terms: Data article; data reuse; data citation; data mention; data reference; indirect data citation
Determinants of reuse analyzed in the regression model appear somewhat ad hoc and insufficiently grounded in prior literature.
We have added multiple references to justify the choice of CC licenses, DOIs, and the presence of data mentions in data availability statements as factors in our analysis.
Some methodological choices (e.g., exclusion of preprints, removal of version information) are not sufficiently justified.
We have added a justification of the exclusion of preprints and version information:
- “preprints were excluded to avoid the consolidation step of removing those preprints which were later also published as peer-reviewed articles (i.e., “double-dipping”); this is all the more important, as peer-reviewed versions appear with a variable delay, and thus the set of preprints to include would be changing over time, increasing the complexity of updating or re-analyzing the data; this choice was also consistent with the detection of Charité articles, which did not include preprints”
- “this was done to detect all uses of a dataset, regardless of its version, based on the assumption that a new version of a dataset remains the same dataset”.
Discussion of limitations is present but could more explicitly connect to implications for interpretation of findings.
Yes, the manuscript includes a thorough limitations section. However, the implications of these limitations (e.g., incomplete detection of reuse, reliance on bibliographic proxies, short time horizon) could be more explicitly linked to how results should be interpreted.
In section “4.7. Outlook” we had written a section to the effect that informing researchers about reuse of datasets could motivate them to share more in the future. We added to this the statement: “In such communication to researchers, the limitations of the study need to be considered, in particular that referencing a dataset often but not always means the data has been reused. In addition, the communication cannot be targeted specifically to those researchers which created, curated and/or deposited the dataset.”
Introducing explicit research questions at the outset and revisiting them in the results would improve clarity and coherence.
In section 2.3., we already state with regard to research questions: “The primary questions of our study are which datasets have been mentioned in the literature, and how often they were mentioned. In addition, we investigated the characteristics of reuse with regard to properties of data (e.g., whether it is data collected from human participants), properties of metadata (e.g., license), and the time lag between data publication and data reference.” The results section follows the outlined sequence, starting with the primary questions. Thus, we did not clearly see how we could address this comment.
The lack of systematic disambiguation in some cases and reliance on partial validation should be acknowledged more explicitly.
We have changed the statement “Some limitations of this analysis need to be mentioned.” to “Some limitations of this analysis, in addition to uncertainty introduced by partial validation, need to be mentioned.” Regarding the systematic disambiguation, we had conducted deduplication and cleaning steps, as described in section 2.4. and we were not sure where the process might have lacked in disambiguation.
More detail on the scope, coverage, and definition of the Data Citation Corpus would strengthen methodological clarity.
Following both reviewer #2 and reviewer #1 comments, section “2.4 Data Citation Corpus as data source” now contains several additional links to DCC methodology.
Improve readability of figures (e.g., scaling nodes in flow diagrams, clarifying ordering in Sankey diagrams).
We were uncertain about this comment. As the nodes already seemed to be sufficiently large and the Sankey plot already had a detailed caption, we did not change anything in this regard.
Ensure captions clarify design choices (e.g., orientation, ordering of variables).
We assume that this refers primarily to figures 4 (Determinants of Institutional Dataset Reuse) and 5 (Properties of Institutional Dataset Reuse), where several variables are presented and ordering could have been different. We were not sure how to address this comment with regard to orientation, as the caption of figure 4 already stated “Values >1 indicate higher odds and values <1 indicate lower odds”. It would be conceivable to make the choice of the sequence in the Sankey plot explicit, but a lot of background knowledge and assumptions about what is most related and most interesting went into this order. For example, we knew that most datasets with accession codes do not provide CC licenses, and that of datasets with CC licenses most are fully open. Also, we assumed that readers will mostly be interested in how repository choice relates to human data and to data openness, which are correspondingly placed to the left and right of the repository column. Making all the knowledge and assumptions would have been very extensive and we don’t think this would have contributed sufficiently to most readers. In addition, we have added to the caption of figure three a clarification that in cases of a tie, repositories are listed in reverse alphabetical order.
Move method-like details currently in the results section back into the methods for clarity.
When looking at specific cases, it often seemed to us that a repetition of some methodological detail in the results was necessary to keep it clear. However, we have now shifted information on logistic regression and multiple-comparison correction (figure 4) to the methods, where it was so far not addressed. Correspondingly, this information has been abridged in the results section.
Additional emphasis on potential biases in the DCC and dataset coverage would be beneficial.
We have added a bullet point to the limitations section which reads: “We have addressed some data quality issues of the DCC by cleaning steps. However, there are no studies as yet investigating potential bias in the DCC, which could affect e.g. certain disciplines, repositories, or identifier types. Thus, it is unclear how faithfully the DCC represents the complete body of dataset references in the literature, and results e.g. on repositories or determinants of data reuse could be systematically skewed.”
Acknowledge limitations of the short temporal window more prominently and consider extending the analysis if possible.
Extending the analysis was unfortunately not possible at this stage, but we included an additional statement in this regard in the Limitations section: “The large majority of datasets was published between 2020 and 2023, so the findings might not be representative of other time periods. This is in particular due to overlap with the COVID-19 pandemic, which might have led to transient changes regarding which datasets were predominantly shared and reused. The analysis of the development of dataset references over time is exploratory and needs to be interpreted with particular caution due to both the small sample and the short time window.” We are planning to extend our analyses to subsequent years and publish this dataset in the future.
Provide more discussion of generalizability of the methods to other institutions or domains.
We added the statement: “We see potential to reuse the procedure described here to conduct similar analyses at other institutions. The overview of institutional datasets we had before collated was very comprehensive and include time-consuming validation steps. However, it would be a valid approach to match all datasets by an institution’s researchers to the DCC, regardless of the compliance of these datasets with specific open data criteria. It would have to be minimally ensured, however, that the datasets matched to the DCC are really own and not reused datasets.”
Include potential policy or practical implications for data sharing and repository practices.
We had stated “The Data Usage Typology might help to capture some of this information on the repository level, although for repositories which do not require registration for download a full picture will remain very difficult to determine.” Instead we now say “The Data Usage Typology might help to capture some of this information on the repository level. We recommend that at least those repositories which require registration for download implement this typology to allow a much more complete picture of which datasets are reused, and for which purpose. In addition, repositories could also leverage the DCC to increase the visibility of data reuse by exposing reuse information on dataset landing pages.”
Consider future integration of alternative data sources (e.g., OpenAlex, Internet Archive) for detecting informal references.
We do not see a way of using these sources to detect informal references (which we understand as equivalent with ‘data mentions’ in the article). To our knowledge, they do not contain any information in this regard within their metadata.
The manuscript could further improve adherence to open research best practices by providing clearer documentation of data processing workflows and identifier standardization procedures for reuse by other researchers.
We point out in the respective session that further details on standardization are reported in the reproducibility package.
Reviewer #3
In my opinion, the major strength of the manuscript is the thorough analysis of research data output and reuse for one institution. As the authors point out on p. 22: “This is, to our knowledge, the first analysis to quantitatively show at the scale of a research institution, how the datasets produced are reused and enter the published literature.” I think this statement would fit well in the introduction.
We kept the statement, but added to the introduction the related statement: “This study is, to our knowledge, the first to analyze dataset references in the published literature with focus on a research-performing institution.”
The discussion of literature in the introduction section is very short, and I recognize that this might be prescribed by the journal the authors chose for this manuscript. The literature is described in detail in the discussion section, which is useful to some extent for contextualizing the authors’ own findings. However, I think readers might need more context early on to be able to follow along. Relevant definitions and findings (for example, what is data reuse, how is it operationalized in studies and how prevalent is it) should be moved from the discussion to the introduction.
We have added relevant literature, including the DCC methodology, in the introduction and the discussion. We also followed the request reviewer #3 as well as the others made and included in the end of the introduction a text box with definitions to summarize and clarify them.
When revising the manuscript, I recommend the authors put a stronger focus on the institutional use case and use that to weave together the different analysis steps.
I suggest that the authors strengthen the institutional perspective and reframe the analysis as an investigation of institutional data output and reuse, using Charité as a use case. This would make the purpose of the manuscript clearer to readers and explain how the individual steps of the analysis tie in together.
I recommend moving section 2.3 to the top and extending the brief description of Charité and, then follow with the other concepts relevant to the analysis.
We had already put the study into institutional perspective to some extent, in particular by including section 1.3 (“Institutional dataset reuse as specific application”), to which the reviewer also referred. We also referred to the Charité as the scope of the study throughout the article. Following the above as well as other reviewer comments, we have now expanded the discussion on communication of the findings to the institution’s researchers as a specific use case (3rd paragraph of the Outlook, section 4.7). We decided not to take it even further, as we would like to keep the focus of the study sufficiently broad to engage anybody interested in data reuse.
The authors should check the results section and move interpretations to the discussion section, especially those that are repeated in both the results and the discussion section. One example for this is this paragraph on p. 16: “However, while the data show reuse to be shaped by data content, the impact of repository practices on reuse is difficult to interpret against the backdrop of the specific practices of a set of highly-used disciplinary repositories.” – in my opinion, statements like that better fit in the discussion section.
We agree that some content was better placed in the discussion and shifted several statements accordingly.
Figure 5: The sankey plot only shows intersections between two variables, maybe a venn diagram that shows intersections between multiple variables might be more useful here?
We looked into this, but we think that a Venn diagram would not be able to capture, at least not in a readable way, the combination of several variables each having several possible values. A chord diagram would be an option here, but for this number of variables even that would be very difficult to read.
I recommend the authors add a section about institutional monitoring, which would be a nice way to wrap up the Charité perspective: What do the results reveal about monitoring the data output of an institution and how it is reused? What would be needed for more reliable processes and results (improvements of data sources, wider agreement on metrics…)?
We have extended the respective section with the following statement (which in part also addresses other comments): “We see potential to reuse the procedure described here to conduct similar analyses at other institutions. The overview of institutional datasets we had before collated was very comprehensive and include time-consuming validation steps. However, it would be a valid approach to match all datasets by an institution’s researchers to the DCC, regardless of the compliance of these datasets with specific open data criteria. It would have to be minimally ensured, however, that the datasets matched to the DCC are really own and not reused datasets. Alternatively, identifiers of published datasets could be collected as part of an institutional bibliography.”
I think the manuscript also raises some very interesting questions regarding gaps between policy and practice. I know that the manuscript is already quite long, but maybe the authors could consider discussing some of these points. For example, how can we reconcile that whereas the value of data is primarily communicated in relation to reuse, only a small fraction of datasets are actually reused?
We included in the Outlook the following discussion regarding this issue: “Although we don’t consider it likely, such communication [i.e., to researchers whose data was referenced] could trigger broader discussions around the value of data sharing, which might revolve around most data not being referenced. In such a discussion, it would be important to highlight that a large fraction of references was probably missed, that datasets continue to accrue references over longer periods of time, and that value can also be created e.g. by increasing transparency and trust. Also, this finding can be put in context by referring to the highly skewed distribution of article references, and the comparably low effort of sharing a dataset vis-à-vis typical research project durations.”













