ro2019
Workshop on Research Objects 2019
Peer Review of RO-8
- Author: David N. Kennedy
- Title: The ReproPub: A hybrid research object for supporting publication-level re-execution and generalization of neuroimaging research findings
- Submission: https://doi.org/10.5281/zenodo.3336609
- Submitted as: RO2019 abstract
- Decision: accept talk (accepted for oral presentation)
Review 1
- Reviewer: Daniel Garijo https://orcid.org/0000-0003-0454-7145
Quality of Writing
Is the text easy to follow? Are core concepts defined or referenced? Is it clear what is the author’s contribution?
- good
The paper is well written and easy to follow
Research Object / Zenodo
URL for a Research Object or Zenodo record provided? Guidelines followed? Open format (e.g. HTML)? Sufficient metadata, e.g. links to software? Some form of Data Package provided? Add text below if you need to clarify your score.
- sufficient (e.g. HTML, detailed Zenodo metadata)
The article was available in Zenodo and appropriately described there.
Overall evaluation
Please provide a brief review, including a justification for your scores. Both score and review text are required.
- accept
This paper presents an approach for capturing all the aspects of a scientific publication as a Research Object in the neuroimaging domain. The approach is highly relevant for the workshop, and I look forward to its presentation. That said, I have many questions that remain unanswered in the abstract:
- What is the difference between repro-pubs and research objects?
- What are the assumptions of a repropub? For example, data size is usually bigger than what it’s allowed in a GitHub release.
- How difficult is to create a repropub? How accessible is to users?
- Does the docker image include all the dependencies in the workflow of a repropub, or does a repropub require setting up a workflow environment that invokes the respective containers of each step?
Review 2
- Reviewer: (anonymous)
Quality of Writing
Is the text easy to follow? Are core concepts defined or referenced? Is it clear what is the author’s contribution?
- good
Research Object / Zenodo
URL for a Research Object or Zenodo record provided? Guidelines followed? Open format (e.g. HTML)? Sufficient metadata, e.g. links to software? Some form of Data Package provided? Add text below if you need to clarify your score.
- sufficient (e.g. HTML, detailed Zenodo metadata)
Overall evaluation
Please provide a brief review, including a justification for your scores. Both score and review text are required.
- weak accept
This abstract introduces the “ReproPub” concept as a publication that includes the provenance of the related experimental data, workflow, execution environment, and results, in order to support re-executability of the research findings.
The ReproPub concept is, in fact, in the same line and spirit as the research object concept. Thus, for instance, a research object can aggregate other research objects (called nested research objects).
However, even though technically a research object could represent each of the mentioned elements (data, workflow, execution environment, results), as proposed by the author, the main purpose of the research object is to encapsulate all the materials and methods relevant to a scientific investigation, the associated annotations and the context where such resources were used and produced (including provenance).
Hence, it’s more about the overarching aggregation mechanism (as the ReproPub concept), rather than the individual element encapsulation.
One question is regarding the proposed implementation for ReproPub. From the text and the proof of concept reference, it seems that the proposed approach is to have a typical publication (e.g., textual document), which includes references to where the individual elements can be accessed, e.g., GitHub, DockerHub, Zenodo, etc. This can, of course, improve human readability and understanding, allowing to potentially reproduce the investigation.
However, as ReproPubs are not addressing the formal representation of relations and metadata supporting machine-readable access, they may leverage further research objects in order to support features like (automatic) quality assessment, decay prevention, findability, indexing, enrichment, etc.
Hence, it will be go to analyse further possible complementarities/relations between ReproPubs and research objects, e.g., having a ReproPub as a named type of resource in a research object, potentially being able to be created automatically from a research object content.