Peer Review of RO-4

Author: Isabelle Perseil, Petr Holub
Title: Towards a reconstructive methodology for building a provenance metamodel that fits the complex bioinformatics pipelines
Submission: https://doi.org/10.5281/zenodo.3362412
Submitted as: RO2019 abstract
Decision: accept talk (accepted for oral presentation)

Review 1

Reviewer: Daniel Garijo https://orcid.org/0000-0003-0454-7145

Quality of Writing

Is the text easy to follow? Are core concepts defined or referenced? Is it clear what is the author’s contribution?

fair

I found this abstract easy to follow until the “proposed approach” section. The authors are not clear with their terminology (e.g., provenance meta-models is never defined), and the phrasing of the approach is a little confusing.

Research Object / Zenodo

_URL for a Research Object or Zenodo record provided?

none (e.g. only abstract in easychair/github)

I have reviewed a pdf without links to other platforms.

Overall evaluation

Please provide a brief review, including a justification for your scores. Both score and review text are required.

borderline

This abstract described an interesting approach, highly related to the topic of the workshop, which is to find a common provenance model in bioinformatics. However, I am confused by several aspects of the proposal:

Why is the approach supposed to be developed at the workshop? I mean, I agree that the authors can present it and discuss it with the participants to refine it, but the authors should develop an approach prior to the event.
How is the model for bioinformatics supposed to be different from existing standards like PROV?
What do the authors mean by evolving provenance artifacts? While it’s true that an entity is mutable, in the context of scientific workflows artifacts are usually static and can be given an id and stored. Or do the authors refer to those cases where researchers use external datasets, too big for them to store, which may go away?
Why does the model (or meta-model) need to be distributed? Why does the model need to evolve?

Review 2

Reviewer: (anonymous)

Quality of Writing

Is the text easy to follow? Are core concepts defined or referenced? Is it clear what is the author’s contribution?

poor

Standards have not been referenced. Until objective I could follow quite well what the idea behind the abstract is but the approach did not clarify the proposed methodology.

Research Object / Zenodo

_URL for a Research Object or Zenodo record provided?

none (e.g. only abstract in easychair/github)

Overall evaluation

Please provide a brief review, including a justification for your scores. Both score and review text are required.

weak accept

I like the objective to build a methodology to come up with domain specific provenance metadata models. However, the approach did not clarify how this would be addressed. The approach was (for me) not structured enough to grasp it, whereas to build a methodology this requires a defined and structured way of working to convey this to others.