Recommendation
The paper “Research workflows, paradata, and information visualisation: feedback on an exploratory integration of issues and practices - MEMORIA IS” (Dudek & Blaise, 2023) describes a prototype of an information system developed to improve the traceability, transmissibility and verifiability of archaeological research workflows. A key aspect of the work with MEMORIA is to make research documentation and the workflows underpinning the conducted research more approachable and understandable using a series of visual interfaces that allow users of the system to explore archaeological documentation, including metadata describing the data and paradata that describes its underlying processes. The work of Dudek and Blaise address one of the central barriers to reproducibility and transparency of research data and propose a set of both theoretically and practically well-founded tools and methods to solve this major problem. From the reported work on MEMORIA IS, information visualisation and the proposed tools emerge as an interesting and potentially powerful approach for a major push in improving the traceability, transmissibility and verifiability of research data through making research workflows easier to approach and understand.
In comparison to technical work relating to archaeological data management, this paper starts commendably with a careful explication of the conceptual and epistemic underpinnings of the MEMORIA IS both in documentation research, knowledge organisation and information visualisation literature. Rather than being developed on the basis of a set of opaque assumptions, the meticulous description of the MEMORIA IS and its theoretical and technical premises is exemplary in its transparence and richness and has potential for a long-term impact as a part of the body of literature relating to the development of archaeological documentation and documentation tools. While the text is sometimes fairly densely written, it is worth taking the effort to read it through. Another major strength of the paper is that it provides a rich set of examples of the workings of the prototype system that makes it possible to develop a comprehensive understanding of the proposed approaches and assess their validity.
As a whole, this paper and the reported work on MEMORIA IS forms a worthy addition to the literature on and practical work for developing critical infrastructures for data documentation, management and access in archaeology. Beyond archaeology and the specific context of the discussed work discussed this paper has obvious relevance to comparable work in other fields.
DOI or URL of the preprint: https://doi.org/10.5281/zenodo.7885903
Version of the preprint: 1
I have uploaded a revised version in line with the reviewers' comments.
Yours sincerely
The reviewers are very positive about the paper, however, I find that revising the text for clarity, style and grammar following the advice from the two first reviewers, and adding some explanatory details laid out in the reviews, would enhance the it before publication.
The authors present a series of visual tools for the description and visualization of research workflows, paradata and other information that foster research output sharing and reproduction. While many will not find the topic of data documentation particularly exciting, it concerns critical infrastructure for the reproducibility and transparency of research data, with the authors offering a much needed solution to a gaping hole.
Introduction and abstract summarize the content neatly. The rationale is the present lack of tools for the systematic documentation and archiving of research results. The risk being a significant information loss over time and through personnel changes within research projects. As a research project director who has tried to reuse my own and other people's data, I sincerely agree and trust others will as well. Concepts and specialized terms, such as metadata and paradata, are explained and contextualized with an excellent bibliography. The topic of documenting research workflows is broken down into different stages, and authors trial different visual formalisms to represent the workflow processes, inputs, and outputs. The details of methods are adequate, language clear and understandable to a domain-expert but will need to be taken down a notch for an informed but non-digital-data person (e.g..stocktaking may be more understandable than 'elicitation cycle', etc).
While the topic of documentation is never the most exciting subject, the authors lighten it up as much as possible by offering a clear prose, and doing an excellent job categorizing the different aspects of documentation (e.g. why and for whom we are documenting) that is both informative, well-structured and empathetic. Occasional typos creep in ("How does the notion of metadata and paradata unfolds", etc. ), needing one more read-over with an eye to spelling and grammer. With practical examples and up-to-date informative references, this paper is a concise and useful guide for both students of archiving and documentation as well as practitioners who attempt to archive their data or reuse the data of others.
This article offers a valuable study on the documentation of scientific research processes aimed at dealing with and overcoming the problems connected with scientific results' traceability, transmissibility, and verifiability and builds on relevant recent and past research performed in the field. The authors address such a research problem by designing and developing an Information System (IS) based on Information Visualization, Memoria, which allows for data understanding and reusability.
The system was created with an approach that takes into account the results of research activities, process descriptions, and the need for comparative analysis.
According to the authors, the Memoria IS can be employed in different fields of research connected with heritage science, and it was tested with research data from architectural heritage and archaeology. Besides presenting the many benefits and advantages of such a system, the authors also caution against the potential issues and encourage the research community to engage in collaborative discussions to address these issues in light of the current concerns of openness and the long-term preservation of research data.
While the manuscript presents valid content, there appear to be some opportunities to improve the style, formatting, and language usage. Addressing these concerns would likely enhance the overall reading experience and bolster the narrative quality.
To enhance the manuscript's quality, it is advisable to anticipate the content of each section and revise paragraphs that sound too informal. The language sometimes sounds more like a speech than a scientific paper, which may require editing to highlight the scientific value of the contribution. Rephrasing sentences and reviewing paragraphs are necessary for this purpose. Improving the language will help increase the article's readability. Some sections lack cohesion, and some paragraphs appear disjointed. Additionally, the formatting is unclear at times, with the use of italics, bold, and ellipsis having no clear purpose. Separating the sections and subsections better would improve the reading experience.
Improving the weak aspects of the paper significantly enhances the validity of the presented ideas and solutions. Doing so would also recognize the authors' important efforts and valuable contributions in tackling a significant issue for the scientific community.
The abstract introduces the topic of the paper and presents its main findings. However, some sentences are unclear and do not flow smoothly. Rephrasing some sentences would help the reading of the abstract.
While the introduction covers the motivation, research questions, and study predictions based on recent and previous research, the language and style used could be improved.
To improve the clarity of the Material and Methods section, it would be beneficial to provide a brief introduction outlining the structure of the section. In the first paragraph, where the authors discuss the principles they based their approach on, it is unclear whether they are presenting ongoing reflections or decisions that have already been made. The research problem is identified in the subsection that discusses workflow processes, and the chosen visual approach is clearly stated. These concepts could be introduced earlier in the introduction section.
In the workflow section from an InfoViz perspective, Figure 3 (line 228) refers to a picture that presents the Memeoria workflow nodes. However, the paragraph is about time representation in more general terms. To avoid confusion, the reference to Figure 3 could be moved to the following section, where the authors describe the memoria workflow diagrams.
The structure of the memoria workflow could be improved by introducing the concept of activities and activity groups beforehand. The caption in Figure 4 describes a node that refers to activities that have not been introduced yet. Additionally, Figure 5 is unclear, as the small text is unreadable, and the A to E description in the text does not match the attached figure. A clear description of the system's structure must be provided to prevent any confusion and ensure full understanding.
The discussion is firmly based on the results and findings. However, it may be helpful to incorporate additional studies from current and previous research to enhance its validity.
All references are correctly included, and the necessary citations are present. Quotations (lines 51, 68, 79, 141, 233, 239) should be properly formatted according to the selected reference style and include a page number reference.
Acronyms must be fully spelt out the first time they are used in the text.
The paper titled "An Exploratory Web Information System for Heritage Studies: Enhancing Traceability and Knowledge Sharing in Scientific Research" presents a comprehensive and valuable contribution to the field of heritage studies. The authors have successfully developed a web-based information system, MEMORIA, which addresses practical and epistemological challenges faced by scientific units studying architectural heritage.
The paper highlights the methodological and analytical potential of the MEMORIA system in terms of describing, analyzing, and sharing research workflows. This system not only ensures traceability, transmissibility, and verifiability of scientific results but also meets the demands of open science by providing free access to the produced content. This is a commendable effort towards fostering transparency and accessibility in scientific research.
The writing of this paper is clear and well-written. The authors effectively communicate their research objectives, the key concepts behind their approach, and how these concepts are implemented in practice. The language used is concise, yet it provides a comprehensive understanding of the MEMORIA system, making it accessible to a broad audience. Furthermore, the paper showcases the authors' proficiency in presenting complex ideas through the effective use of figures. The figures are of excellent quality, aiding in the visualization and comprehension of the concepts discussed. This approach has significant implications for heritage studies, as it enables a deeper understanding of research outcomes and facilitates collaboration among researchers in the field.
This paper demonstrates a systematic approach to scientific process documentation that addresses the challenges faced in heritage studies. According to my perception, the paper fulfills the criteria of a well-structured and well-presented scientific contribution. It brings novel insights to the field of heritage studies and offers a robust framework for enhancing traceability and knowledge sharing in scientific research.