Why-Diff: Explaining differences amongst similar workflow runs by exploiting scientific metadata

Thavasimani, P; Cala, J; Missier, P

doi:10.1109/BigData.2017.8258275

Why-Diff: Explaining differences amongst similar workflow runs by exploiting scientific metadata

Lookup NU author(s): Priyaa Thavasimani, Dr Jacek Cala ORCiD, Professor Paolo Missier ORCiD

Downloads

Full text for this publication is not currently held within this repository. Alternative links are provided below where available.

Abstract

© 2017 IEEE. Majority of workflows executed nowadays need to process a massive amount of data. Re-execution of such dataintensive scientific workflows often results in different outputs. Scientific research progresses when discoveries are reproduced and verified. However, simply re-enacting a scientific computation, such as a workflow, does not guarantee the correctness of results because of unintentional changes that may have interfered with the re-enactment process. We investigate the hypothesis that the metadata of a workflow execution can be used to explain why the experimenter observes different results (cause analysis). Similarly, Scientific metadata can be used to determine the impact of intentional variations that the experimenter may have injected into a new version of the workflow. We explore these two complementary cases using a simple algorithm for traversing two metadata traces in lock-step mode, which we illustrate through two human genomics data analysis workflows.