Skip to Main content Skip to Navigation

Keeping Track of User Steering Actions in Dynamic Workflows

Renan Souza 1 Vítor Silva 1 José Camata 2 Alvaro Coutinho 1 Patrick Valduriez 3 Marta Mattoso 1
3 ZENITH - Scientific Data Management
LIRMM - Laboratoire d'Informatique de Robotique et de Microélectronique de Montpellier, CRISAM - Inria Sophia Antipolis - Méditerranée
Abstract : In long-lasting scientific workflow executions in HPC machines, computational scientists (the users in this work) often need to fine-tune several workflow parameters. These tunings are done through user steering actions that may significantly improve performance (e.g., reduce execution time) or improve the overall results. However, in executions that last for weeks, users can lose track of what has been adapted if the tunings are not properly registered. In this work, we build on provenance data management to address the problem of tracking online parameter fine-tuning in dynamic workflows steered by users. We propose a lightweight solution to capture and manage provenance of the steering actions online with negligible overhead. The resulting provenance database relates tuning data with data for domain, dataflow provenance, execution, and performance, and is available for analysis at runtime. We show how users may get a detailed view of the execution, providing insights to determine when and how to tune. We discuss the applicability of our solution in different domains and validate its ability to allow for online capture and analyses of parameter fine-tunings in a real workflow in the Oil and Gas industry. In this experiment, the user could determine which tuned parameters influenced simulation accuracy and performance. The observed overhead for keeping track of user steering actions at runtime is less than 1% of total execution time.
Document type :
Journal articles
Complete list of metadatas

https://hal-lirmm.ccsd.cnrs.fr/lirmm-02127456
Contributor : Patrick Valduriez <>
Submitted on : Thursday, May 16, 2019 - 4:53:38 PM
Last modification on : Saturday, May 18, 2019 - 1:28:46 AM

File

fgcs-19-preprint-KeepingTrackO...
Files produced by the author(s)

Identifiers

Collections

Citation

Renan Souza, Vítor Silva, José Camata, Alvaro Coutinho, Patrick Valduriez, et al.. Keeping Track of User Steering Actions in Dynamic Workflows. Future Generation Computer Systems, Elsevier, 2019, 99, pp.624-643. ⟨10.1016/j.future.2019.05.011⟩. ⟨lirmm-02127456⟩

Share

Metrics

Record views

57

Files downloads

110