Online Input Data Reduction in Scientific Workflows - Archive ouverte HAL Access content directly
Conference Papers Year : 2016

Online Input Data Reduction in Scientific Workflows

(1) , (1) , (1) , (2, 3) , (1)
1
2
3

Abstract

Many scientific workflows are data-intensive and need be iteratively executed for large input sets of data elements. Reducing input data is a powerful way to reduce overall execution time in such workflows. When this is accomplished online (i.e., without requiring users to stop execution to reduce the data and resume execution), it can save much time and user interactions can integrate within workflow execution. Then, a major problem is to determine which subset of the input data should be removed. Other related problems include guaranteeing that the workflow system will maintain execution and data consistent after reduction, and keeping track of how users interacted with execution. In this paper, we adopt the approach " human-in-the-loop " for scientific workflows by enabling users to steer the workflow execution and reduce input elements from datasets at runtime. We propose an adaptive monitoring approach that combines workflow provenance monitoring and computational steering to support users in analyzing the evolution of key parameters and determining which subset of the data should be removed. We also extend a provenance data model to keep track of user interactions when users reduce data at runtime. In our experimental validation, we develop a test case from the oil and gas industry, using a 936-cores cluster. The results on our parameter sweep test case show that the user interactions for online data reduction yield a 37% reduction of execution time.
Fichier principal
Vignette du fichier
WORKS 2016.pdf (3.84 Mo) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

lirmm-01400538 , version 1 (22-11-2016)

Identifiers

  • HAL Id : lirmm-01400538 , version 1

Cite

Renan Souza, Vítor Silva, Alvaro L. G. A. Coutinho, Patrick Valduriez, Marta Mattoso. Online Input Data Reduction in Scientific Workflows. WORKS: Workflows in Support of Large-scale Science, Nov 2016, Salt Lake City, United States. ⟨lirmm-01400538⟩
380 View
321 Download

Share

Gmail Facebook Twitter LinkedIn More