Online Input Data Reduction in Scientific Workflows - LIRMM - Laboratoire d’Informatique, de Robotique et de Microélectronique de Montpellier Access content directly
Conference Papers Year : 2016

Online Input Data Reduction in Scientific Workflows


Many scientific workflows are data-intensive and need be iteratively executed for large input sets of data elements. Reducing input data is a powerful way to reduce overall execution time in such workflows. When this is accomplished online (i.e., without requiring users to stop execution to reduce the data and resume execution), it can save much time and user interactions can integrate within workflow execution. Then, a major problem is to determine which subset of the input data should be removed. Other related problems include guaranteeing that the workflow system will maintain execution and data consistent after reduction, and keeping track of how users interacted with execution. In this paper, we adopt the approach " human-in-the-loop " for scientific workflows by enabling users to steer the workflow execution and reduce input elements from datasets at runtime. We propose an adaptive monitoring approach that combines workflow provenance monitoring and computational steering to support users in analyzing the evolution of key parameters and determining which subset of the data should be removed. We also extend a provenance data model to keep track of user interactions when users reduce data at runtime. In our experimental validation, we develop a test case from the oil and gas industry, using a 936-cores cluster. The results on our parameter sweep test case show that the user interactions for online data reduction yield a 37% reduction of execution time.
Fichier principal
Vignette du fichier
WORKS 2016.pdf (3.84 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

lirmm-01400538 , version 1 (22-11-2016)


  • HAL Id : lirmm-01400538 , version 1


Renan Souza, Vítor Silva, Alvaro L. G. A. Coutinho, Patrick Valduriez, Marta Mattoso. Online Input Data Reduction in Scientific Workflows. WORKS: Workflows in Support of Large-scale Science, Nov 2016, Salt Lake City, United States. ⟨lirmm-01400538⟩
392 View
363 Download


Gmail Facebook X LinkedIn More