Skip to Main content Skip to Navigation
Conference papers

Algebraic Dataflows for Big Data Analysis

Abstract : Analyzing big data requires the support of dataflows with many activities to extract and explore relevant information from the data. Recent approaches such as Pig Latin propose a high-level language to model such dataflows. However, the dataflow execution is typically delegated to a MapReduce implementation such as Hadoop, which does not follow an algebraic approach, thus it cannot take advantage of the optimization opportunities of PigLatin algebra. In this paper, we propose an approach for big data analysis based on algebraic workflows, which yields optimization and parallel execution of activities and supports user steering using provenance queries. We illustrate how a big data processing dataflow can be modeled using the algebra. Through an experimental evaluation using real datasets and the execution of the dataflow with Chiron, an engine that supports our algebra, we show that our approach yields performance gains of up to 19.6% using algebraic optimizations in the dataflow and up to 39.1% of time saved on a user steering scenario.
Complete list of metadatas
Contributor : Patrick Valduriez <>
Submitted on : Monday, September 9, 2013 - 5:53:12 PM
Last modification on : Monday, October 19, 2020 - 2:34:03 PM




Dias Jonas, Eduardo Ogasawara, Oliveira Daniel De, Fabio Porto, Patrick Valduriez, et al.. Algebraic Dataflows for Big Data Analysis. IEEE BigData, Oct 2013, Santa Clara, United States. pp.6, ⟨10.1109/BigData.2013.6691567⟩. ⟨lirmm-00857221⟩



Record views