Algebraic Dataflows for Big Data Analysis - LIRMM - Laboratoire d’Informatique, de Robotique et de Microélectronique de Montpellier Access content directly
Conference Papers Year : 2013

Algebraic Dataflows for Big Data Analysis


Analyzing big data requires the support of dataflows with many activities to extract and explore relevant information from the data. Recent approaches such as Pig Latin propose a high-level language to model such dataflows. However, the dataflow execution is typically delegated to a MapReduce implementation such as Hadoop, which does not follow an algebraic approach, thus it cannot take advantage of the optimization opportunities of PigLatin algebra. In this paper, we propose an approach for big data analysis based on algebraic workflows, which yields optimization and parallel execution of activities and supports user steering using provenance queries. We illustrate how a big data processing dataflow can be modeled using the algebra. Through an experimental evaluation using real datasets and the execution of the dataflow with Chiron, an engine that supports our algebra, we show that our approach yields performance gains of up to 19.6% using algebraic optimizations in the dataflow and up to 39.1% of time saved on a user steering scenario.
No file

Dates and versions

lirmm-00857221 , version 1 (09-09-2013)



Dias Jonas, Eduardo Ogasawara, Oliveira Daniel De, Fabio Porto, Patrick Valduriez, et al.. Algebraic Dataflows for Big Data Analysis. IEEE BigData, Oct 2013, Santa Clara, United States. pp.6, ⟨10.1109/BigData.2013.6691567⟩. ⟨lirmm-00857221⟩
532 View
0 Download



Gmail Facebook X LinkedIn More