Database System Support of Simulation Data

Hermano Lustosa 1 Fabio Porto 1 Pablo Blanco 1 Patrick Valduriez 2, 3
3 ZENITH - Scientific Data Management
LIRMM - Laboratoire d'Informatique de Robotique et de Microélectronique de Montpellier, CRISAM - Inria Sophia Antipolis - Méditerranée
Abstract : Supported by increasingly efficient HPC infrastructure , numerical simulations are rapidly expanding to fields such as oil and gas, medicine and meteorology. As simulations become more precise and cover longer periods of time, they may produce files with terabytes of data that need to be efficiently analyzed. In this paper, we investigate techniques for managing such data using an array DBMS. We take advantage of multidimensional arrays that nicely models the dimensions and variables used in numerical simulations. However , a naive approach to map simulation data files may lead to sparse arrays, impacting query response time, in particular, when the simulation uses irregular meshes to model its physical domain. We propose efficient techniques to map coordinate values in numerical simulations to evenly distributed cells in array chunks with the use of equi-depth his-tograms and space-filling curves. We implemented our techniques in SciDB and, through experiments over real-world data, compared them with two other approaches: row-store and column-store DBMS. The results indicate that multidi-mensional arrays and column-stores are much faster than a traditional row-store system for queries over a larger amount of simulation data. They also help identifying the scenarios where array DBMSs are most efficient, and those where they are outperformed by column-stores.
Type de document :
Article dans une revue
Proceedings of the VLDB Endowment (PVLDB), VLDB Endowment, 2016, 9 (13), pp.1329-1340. <http://www.vldb.org/pvldb/vol9/p1329-lustosa.pdf>
Liste complète des métadonnées


https://hal-lirmm.ccsd.cnrs.fr/lirmm-01363738
Contributeur : Patrick Valduriez <>
Soumis le : dimanche 11 septembre 2016 - 16:21:12
Dernière modification le : vendredi 9 juin 2017 - 10:43:08
Document(s) archivé(s) le : lundi 12 décembre 2016 - 12:16:36

Fichier

vldb2016.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : lirmm-01363738, version 1

Collections

Citation

Hermano Lustosa, Fabio Porto, Pablo Blanco, Patrick Valduriez. Database System Support of Simulation Data. Proceedings of the VLDB Endowment (PVLDB), VLDB Endowment, 2016, 9 (13), pp.1329-1340. <http://www.vldb.org/pvldb/vol9/p1329-lustosa.pdf>. <lirmm-01363738>

Partager

Métriques

Consultations de
la notice

150

Téléchargements du document

228