A Hybrid Memory Data Cube Approach for High Dimension Relations

Rodrigo Rocha Silva, Celso Massaki Hirata, Joubert de Castro Lima

2015

Abstract

Approaches based on inverted indexes, such as Frag-Cubing, are considered efficient in terms of runtime and main memory usage for high dimension cube computation and query. These approaches do not compute all aggregations a priori. They index information about occurrences of attributes in a manner that it is time efficient to answer multidimensional queries. As any other main memory based cube solution, Frag-Cubing is limited to main memory available, thus if the size of the cube exceeds main memory capacity, external memory is required. The challenge of using external memory is to define criteria to select which fragments of the cube should be in main memory. In this paper, we implement and test an approach that is an extension of Frag-Cubing, named H-Frag, which selects fragments of the cube, according to attribute frequencies and dimension cardinalities, to be stored in main memory. In our experiment, H-Frag outperforms Frag-Cubing in both query response time and main memory usage. A massive cube with 60 dimensions and 109 tuples was computed by H-Frag sequentially using 110 GB of RAM and 286 GB of external memory, taking 64 hours. This data cube answers complex queries in less than 40 seconds. Frag- Cubing could not compute such a cube in the same machine.

Paper Citation

in Harvard Style

Silva R., Hirata C. and Lima J. (2015). A Hybrid Memory Data Cube Approach for High Dimension Relations . In Proceedings of the 17th International Conference on Enterprise Information Systems - Volume 1: ICEIS, ISBN 978-989-758-096-3, pages 139-149. DOI: 10.5220/0005371601390149

in Bibtex Style

@conference{iceis15,
author={Rodrigo Rocha Silva and Celso Massaki Hirata and Joubert de Castro Lima},
title={A Hybrid Memory Data Cube Approach for High Dimension Relations},
booktitle={Proceedings of the 17th International Conference on Enterprise Information Systems - Volume 1: ICEIS,},
year={2015},
pages={139-149},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005371601390149},
isbn={978-989-758-096-3},
}

in EndNote Style

TY - CONF
JO - Proceedings of the 17th International Conference on Enterprise Information Systems - Volume 1: ICEIS,
TI - A Hybrid Memory Data Cube Approach for High Dimension Relations
SN - 978-989-758-096-3
AU - Silva R.
AU - Hirata C.
AU - Lima J.
PY - 2015
SP - 139
EP - 149
DO - 10.5220/0005371601390149