Recognition of logical units in log files

Hassan Saneifar 1 Stéphane Bonniol 1 Pascal Poncelet 2 Mathieu Roche 3, 4
2 ADVANSE - ADVanced Analytics for data SciencE
LIRMM - Laboratoire d'Informatique de Robotique et de Microélectronique de Montpellier
3 TEXTE - Exploration et exploitation de données textuelles
LIRMM - Laboratoire d'Informatique de Robotique et de Microélectronique de Montpellier
Abstract : With the development of new technologies more and more information is stored in log files. Analyzing such logs can be very useful for the decision maker. One of the probably best known example is the Web log file analysis where lots of efficient tools have been proposed to extract the top-k accessed pages, the best users or even the patterns describing the behaviors of users on a Web site. These tools take advantages of the well-formed structures of the data. Unfortunately, logs files from the industrial world have very heterogeneous complex structures (e.g., tables, lists, data blocks). For experts, analyzing logs to find messages helping to better understand causes of a failure, if a problem have already occurred in the past or even knowing the main consequences of a failure is a hard, tedious, time-consuming and error-prone task. There is thus a need for new tools helping the experts to easily recognize the appropriate part in logs. Passage retrieval methods have proved to be very useful for extracting relevant parts in documents. In this paper we propose a new approach for automatically split logs files into relevant segments based on their logical units. We characterize the complex logical units found in logs according to their syntactic characteristics. We also introduce the notion of generalized vs-grams which is used to automatically extract the syntactic characteristics of special structures found in log files. Conducted experiments are performed on real datasets from the industrial world to demonstrate the efficiency of our proposal on the recognition of complex logical units.
Type de document :
Article dans une revue
Intelligent Data Analysis, IOS Press, 2015, 19 (2), pp.431-448
Liste complète des métadonnées
Contributeur : Mathieu Roche <>
Soumis le : samedi 9 août 2014 - 21:46:45
Dernière modification le : jeudi 11 janvier 2018 - 06:27:21


  • HAL Id : lirmm-01054913, version 1


Hassan Saneifar, Stéphane Bonniol, Pascal Poncelet, Mathieu Roche. Recognition of logical units in log files. Intelligent Data Analysis, IOS Press, 2015, 19 (2), pp.431-448. 〈lirmm-01054913〉



Consultations de la notice