Graph and Semantic Analysis Approach for Template Recognition in Large Scale Log Data

Log files generated by software systems can be utilized as a valuable resource in data-driven approaches to improve system health and stability. These files often contain valuable information about runtime execution and to properly monitor them it is necessary to analyze an increasingly large volume of data logs. In this report, a graph mining technique for parsing logs that is source agnostic to the system is presented. This means that the technique can function regardless of the source of the logs, making it more scalable and reusable. This approach differs from existing techniques that rely heavily on domain knowledge and regular expression patterns, as it uses graph models and semantic analysis to detect patterns in the data with minimal user input. This makes it easy to implement in a variety of scenarios where application-based logs may differ significantly. This proposed technique has the potential to improve the observability and reliability of scalable software systems.