Distributed Association Rule Mining on Batchwise Data

Abstract: With the growing amount of data generated from millions of transactions taking place every second all over the world, it has become increasingly necessary to find interesting patterns from this data. Multinational companies, being spread over the globe, have to integrate data from various geographically dispersed sites. This data is generated from varied sources in heterogeneous forms and has to be processed before mining can be carried out on it. Mining association rules from data involves finding correlations between two or more variables in a dataset. Algorithms like Fast Distributed Mining (FDM) and Count Distribution Algorithm (CDA) have been used for association rule mining in a distributed environment. However, these algorithms prove to be inefficient when it comes to dynamically streaming input. Our proposed solution suggests a methodology to implement Association Rule Mining on Distributed Systems using ODAM (Optimized Distributed Association Rule Mining) algorithm on batchwise data. Further, in an effort to aid the user in analysing the output, we display the association rules generated for each item as well as for the entire dataset. Also, the user can view the time series analysis for each association rule.

Keywords: ODAM, Batchwise Data, Apriori, Distributed Association Rule Mining.