.png){kind=icon}
返回
Mining association rules on Big Data through MapReduce genetic programming
.png){kind=icon}
PRE
AI
DOI：10.3233/ICA-170555.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Association rule mining is one of the most important tasks to describe raw data. Although many efficient algorithms have been developed to this aim, existing algorithms do not work well on huge volumes of data. The aim of this paper is to propose a new genetic programming algorithm for mining association rules in Big Data. The genetic operators of our proposal have been specifically designed to avoid a growing in the complexity of the solutions without an improvement in their fitness function values. Furthermore, it introduces a repairing operator to improve the convergence. Additionally, to facilitate its application on real world problems a grammar has been included, allowing it to introduce subjective knowledge into the mining process and to reduce the search space. Due to the growing interest in data gathering, a unique implementation of the proposed algorithm is not useful so different implementations (considering different architectures such as RMI, Hadoop and Spark) are required depending on the data size. All these adaptations obtain exactly the same solutions as those of the original algorithm since they only differ on the software architectures. The experimental study considers more than 75 datasets and 14 algorithms and the results reveal that the proposed algorithm obtains excellent results for more than 12 quality measures. The scalability of the proposal is also analyzed by considering the three parallel implementations on high dimensional datasets (3,000 millions of instances) and file sizes up to 800 GB.
Keyword:
Association rules
Big Data
MapReduce
Hadoop
Spark
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
I
IF:
5.3
论文数:
491
被引数:
735
机构
引用论文
An evolutionary algorithm for the discovery of rare class association rules in learning management systems.png)
APPLIED INTELLIGENCE
APPLIED INTELLIGENCEIF3.5
PREAI
分享收藏Progressive preference articulation for decision making in multi-objective optimisation problems
OAAI
OAAI分享收藏Big Data: Tutorial and guidelines on information and process fusion for analytics algorithms with MapReduce大数据: 关于使用MapReduce进行分析算法的信息和流程融合的教程和指南INFORMATION FUSION
INFORMATION FUSIONIF15.5
PREAI
分享收藏
查看更多内容