.png){kind=icon}
返回
Random forest implementation and optimization for Big Data analytics on LexisNexis's high performance computing cluster platform
.png){kind=icon}
OA
AI
DOI：10.1186/s40537-019-0232-1.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
In this paper, we comprehensively explain how we built a novel implementation of the Random Forest algorithm on the High Performance Computing Cluster (HPCC) Systems Platform from LexisNexis. The algorithm was previously unavailable on that platform. Random Forest's learning process is based on the principle of recursive partitioning and although recursion per se is not allowed in ECL (HPCC's programming language), we were able to implement the recursive partition algorithm as an iterative split/partition process. In addition, we analyze the flaws found in our initial implementation and we thoroughly describe all the modifications required to overcome the bottleneck within the iterative split/partition process, i.e., the optimization of the data gathering of selected independent variables which are used for the node's best-split analysis. Essentially, we describe how our initial Random Forest implementation has been optimized and has become an efficient distributed machine learning implementation for Big Data. By taking full advantage of the HPCC Systems Platform's Big Data processing and analytics capabilities, we succeed in enhancing the data gathering method from an inefficient Pass them All and Filter approach into an effective and completely parallelized Fetching on Demand approach. Finally, based upon the results of our learning process runtime comparison between these two approaches, we confirm the speed up of our optimized Random Forest implementation.
Keyword:
Random forest
LexisNexis's high performance computing cluster (HPCC) systems platform
Optimization for Big Data
Distributed machine learning
Turning recursion into iteration
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
6.4
论文数:
1.5K
被引数:
1.1W
机构
引用论文
Adaptive Impedance Control to Enhance Human Skill on a Haptic Interface System自适应阻抗控制以增强触觉接口系统上的人类技能OAAI
.png){kind=icon}
OAAI分享收藏An Introduction to Recursive Partitioning: Rationale, Application, and Characteristics of Classification and Regression Trees, Bagging, and Random Forests递归分区简介: 分类和回归树，Bagging和随机森林的原理，应用和特征OAAI
OAAI分享收藏分享收藏
查看更多内容