.png){kind=icon}
返回
Optimal classification trees
.png){kind=icon}
OA
AI
DOI：10.1007/s10994-017-5633-9.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
State-of-the-art decision tree methods apply heuristics recursively to create each split in isolation, which may not capture well the underlying characteristics of the dataset. The optimal decision tree problem attempts to resolve this by creating the entire decision tree at once to achieve global optimality. In the last 25 years, algorithmic advances in integer optimization coupled with hardware improvements have resulted in an astonishing 800 billion factor speedup in mixed-integer optimization (MIO). Motivated by this speedup, we present optimal classification trees, a novel formulation of the decision tree problem using modern MIO techniques that yields the optimal decision tree for axes-aligned splits. We also show the richness of this MIO formulation by adapting it to give optimal classification trees with hyperplanes that generates optimal decision trees with multivariate splits. Synthetic tests demonstrate that these methods recover the true decision tree more closely than heuristics, refuting the notion that optimal methods overfit the training data. We comprehensively benchmark these methods on a sample of 53 datasets from the UCI machine learning repository. We establish that these MIO methods are practically solvable on real-world datasets with sizes in the 1000s, and give average absolute improvements in out-of-sample accuracy over CART of 1-2 and 3-5% for the univariate and multivariate cases, respectively. Furthermore, we identify that optimal classification trees are likely to outperform CART by 1.2-1.3% in situations where the CART accuracy is high and we have sufficient training data, while the multivariate version outperforms CART by 4-7% when the CART accuracy or dimension of the dataset is low.
Keyword:
Classification
Decision trees
Mixed integer optimization
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
2.9
论文数:
2.7K
被引数:
3.4W
机构
暂无机构信息
引用论文
.png){kind=icon}
分享收藏分享收藏Sustained increase in hippocampal sharp-wave ripple activity during slow-wave sleep after learningOAAI
OAAI分享收藏
查看更多内容