.png){kind=icon}
返回
Leakage in Data Mining: Formulation, Detection, and Avoidance
.png){kind=icon}
PRE
AI
DOI：10.1145/2382577.2382579.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Deemed one of the top ten data mining mistakes, leakage is the introduction of information about the data mining target that should not be legitimately available to mine from. In addition to our own industry experience with real-life projects, controversies around several major public data mining competitions held recently such as the INFORMS 2010 Data Mining Challenge and the IJCNN 2011 Social Network Challenge are evidence that this issue is as relevant today as it has ever been. While acknowledging the importance and prevalence of leakage in both synthetic competitions and real-life data mining projects, existing literature has largely left this idea unexplored. What little has been said turns out not to be broad enough to cover more complex cases of leakage, such as those where the classical independently and identically distributed (i.i.d.) assumption is violated, that have been recently documented. In our new approach, these cases and others are explained by explicitly defining modeling goals and analyzing the broader framework of the data mining problem. The resulting definition enables us to derive general methodology for dealing with the issue. We show that it is possible to avoid leakage with a simple specific approach to data management followed by what we call a learn-predict separation, and present several ways of detecting leakage when the modeler has no control over how the data have been collected. We also offer an alternative point of view on leakage that is based on causal graph modeling concepts.
Keyword:
Data mining
leakage
predictive modeling
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
4.8
论文数:
1.3K
被引数:
4.4K
机构
引用论文
.png){kind=icon}
Phenology and seasonal above-ground biomass in twoScirpus maritimus marshes in the Camargue两种Scirpus maritimus沼泽地的物候学和季节性地上生物量在Camargue地区
PREAI
分享收藏分享收藏Barrier height for the abstraction reaction of methylene with hydrogen, CH2(3B1) + H2 → CH3 + H
PREAI
分享收藏
查看更多内容