返回
p-Values for High-Dimensional Regression
DOI:10.1198/jasa.2009.tm08647.png)
摘要
En 中文
Assigning significance in high-dimensional regression is challenging. Most computationally efficient selection algorithms cannot guard against inclusion of noise variables. Asymptotically valid p-values are not available. An exception is a recent proposal by Wasserman and Roeder that splits the data into two parts. The number of variables is then reduced to a manageable size using the first split, while classical variable selection techniques can be applied to the remaining variables, using the data from the second split. This yields asymptotic error control under minimal conditions. This involves a one-time random split of the data, however. Results are sensitive to this arbitrary choice, which amounts to a p-value lottery and makes it difficult to reproduce results. Here we show that inference across multiple random splits can be aggregated while maintaining asymptotic control over the inclusion of noise variables. We show that the resulting p-values can be used for control of both family-wise error and false discovery rate. In addition, the proposed aggregation is shown to improve power while reducing the number of falsely selected variables substantially.
Keyword:
Data splitting
False discovery rate
Family-wise error rate
High-dimensional variable selection
Multiple comparisons
期刊
J
IF:
3
论文数:
5.2K
被引数:
4.8W
机构
引用论文
Compression and Air Storage Systems for Small Size CAES Plants: Design and Off-design Analysis小型CAES工厂的压缩和空气存储系统: 设计和非设计分析
LASSO-TYPE RECOVERY OF SPARSE REPRESENTATIONS FOR HIGH-DIMENSIONAL DATA高维数据稀疏表示的LASSO型恢复
ANNALS OF STATISTICS
IF3.7

