arrow
Return

On some graph-based two-sample tests for high dimension, low sample size data

delete2019-11-13
delete9
delete
OA
AI
S
Soham Sarkar *
R
Rahul Biswas
A
Anil K. Ghosh
DOI:10.1007/s10994-019-05857-4delete
deleteOriginal
deleteShare
deleteSave
View PDF
Abstract

Abstract

En 中文
Testing for equality of two high-dimensional distributions is a challenging problem, and this becomes even more challenging when the sample size is small. Over the last few decades, several graph-based two-sample tests have been proposed in the literature, which can be used for data of arbitrary dimensions. Most of these test statistics are computed using pairwise Euclidean distances among the observations. But, due to concentration of pairwise Euclidean distances, these tests have poor performance in many high-dimensional problems. Some of them can have powers even below the nominal level when the scale-difference between two distributions dominates the location-difference. To overcome these limitations, we introduce some new dissimilarity indices and use them to modify some popular graph-based tests. These modified tests use the distance concentration phenomenon to their advantage, and as a result, they outperform the corresponding tests based on the Euclidean distance in a wide variety of examples. We establish the high-dimensional consistency of these modified tests under fairly general conditions. Analyzing several simulated as well as real data sets, we demonstrate their usefulness in high dimension, low sample size situations.
Keywords:
Distance concentration
High-dimensional consistency
Minimum spanning tree
Nearest neighbor
Non-bipartite matching
Permutation test
Shortest Hamiltonian path
AI Summary

AI Summary

Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.

Journal

Machine Learning cover
Machine Learning
IF:
2.9
Papers:
2.6K
Citations:
3.4W

Organization

U
University of Washington
Scholars:
8.0W
Papers: 7.0W
Citations: 12.5W
E
Ecole Polytechnique Federale de Lausanne
Scholars:
1.7W
Papers: 1.3W
Citations: 25
S
swiss federal institutes of technology domain
Scholars:
9.0W
Papers: 8.0W
Citations: 163
researcher View more organizations