Journal
JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION
Volume 103, Issue 483, Pages 1281-1293Publisher
AMER STATISTICAL ASSOC
DOI: 10.1198/016214508000000454
Keywords
Clustering; High-dimension low-sample data; k-means; Microarray gene expression data; p value; Statistical significance
Categories
Funding
- National Science Foundation (NSF) [DMS 0747575, DMS 0606577]
- National Institutes of Health [K12 RR023248]
- U.S. Environmental Protection Agency [RD-83272001]
Ask authors/readers for more resources
Clustering methods provide a powerful tool for the exploratory analysis of high-dimension, low-sample size (HDLSS) data sets, such as gene expression microarray data. A fundamental statistical issue in clustering is which clusters are ''really there'', as opposed to being artifacts of the natural sampling variation. We propose SigClust as a simple and natural approach to this fundamental statistical problem. In particular, we define a cluster as data coming from a single Gaussian distribution and formulate the problem of assessing statistical significance of clustering as a testing procedure. This Gaussian null assumption allows direct formulation of p values that effectively quantify the significance of a given clustering. HDLSS covariance estimation for SigClust is achieved by a combination of invariance principles, together with a factor analysis model. The properties of SigClust are studied. Simulated examples, as well as an application to a real cancer microarray data set, show that the proposed method works remarkably well for assessing significance of clustering. Some theoretical results also are obtained.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available