A Simple Permutation Test for Clusteredness

• Authors: Michael Greenacre.
• BSE Working Paper: 110043 | September 15
• Keywords: Distance , Hierarchical clustering , permutation test
• JEL codes: C19, C88
• Distance
• Hierarchical clustering
• permutation test

Hierarchical clustering is a popular method for finding structure in multivariate data, resulting in a binary tree constructed on the particular objects of the study, usually sampling units. The user faces the decision where to cut the binary tree in order to determine the number of clusters to interpret and there are various ad hoc rules for arriving at a decision. A simple permutation test is presented that diagnoses whether non-random levels of clustering are present in the set of objects and, if so, indicates the specific level at which the tree can be cut. The test is validated against random matrices to verify the type I error probability and a power study is performed on data sets with known clusteredness to study the type II error.