Measuring cluster similarity across methods.

Abstract

Cluster analysis techniques delineate groupings or categories of observations based on some shared commonality over a set of variables. If such groupings can be formed, their commonality may be investigated to define relationships that may otherwise go undetected given their complexity. However, the cluster analyses are inappropriate unless the results can be replicated. A number of clustering techniques are available, differing mostly in the technical criteria used to judge the similarity of the observations. There is added validity to the cluster structure when different methods produce similar groupings; however, in most cases, different clustering techniques will not produce identical clusters and the extent of cluster similarity becomes an important measure. In this paper the hypergeometric distribution is used to gauge cluster similarity across different methods, providing an appropriate measure of consistency. This measure is used to validate reproducibility of the clusters.