, np unravel_index ( argmin ( shape minimizer pair sizes np append (sizes sizes [isizes [ ] [ ,:]np array ([ijnp sqrt( [ , ]sizes - ]] update_distances (di,jsizes update distance matr return import scipy cluster hierarchy as np genfromtxt ('clusterdata csv ',delimiter =','read the data agg_cluster (xform the linkage matrix dendrogram (zscipy can produce dendrogram from fcluster function assigns cluster ids to all points based on cl fcluster (zcriterion 'maxclust ' = import matplotlib pyplot as plt plt figure ( plt clf (cols ['red ','green ','blue 'colors [cols[ - for in clplt scatter ( [, [, ,ccolors plt show (note that the distance matrix is initialized with the squared euclidean distanceso that the ward linkage is rescaled by factor of alsonote that the linkage matrix stores the square root of the minimal cluster distances rather than the distances themselves we leave it as an exercise to check that by using these modifications the results agree with the linkage method from scipysee exercise in contrast to the bottom-up (agglomerativeapproach to hierarchical clusteringthe divisive approach starts with one clusterwhich is divided into two clusters that are as "dissimilaras possiblewhich can then be further dividedand so on we can use the same linkage criteria as for agglomerative clustering to divide parent cluster into two child clusters by maximizing the distance between the child clusters although it is natural to try to group together data by separating dissimilar ones as far as possiblethe implementation of this idea tends to scale poorly with the problem is related to the well-known max-cut problemgiven an matrix of positive costs ci ij { }partition the index set { ninto two subsets and such that the total cost across the setsthat isxx jk max-cut problem jj kk is maximal if instead we maximize according to the average distancewe obtain the group average linkage criterion example (divisive clustering via cethe following python code is used to divide small data set (of size into two parts according to maximal group average linkage it uses short cross-entropy algorithm similar to the one presented in example given vector of probabilities {pi }the algorithm generates an matrix of bernoulli random variables with success probability pi for column for each rowthe and divide the index set into two clustersand the corresponding average linkage