1. A method of clustering a plurality of vector representations of speech utterances, by a speech processing device, into multiple clusters associated with multiple speakers of the speech utterances, the method comprising: defining a clustering pattern having multiple clusters, each of the multiple clusters including at least one of the plurality of vector representations; iteratively, (1) evaluating a clustering confidence score in terms of silhouette width criterion values associated with at least a subset of the plurality of vector representations, the clustering confidence score representing a clustering validation metric, and (2) updating the clustering pattern by merging a pair of nearest clusters, among clusters associated with the clustering pattern, into a single cluster, the pair of nearest clusters merged being determined based on a similarity score indicative of similarity between vector representations associated with different clusters; and providing an indication of a final clustering pattern, the final clustering pattern being determined as the clustering pattern corresponding to a highest value of the clustering confidence score.