1. A method of clustering a plurality of vector representations of speech utterances, by a speech processing device, into multiple clusters associated with multiple respective speakers of the speech utterances, the method comprising: defining a clustering pattern having multiple clusters, each of the multiple clusters including at least one of the plurality of vector representations; iteratively, (1) evaluating a clustering confidence score in terms of silhouette width criterion values associated with at least a subset of the plurality of vector representations, the clustering confidence score representing a clustering validation metric, and (2) updating the clustering pattern by merging a pair of nearest clusters, among clusters associated with the clustering pattern, into a single cluster, the pair of nearest clusters merged being determined based on a similarity score indicative of similarity between vector representations associated with different clusters; employing a modified silhouette width criterion function with a penalty term, the penalty term regularizing the silhouette width criterion values, the modified silhouette width criterion function enabling a stopping condition for the evaluating and updating; and providing an indication of a final clustering pattern to a speaker recognition engine, the final clustering pattern being determined as the clustering pattern corresponding to a highest value of the clustering confidence score, the final clustering pattern employed by the speaker recognition engine in associating speech utterances from at least one speaker of the multiple respective speakers with the at least one speaker.