Given MFCC values for some wav’s, how can I easily tell if all speakers are the same?
Given MFCC values for some wav’s, how can I easily tell if all speakers are the same?


In general, you would deal with this kind of problem using some form of Clustering.

The fundamental premise behind this idea is that the features of similar data items would have similar values.

How you assess similarity, how you assess "a cluster" and what this means for the particular application you are dealing with are very much context specific issues.

In this particular example, the features are the MFCC coefficients. If these are similar between two frames we would tend to think that the frames contain more or less signals produced from the same source.

The simplest clustering to apply would be k-means clustering but this is not going to help too much here, especially if you don't know how many speakers you might get in a recording. So, long story short, you will most likely gravitate to something like hierarchical clustering.

You don't have to extract the means of each feature to cluster them. Doing so would be equivalent to training and using a minimum distance classifier. The only thing that is needed here to complete that is the "ground truth" knowledge that will tell you which centroids belong to which speakers.

Note here that if you have two speakers talking in a mostly silent room, the "feature space" of the clustering is not necessarily expected to be divided into two distinct "point clouds". It could, for example, contain separate regions for background silence, one speaker talking, the other speaker talking, both speakers talking at the same time. This last class might contain further grading depending on how loudly was each speaker talking while interrupting the other. 

Hierarchical clustering allows you to focus on details with varying degrees of "attention", provided that the MFCC clusters the feature space with sufficient accuracy.

The other thing you can do, is a Principal Component Analysis on the MFCC matrix which (hopefully) will return one component for each one of these "major events" happening in the file.

Hope this helps.

EDIT:
Forgot to add, if you are looking for the terminology that could return to you some useful results in varying levels of complexity, this is "Speaker Diarisation". See for example here.