ML models: average of all versus average of averages? I have data for 20 different people and am training a model (e.g. a neural network with the same hyperparameters) on the data from each person; so this gives me 20 models.

I chose to use RMSE to assess the performance. However, since the training data is shuffled, the computed RMSE is nondeterministic and so oscillates. So I thought running each model 10 times and averaging the results, i.e. the RMSE's, would give me a better estimate of performance. But this is for a single person/model. How do I combine the performance of everything, i.e. all 20 models, into a single measure?

Run each of the 20 models 10 times for a total of 200 RMSE values, and average all? Or first compute the average for each person, and then the average of these averages?

Maybe a different method is better? The end goal is to compare a couple of models (e.g. NN vs SVM).
It shouldn't matter if you average all of them or take the average of the average for each person. It turns out that it will give you the same number. This is known as the law of total expectation. Essentially, 

E(X)=E(E(X|Y)) 

Here's the wikipedia article for more details:

https://en.wikipedia.org/wiki/Law_of_total_expectation