Use RMS or peak value of current for transformer selection
There are several considerations to take into account, the two main ones being transformer heating, and output voltage regulation.

For transformer heating, you're interested in the RMS current. You could compute the RMS of the current waveform drawn, or you could use the mean DC output as a 'correct ballpark but likely under-estimate' estimate.

Transformers specifications will usually give you an amount by which to derate their rated current for rectifier/capacitor loads. Here is a guide from a random transformer manufacturer giving typical formulae for transformer current rating versus the type of load. They recommend derating to 62% for a rectifier/capacitor load.

For voltage regulation, you're interested in the peak current, as this is what reduces the voltage the reservoir capacitor charges to. However the difference in the effect between rms and peak is usually minor compared to the poor total regulation of small transformers, so a detail not worth worrying about.