What is the reasoning behind Barkhausen's criteria?
Yes - in principle, you are right...the CLOSED-LOOP gain (effective for an external signal) would be, theoretically, infinite. However, it is not the purpose of such a circuit to amplify any external signal. 

The oscillator principle is that it produces the required input signal for the amplifier itself by compensating the gain factor for the attenuation in the feedback circuit. This is equivalent to a unity loop gain.

However, due to tolerances and some other uncertainties within each electronic circuit, it is not possible to designa circuit having a loop gain of exactly "1".
Hence, we design it for a slightly larger loop gain and use an extra gain control (diodes, thermistor, FET as a resitor,..) which automatically brings the loop gain back to unity as soon as a certain output is available. (In reality, due to the time constant of the regulation mechanism, the loop gain swings around the nominal value of unity).     

Comment: In many publications it is not mentioned that the Barkhausen criterion is only a necessary one. That means: It is not a sufficient criterion ....there are circuits which have unity loop gain for one single frequency only - but the circuit does not oscillate.