Purpose of capacitor across transformer in AC-DC power supply?
Yes the yellow thing is a transformer, since it is used at a much higher frequency than 50 or 60 Hz mains, it can be much smaller.

Isolation is indeed very important. A user must be able to safely touch the secondary (5V) side without feeling a thing.

The optocoupler senses the output voltage and feeds back a signal that tells the input (mains voltage) side if the output voltage is too high or too low. This is called feedback.

C5 is required ! It is needed to provide a path for high frequency signals caused by the switching of the input side DC voltage. This switching is what makes the DC at the input side into AC, which is transformed to a lower voltage and then rectified again and smoothed by a capacitor. Note that C5 needs to have a very high voltage rating. Imagine what would happen if C5 broke, became a short and then you touched the output.

The black transformer at the input is a common-mode suppressor. It attenuates (filters) unwanted signals generated by this supply and tries to not let them escape from the mains side. Without this common-mode suppressor the supply will still work but it might not meet EMI requirements.

Important last remark: Do NOT buy the cheaper one without the common-mode choke and the worse isolation !

Looking at it, I think it has no isolation at all ! It is not suitable for making a safe 5 V supply from mains voltage. It would only be OK-ish in an isolated box where it is impossible to touch any the wires and which has no connections a user can touch.