How to connect the PHY crystal for RMII?
Figure 3-4 shows an oscillator, not a crystal.
Similarly, the datasheet says "RMII - 50 MHZ CLOCK MODE [...] An external 50 MHz clock source (oscillator) connected to XI"

A "crystal" would be excited by XO and the desired clock frequency would come into XI. In this context, an "oscillator" is something which excites itself externally, and just provides a clock to your device's XI or REF_CLK input.

The datasheet is trying to tell you to use some external device which provides a 50MHz clock without needing to be excited by XO.