How am I supposed to interpret transformer value only given in impedance?
Is this value for the primary side or secondary side?

I believe that is the impedance per phase looking into the transformer from the primary side with the secondary short circuited. It is the primary impedance plus the secondary impedance referred to the primary for the per phase equivalent circuit.

How would I substitute the per unit base for it?

The 1 per unit base impedance is the load impedance with the same L/R that would draw rated current referred to the primary.