Communicate with unpowered CMOS
You should not assume that it's safe. Input protection diodes may cause the IC to become powered when inputs are applied.

If you switch to a device family that supports "level 1 live insertion", what Texas Instruments calls "Ioff", then you can apply logic levels to the inputs since they will be high-impedance when VCC is not applied. TI logic families that support Ioff include (but are not limited to) LVC, LVT, and AVC.

Unfortunately the 4021 does not come in any Ioff varieties, and I have been unable to find any other CMOS shift registers that support it. You will need to put another IC in front of it that either supports Ioff or can tri-state the connection if you want to be able remove power from the shift register.