Why is the only way to make this device work to manually unplug and re-plug the supply wire while the master is on?
There are multiple voltages going into that chip (analog + digital supplies) so DKNguyen's answer sounds good.  The design of the WROVER may not be performing the power-up sequence correctly.
You should be able to give the ADXL355 Accel chip a software reset (write 0x52 to 0x2F) which is 'similar' to a Power-On-Reset.  This is buggy and the datasheet specifies:


Read the shadow registers, Register 0x50 to Register 0x54 (five 8-bit registers) after power-up, but before any software reset.
Store these values in a host device (for example, a host microprocessor).
After each software reset, read the same five registers. If the values differ, perform a software reset again until they match.


So on boot, read the five registers, set 0x2F to 0x52, delay a while and a half, then read the 5 again and make sure they match, and repeat up to X times if they don't.