External Oscillator charging RST Cap when chip is not powered on
From your description, I assume you have negative reset logic, where LOW level means "reset". In that case, tie the reset pin to 5V via a resistor. This will sink any residual voltage the pin may pick through the oscillator circuit while the 5V voltage is off. When the 5V voltage appears, it will drive your reset pin HIGH, enabling the chip to start.