PIC32 external oscillation
According to a datasheet, the REGISTER 4-7 comes up with all 1's at power-up, which disables the primary oscillator, so the answer should be no, assuming you are referring to the primary oscillator. 

Execution proceeds clocked by the internal RC oscillator until/if another clock source is configured. If the flash memory has been erased, that won't happen.