Polyfuse with Crowbar for Over Voltage
Polyfuses don't oscillate like that. They maintain their high-resistance state (allowing a small current to flow) indefinitely, until the current is interrupted altogether by some other means. You'll be fine.