Practical limits on Raspberry Pi GPIO pin voltages
The only safe answer is to do only what the datasheet says is allowed.

That doesn't mean that abusing the limits always causes the device to operate incorrectly or to damage it.  It only means these things might happen, and the manufacturer is off the hook if they do.

Probably what happend in your case is that the protection diodes of the pins in the processor were forward biased.  Let's say the went to 600 mV forward drop, so 3.9 V relative to ground.  That means the current thru them was (1.1 V)/(3 k&Omega;) = 370 &micro;A.  That could be bad, depending on what exactly the datasheet says about it.

For some devices, unexpected currents like that can cause them to crowbar on power up.  For others, they will mostly work OK, but some things, particularly anything analog, gets flaky or out of spec.  For others, they continue to operate fine.

When you violate datasheet parameters you don't know what you get.  Asking what you can get away with is the wrong question and just plain irresponsible anyway.  We are not here to help you practise irresponsible electrical engineering.

In your particular case, just connect the pullups to 3.3 V.  The IIC voltage thresholds are specified so that this should work.  Check your slave devices to make sure that 3.3 V is solidly above the minimum guaranteed logic high level.