Brief Startup Voltage in DC RL Circuit
You are correct, the most likely cause is the non-ideal behaviour of ...

the switch.

Mechanical switches don't deliver a clean On/Off behaviour, they make and break contact for (typically) a few milliseconds  before they settle to a final state. Age, wear, and dirt/oxidation on the contacts make the problem worse.

This is called switch bounce and you'll find many articles describing it.

It's usually encountered when controlling digital circuits with a push button ("why did pushing the button once count to 17?") and there are whole families of switch de-bouncing circuits and software routines, as in the linked article from Maxim.

But it can also be used to generate interesting unexpected voltages with inductive circuits, as you found.