When a current flow in an inductor is interrupted, what limits the voltage rise?
TL;DR: It also depends on the actual setup.

It can be useful to see the problem from an energy point of view. Since the energy stored in the inductor is \$E={1 \over 2} L I^2\$, when you "instantly" bring the current to zero that energy has to go somewhere. It cannot vanish.

If you model the current interruption as an extremely rapid increase of resistance at the point of the circuit where the switching element is placed, what you get is that the voltage across the inductor begins to raise rapidly. When does it stop raising? It depends on the inductor setup and its surrounding environment.

Marko Bursic has already mentioned self capacitance in his answer. 

You say "assuming extremely good insulation", but "extremely good" is still "not perfect". There is resistance and capacitance toward anything surrounding the inductor. You can get an increase of leakage currents due to voltage increase or displacement current into nearby object due to any tiny parasitic capacitance, whose "reactance" becomes low when the rise time is small.

For example If you have a regular, mechanical switch, you may get an arc between the contacts that dissipates the energy into heat, ionization of the air and generation of EM waves.

At high enough voltages, electrons can be extracted from the metal and an electric arc can be formed in vacuum, too (there are also switches that operate using a vacuum arc).

BTW, unless you do want to get high voltage spikes, that's why you put overvoltage protection devices or snubber circuits in parallel to inductors whose current may be interrupted. So, in practice it is you (the circuit designer) that wants to limit the voltage rise. 

In a sense, controlled generation of inductor overvoltage spikes is what is done in some step-up DC-DC converters: you "interrupt" the current in an inductor in order to get an increase in voltage, then you "dump" the increased voltage into an output capacitor to "store" it for the load. Of course the switching controller is key to useful operation.

On a related note: it can come as a surprise, but vacuum has not "infinite resistance", i.e. it doesn't impede the flow of current at all! It simply has no free charges to support currents. Once a charge is extracted from a nearby object things change dramatically. This is explained in detail in this article by Charles Chandler. Excerpts (emphasis mine):


  The answer is that the vacuum doesn't impede the electrons at all.
  In a perfect vacuum (except for the test charge), the charged
  particle's behavior can be calculated by just three factors: 1) its
  mass, which gives it inertial forces, 2) its electric charge, which
  makes it responsive to an electric field, and 3) the electric field
  acting on the charged particle. Then the acceleration of the particle
  is just the equilibrium between the inertial and electric forces.


[...]


  This has been mistaken to be a measure of resistance, which seems to
  become infinite at low pressures, but this is not correct. At higher
  pressures, the breakdown voltage increases steadily with pressure (off
  the right side of Figure 2), and so does the resistance. So there is a
  direct relationship between breakdown voltage and resistance in that
  range. If we erroneously take that as a hard and fast rule, and we
  observe the breakdown voltage shooting up at very low pressures, we
  conclude that the resistance must be increasing at very low pressures,
  asymptotically approaching infinite resistance at a pressure
  characteristic for that gas. But here we have to remember that Paschen
  was studying breakdown voltages, and made no mention of resistance. If
  we make direct measurements of the resistance, we find that it varies
  directly with the pressure, and continues straight down to nothing at
  no pressure, without the sudden deviation at the threshold discovered
  by Paschen. This can be confirmed by putting an ammeter into the
  circuit, and finding the resistance by the volts divided by the amps.


So an inductor placed in an "ideal" vacuum could have its terminal voltage increase enormously, but once the vacuum breaks down, because even a small quantity of electrons is extracted from nearby objects (e.g. the terminals of the inductor itself, for example), a large current may be produced in the vacuum (hence an arc).

That's enough for the concept: the rest boils down of how good are the insulators surrounding the inductor and how big is the extraction potential of electrons from the materials on which the inductor voltage spike is applied.

Some interesting data on this point can be found in this lengthy slide collection from NASA site (it is a set of presentations by several authors); the first part is: High Voltage Engineering Techniques For Space Applications by Steven Battel

In slide #29 (page 25 of the PDF) you get (emphasis mine):


  Intrinsic Dielectric Strength Limits
  
  
  The breakdown of Gasses at STP is dependent on type.
  
  
  Air : ~3 kV/mm (75 V/mil)
  He : ~0.37 kV/mm (9.3 V/mil)
  SF 6 : ~9 kV/mm (222 V/mil)
  
  High Vacuum breakdown is surface and configuration
  dependent but is in the range of 20 to 40 kV/mm (500 to 1000
  V/mil)
  Ignoring surface effects, Liquids and Solids bulk properties are
  generally similar in the same range of 15 to 20 kV/mm (375
  V/mil to 500 V/mil).
  


So here you go: even in vacuum the best you can get is a dielectric strength of 40kV/mm. Can this lead to MV voltages? It depends on the setup, but probably something will break down before that happens.