How does an inductor limit a voltage booster?
For a 12 A load at 5 V you will have an input current of more than 12 A * 5 V = 60 W => 60 W / 3 V = 20 A average current.

As the current isn't continuous the peak currents will be much higher. I didn't take any losses (switch resistance, wire resistance) into account which will be significant due to the high current and low voltage (even a small voltage drop can be significant). This means you'll need an inductor with a very high saturation current as Hearth mentions in a comment. My guess is that you'll need an inductor with a saturation current higher than 30 A.

When the inductor saturates, it behaves as a short and current will shoot up and efficiency drops. The DCDC converter is then unable to deliver more power to the output so the output voltage will drop. Depending on the DCDC converter implementation the chip might detect the high current though the inductor and shut everything off or just "do what it can" meaning you will not get 5 V at the output but some lower voltage. In a converter without overcurrent protection it is possible that the switch will be damaged.
Overheating of the inductor is indeed also a possibility.

For a proper design, you simply want to avoid the inductor from saturating.

For your design goal with such a low input voltage but a high output current (meaning very a high input current) it will be challenging to make a good design as it will be very difficult to keep all the series resistances small enough. Combined with a "small footprint" it might be impossible (depends on what you call small).

Try to find a commercial product with a similar specification, if you cannot find any then you now know why.