DC Voltage stabiliser
You need a buck-boost DC-DC converter (because your input voltage range is both above and below your output voltage). You can buy an existing converter or build your own.

Keep in mind that it may be better to ditch your existing adapter and use something else e.g. get a 12V adapter, and put that through a 9V linear regulator. Or get a different adapter that outputs 9V on its own, depending on your tolerances for ripple/noise/etc.