Can I safely run a 6V DC air pump using a 9V Car adapter
6v from 4 off D batteries will be 6volts or less, not 6v or more, when driving a reasonable load.

Depending how much you paid for the pump (so how well made it's likely to be), the commutator sparking, the bearings rattling, the windings and brushes over heating are just a few of the obvious wearout mechanisms that would be exacerbated by excess speed, voltage and load. The lifetime at 9v may be hours, or may be minutes. Are you feeling lucky?

Look on Amadong or fleaBay and find a DC to DC converter that does go down to 6v, they're not expensive. Cheaper to buy one before you destroy the pump rather than after, and another pump.