Capacitors with an LDO voltage regulator - how necessary is it?
The data sheet for the regulator suggests ... How important is this?

About as important as that your circuit work reliably.

Trying to second-guess datasheets is a bad idea.  Unless the datasheet explains exactly what is going on and gives you guidance on different choices, the specifications are requirements, not options.

Only Microchip knows the limits of stability of the MCP1700.  Their engineers have analyzed this over many cases of of current, headroom, output impedance, temperature, and other parameters.  They have distilled the result of all this analysis down to a simple range of capacitance that need to be on the input and output for the device to reliably work.  Why would you not follow that?

When you violate any specification in the datasheet, all remaining specifications become null and void.  There is no longer any guarantee what the device will do.  One or a few individual devices seeming to operate correctly at some limited combinations of current, dropout voltage, temperature, source impedance, output impedance, etc, is not useful evidence of anything.

Speaking in general, the input cap is to guarantee that the regulator sees some minimum impedance at certain frequencies.  Ideally the input voltage has 0 impedance.  Since that's not possible, they tell you the minimum input capacitance to put right by the regulator to guarantee the input impedance the design assumes.

The output capacitance is part of the overall feedback loop, so effects stability.  The requirements vary considerably across regulators, especially LDOs like the MCP1700.  Early LDOs were intended for tantalum capacitors on the output and actually relied on some minimum ESR (effective series resistance) of the cap.  Others specify a range of capacitance, with both higher and lower being bad.  One nice thing about the MCP1700 is that there is no minimum ESR requirement.  You can connect a ceramic cap directly to its output.  In fact, you need to.

Do what the datasheet says, else you're a test pilot.