LDO with tantalums and circuit with ceramic bypass capacitors
The distance of the ceramic cap from the LDO will not matter much. It will add some nH of inductance which will only matter at frequencies much higher than the bandwidth of the LDO. The ceramic capacitor however will be in parallel with the tantalum capacitors and could degrade the phase margin. There are many LDOs you could use instead that are stable with ceramic capacitors. This link might be helpful:

AN-1482