Compact Voltage Controlled Current Source Array - PCB
do you think that the +5 V from the USB port of the Arduino board is able to power up also the 4 nanoDACs and the 64 dual op-amps?

At zero load, yes. If I read the specsheets correctly, quiescent current total would be 4 * 4.2mA + 64 * 1mA = 80.8mA.
Since you have not shown any circuit diagram at all, I don't know what your load looks like, so subtract 500mA - 80.8mA = ~400mA of room for load of these stages.

since each board might sink more than 0.8 A, I don't think I can use any Arduino board pin to deliver this power supply. What is the best way to power up all boards with one connector?

You haven't specified a connector, so at a guess: the easiest thing to do is a screw-wire-trap-style terminal block. These are somewhat tall so you'll only want to populate the topmost one in the board stack, unless you have adequate stand-off between the boards and you dearly want them to be identical.
The shields all use standard 100mil-pitch header. You can add more of this or, if you're careful, reuse existing shield pin positions for this purpose, dedicated to power distribution. It's not difficult to find such header rated for 3A.
If power distribution pin current capacity becomes an overriding concern, put your exterior power input block in the middle of the board stack rather than the top, so that the worst-case inter-board current will scale by roughly n/2 instead of n. This scheme has other risks, though, that if there is regulator heat dissipation on this board, convection may be hindered.
You will not need to duplicate regulators on each board, but you will need to duplicate power stabilisation capacitors on each board.
That said, 10,000 amps is &quot;more than 0.8 A&quot;, and none of this would work for such a load, so you're going to need to choose a maximum.

[Could] the DAC [voltage reference] be provided from a pin?

Yes. You won't need to care about pin current capacity, but depending on a long list of things you haven't specified, you'll need to care about noise, interference from close-by digital switching, etc.