Mac Pro Early 2008 fails during boot
From what you've described/tried so far this sounds like it's a RAM related issue.

Keep track of which RAM is which (label each stick) and experiment with different configurations. The most important thing to try out would be swapping the RAM between the reisers and testing. This should help you ascertain if it's the particular reiser that's at fault or the RAM in that reiser.

Also, make sure that the RAM configuration is correct!
Check that it is here

You may also want to consider the PSU, logic board or CPUs - really though only an Apple tech with access to known good spares can really confirm this.

Regarding the SMC; When you unplug the machine for more than 15 secs the SMC resets. This is the official way to reset the SMC, hence the lack of a reset button.

When you press the diag button you are doing nothing more than telling the system to light up some extra diag LEDs that are usually not lit during standard boot procedure.

See this table for a list of boot/diag LEDs


Note the LEDs which will not light unless the diag button is pressed


The system will boot when you press this button because the LEDs are used to check that the system is getting through the various POST (power on self tests) during boot. The LEDs can tell you where and why the boot is failing.