SATA 2 SSD in Mac Pro '09
Empirically, the answer is 'Yes'.

I have a very similar machine - 4,1 -> 5,1, dual 3.46GHZ Xeons, 64GB RAM &amp; also an older 3,1.

Both now have an SATA III 840 EVO 1TB as boot drive [best available at the time of purchase] in the first optical drive bay [leaving room for your usual 4 HDs in the regular bays. Adapter bracket a few quid/dollars/shekels from eBay.
On the 4,1 it connects directly to the cabling in the bay, the 3,1 needs a longer SATA cable down into the main body, under the cooling fans, where the two 'secret' SATA ports are.

No, they don't run at SATA III speeds, but boy do they make the entire machine more responsive. Boot time down from a couple of minutes to 15 seconds. iTunes [the slowest to launch that I know of] now launches in about 5 seconds. Finder is zippy &amp; responsive, no more waiting for it to fill a window.

You get so used to the speed you don't notice it until you have to use a machine with a spinning HD again.

Speed test - SSD

 

Speed test - HD