Option/speed Q's about best configs to upgrade early 2008 Mac Pro 8-core to SSD
Maxing out RAM can only situationally improve your performance. If your computer isn't ever using a ton of swap space (less than a gig), adding more RAM isn't likely to help much. The SSD will almost certainly improve the performance of your computer. Yosemite really is designed for SSDs. It seems to do a ton more arbitrary reads/writes to disk during the course of its operation than earlier versions did, so it will be dog-slow on a HDD.

As for the SSD on PCI, that is definitely the better solution. SATA 2 (which the 2008 Mac Pro comes with) is far too slow for the faster SSDs. There's little point in hooking up a modern SSD to that. PCI should be able to handle it.