2006 MacPro - Worth upgrading for use as a VM host?
Short answer, yes.

The MacPro 1,1 can run 10.7.5 as it's latest OS.  I have a 2008 Black MacBook which runs the same.  Despite being on an OS from 2011, you can still run the latest version (v7.1.0) of VMWare Fusion.  You can also jam 32 GB of RAM in there.  Plus, the Xeon from 2006 will almost be comparable to a Core i5 from a couple of years ago.

Like you, I also updated to an SSD.  Even though the RAM in my MacBook is only 2 GB, it is an amazingly fast machine considering it's specs.  SSDs are simply amazing, they are the best upgrades you can make.

I also have a 2012 Mac Mini.  I installed the maximum 16GB of RAM and a 240 GB SSD.  I am able to run OS X as well as a Windows VM and 2 Linux VMs with almost no degrdation in performance.

It's worth noting 10.7.5 might soon be cut off from receiving security updates, however I imagine you can configure some networking where the VMs have internet access yet the host machine only has access to the local LAN.

I'd say go for it.  That machine is far from dead, there are still some amazing things you can do with it.