Downgrade Yosemite from back-up (not Time Machine)
I would not use Time Machine in this case, use SuperDuper! of Carbon Copy Cloner to make a bootable image on another drive, upgrade, see how it is and if it fails to work, then go boot off the bootable image and clone it back to where it was.

I would suggest taking a Time Machine backup in addition so you always have 2 copies of the data.