Is deleting the yosemite install partition safe?
There is no uninstall per-se for any OS X installation. 

You can just boot into your other OS to test that it boots before deleting your current OS partition. I can't speak to any changes that Yosemite has made to your Mac or the recovery partition, but I'd start with powering it off and then booting while holding the option key. From that boot manager, you can test that your Mavericks partition is working and then decide to delete the other partition(s) or if you need to fix things before getting to a point where you have no bootable volumes. In fact, you can't self delete a running partition so you'll likely have to boot to Mavericks (or Recovery HD) to delete a new install as you have done.