How can I limit SQL Server Agent's simultaneous jobs number?
This is best done outside of SQL Agent.  Honestly, at 800+ jobs running every hour you should have moved to a different method long before now.  

My preference for this would be to create a single master SSIS package that reads in a list of jobs that need to be run and then starts up the first X in the list, then sleeps until the number of running jobs is below threshold, then starts up the next X in the list.  You could randomize the list or sort it, as you choose.  I've done something similar to this in my environment where I needed to make sure that certain classes of packages (client imports) were done one at a time.

The less dynamic approach would be to just break the list of jobs that need to be run into however many simultaneous you want running.  Each of those would have a list of steps that call the actual job that needs running.  It takes less fiddling about with timing, sleeps and querying active jobs but is less dynamic (for example, changing the number of max simultaneous threads will take work.

But in any case, I would leave the jobs as they are now, just remove/disable their schedules.  Then whatever solution you want can just call the sp_start_job procedure.  This lets you still easily run any job you want out of cycle and you see independent reports of success/failure.