Is there a way to disable a Jenkins plugin for a single job?
Sadly I just switched jobs and I don't have access to a Jenkins instance (least not yet) so I cannot try it out for you.

That said some ideas quickly jump in my mind that could get you there.

I've had some success with the conditional build step plugin though I don't know if it would work for build timeouts.  

If not directly perhaps as a means to change some values through the rest API ? Maybe using job parameters ? You could also split the job in two and call the appropriate one (with or without timeout) using a conditional build step (mentioned above)...

If you are int a pipeline you could use the timeout in a jenkins file.
https://jenkins.io/doc/pipeline/steps/workflow-basic-steps/#code-timeout-code-enforce-time-limit.

let me know if any of this is helping you.