The "complexity" of the Turing machine state specification
Complexity theory typically handles asymptotics, not finite cases.  In particular, complexity theory looks at the asymptotic running time as a function of $N$.  If you have only a single fixed $N$, then the standard framework doesn't apply.  It's not possible to talk about the running time being asymptotically an exponential function of $N$, because with only a single value of $N$, it is just a fixed constant; it's not a function of anything.  Complexity classes like EXPTIME are defined in terms of these asymptotics, so they aren't applicable if you have just a single function you want to compute, for a single fixed value of $N$.

So, if you want to apply the machine of complexity theory, you need to consider a family of boolean functions, one for each $N$, and a single Turing machine that computes all of them.  Then you can characterize the running time of that Turing machine as a function of $N$ (the length of the input).  At that point, there is no opportunity to "hide exponentialness" in the definition of the Turing machine, because you need to pick a single fixed Turing machine that computes all of them, and your idea for "smuggling exponentialness" can't be done if you have a fixed Turing machine.  Whatever you do, you only get a constant-factor speedup.



I have seen some formalizations that characterize the complexity not by the running time of the machine, but by the running time plus the length of the machine specification.  For instance, in the RAM model, you can use the running time of the algorithm plus the length of the code of the program.  Then, you can measure how this behaves as a function of $N$.  You can even characterize it for a single $N$, but then you're not dealing with asymptotics or complexity theory or complexity classes like EXPTIME; you're just saying that the least-complexity program to implement this one function has time+size equal to such-and-such.  For instance, this approach is sometimes used in theoretical cryptography (called "concrete security"), where we care about a single value of $N$, rather than the asymptotic behavior as $N \to \infty$.  But again, then you can't talk about complexity classes like P or EXPTIME.