How is it valid to use oracles in mathematical arguments?
Oracles are a very general formalization of the idea, "If I could solve&nbsp;$X$ efficiently, I could use that to solve&nbsp;$Y$ efficiently."  I accept that it sounds a bit silly to go as far as "If I could solve problem&nbsp;$X$ in constant time, I could use that to solve&nbsp;$Y$ efficiently" but, actually, that doesn't make any real difference at the level of detail we're working at, here.

For example, consider an oracle&nbsp;$O_1$ that can solve SAT in constant time, versus an oracle&nbsp;$O_2$ that solves SAT in, say, cubic time. The problems you can solve in polynomial time with access to&nbsp;$O_1$ are exactly the same as the ones you can solve in polynomial time with access to&nbsp;$O_2$. Sure, the polynomial will be better if you use&nbsp;$O_1$, but it's still a polynomial if you use&nbsp;$O_2$. In fact, the answer won't change whatever polynomial bound you assume on the running time of the oracle so we choose to simplify the analysis by choosing the simplest polynomial possible:&nbsp;$1$.

In some cases, you can see oracle arguments as hedging your bets. Although you lead your question with "Oracles don't exist", we don't actually know that. Suppose somebody produced a polynomial-time algorithm for SAT tomorrow. Wouldn't it be a shame if all we could solve with it was SAT? But because we have all kinds of oracle reductions available to us, we could already solve any NP-complete problem efficiently (the many-one reductions used to prove problems NP-complete are just a very restricted kind of oracle reduction). Conversely, if somebody proves tomorrow that SAT has no efficient algorithm, then all that work on oracles tells us that all the other NP-complete problems are also genuinely hard.

But remember that oracle arguments are conditional: if I could do&nbsp;$X$, then I could also do&nbsp;$Y$. If it turns out that you can't do&nbsp;$X$ then, OK, maybe you can't do&nbsp;$Y$ after all. But it's still a perfectly valid what-if question. So, for example, it still makes sense to consider Turing machines with an oracle for the halting problem, even though we know for a provable fact that no Turing machine can actually solve the halting problem. There's nothing wrong with asking the what-if, as long as we remember that it is a what-if.

To address a couple of other points,


  I don't understand how one can make an mathematical argument based on something that does not exist (excluding unbounded memory machines, of course).


So we're allowed to use some non-existent things but not others?  Why?


  Turing Machines (bizarre fetish formalism if you ask me)


Are you trying to get a rise out of me, Agent Kujan?


  Also, that X historic proofs rely on oracles makes oracle arguments valid is a circular argument.


Yes, that's circular. But it's also a strawman: I've never heard anyone make that argument.