Why do floating point values have infinity?
Because sometimes there just isn't an exponent high enough.

First of all, there is no way to effectively represent all real numbers in a reasonable computational model: it is sufficient to observe that the set of configurations of e.g. a Turing Machine is countably infinite, and the set of real numbers is uncountably infinite. Floating point numbers, are ultimately an approximation of what the "actual" computation would be; models for exact real computation exist but as we said before they necessarily "miss" some numbers.

INF and NaN are part of the IEEE standard for rather practical reasons, namely to give meaning to computations that should fail, for a reason or another, and handle such cases in a safe way. We could, as you are basically suggesting, take a floating point model that is finite but unbounded, dynamically adding more space as we need it. As others have already commented, this kind of generalization is actually missing the point: the strength of floating point computations is that they can be performed very efficiently on hardware, and a dynamic solution would be infeasibly more complex.

But even assuming such a model, we aren't really addressing the core of the issue. Let's say that at a certain point we ask the machine to perform the computation $\log(0)$. Mathematically, it doesn't make any sense, because $\log$ isn't defined on $0$. If we didn't have a special value like -INF to return, we would have to abort the computation (or have the program diverge, or terminate in some kind of error state). You might think that such a thing is reasonable: after all, if we had the machine compute $\log(0)$ the program must be wrong in a way or another. 

But it gets worse. In any floating point model, there is a smallest positive number that we can distinguish from zero, let that number be $\varepsilon$. Now, we ask the machine to compute $\delta = e^{-10}\varepsilon$ and then $\log(\delta)$. Observe that all those operations are mathematically legit, for instance we can easily see that $\log(\delta) = \log(\varepsilon)-10$. However $\delta &lt; \varepsilon$, which means that our model literally cannot tell $\delta$ and $0$ apart, by definition of $\varepsilon$. It is much less reasonable to make the entire program fail, now.

Special values like NaN and INF manage to at least salvage the "logical" aspect of the computation, correctly handling equality/inequality checking and the sign, if applicable (i.e. +INF is greater than everything else, NAN is not equal to NAN and so on).