Why do compute time complexity for algorithms?
The problem you are describing is indeed a shortcoming of asymptotic notation. Asymptotic notation has several advantages:


It doesn't depend as much on the underlying computation model. For example, the microprocessor executing the compiled code shouldn't affect the running time by more than a constant.
It avoids cumbersome exact results, for example instead of $n^2 + 3.5n\log n + 700\log n\lfloor \log\log n \rfloor$ we can write simply $O(n^2)$ (or even $\Theta(n^2)$).
In many cases it is hard to analyze exactly the running time (even ignoring the first issue), but easy to come up with asymptotics. Standard examples are recursive algorithms. In those case, however, it is often possible to come up with first-order estimates, that is, perhaps we can say that the running time satisfies $T(n) \sim 2n^2$, which is more accurate than $T(n) = \Theta(n^2)$.
Practically speaking, in many cases the constants hidden by the big O notation are "small" or at least "reasonable", and therefore for $n$ which is not too small, a $\Theta(n)$ algorithm will usually be faster than a $\Theta(n^2)$ algorithm.
Big O estimates function as ball park heuristics, which allow us to guess which algorithm is better or which part of an algorithm needs optimization the most. If in doubt, however, it is better to use profiling.


On the other hand, in some cases the problem you're indicating is serious. A classical case is fast matrix multiplication, where the trivial $O(n^3)$ algorithm often beats Strassen's $O(n^{\log_2 7})$, and always beats all other algorithms (which have better asymptotic running time).

Summarizing, there is no easy answer. Asymptotic notation is a good model of reality in many cases, but not always. Still, it is useful since it is concise and allows quick comparison between different algorithms.