Will hardware/implementation affect the time/space complexity of algorithms?
Sure.  Certainly.  Here's how to reconcile your discomfort.

When we analyze the running time of algorithms, we do it with respect to a particular model of computation.  The model of computation specifies things like the time it takes to perform each basic operation (is an array lookup $O(\log n)$ time or $O(1)$ time?).  The running time of the algorithm might depend on the model of computation.

Once you've picked a model of computation, the analysis of the algorithm is a purely abstract, conceptual, mathematical exercise that no longer depends on hardware.

However, in practice we usually want to pick a model of computation that reflects the reality of our hardware -- at least to a reasonable degree.  So, if hardware changes, we might decide to analyze our algorithms under a different model of computation that is more appropriate to the new hardware.  That is how the hardware can affect the running time.

The reason this is non-obvious is because, in introductory classes, we often don't talk about the model of computation.  We just implicitly make some assumptions, without ever making them explicit.  That's reasonable, for pedagogical purposes, but it has a cost -- it hides away this aspect of the analysis.  Now you know.