Why is Mars considered the outer edge of the "goldilocks zone"?
The circumstellar habitalbe zone can be defined as the distance range around a star, where the mean temperature of a rotating planet would be between 0 and 100 centigrades, if radiation (heat) received from the star and thermal radiation emitted by the planet form an equilibrium.

But that's only a rule of thumb. It has been redefined, and is still disputed.
Hence your criticism is justified to some degree.
There may exist exceptions of habitable planets outside the presumed habitable zone; and there exist certainy many planets within the habitable zone which aren't habitable.
But the probability for a planet similar to Earth to be habitable on its surface is thought to be highest in the habitable zone. When looking for habitable exoplanets, it's therefore more efficient to focus on the habitable zone.