Extrapolation of the expansion of the Universe backwards in time using general relativity yields an infinite density and temperature at a finite time in the past.

If the matter contained within our galaxy were concentrated within a small radius wouldn't that lead to the entire universe being a black hole? If so how is it that all the matter of our observable universe could have originated from a region of infinite density?