Glass acting as a lens?
It looks like the glass object has no thickness.
Without any thickness, rays are bent only once, and behave as if everything behind the plane were inside the glass, rather than behind it.

After adding a solidify modifier (or thickness via any other method), rays passing through the glass will be bent upon entering it, then bent back upon exiting it. This should result in the expected appearance: