In [Shabbos 31a](http://hebrewbooks.org/shas.aspx?mesechta=2&daf=31&format=pdf), Hillel addresses a potential convert who asks him to summarize the whole Torah, and he says, "What is hateful to you, do not do to others."

This is a nice d'var Torah, but it seems to be lacking when trying to explain the whole Torah. This summary may cover all of the halachos "Bein Adam L'chaveiro" (between people), but it seems to fail to cover the halachos "Bein Adam LaMakom" (between man and God).

How has this been explained?

Related: [Elucidation On one foot](https://judaism.stackexchange.com/q/12620/1713)