According to a mythological account, a sage named Matanga manifested in form of a lingam, and controlled the god of love. His hermitages were located at Khajuraho, Kedarnath, Varanasi and Gaya. These are now the sites of four Matangeshvara (Lord of Matanga) temples. Another account states that the temple is named after an aspect of Shiva who controlled the god of love.