Guptesvara Siva temple is located within the Kapilesvara temple precinct and the enshrined deity is a Shiva lingam within a circular yoni pitha (basement) at the center of the sanctum. The temple was built in 14th -15th Century A.D.