These caves were carved out of solid sandstone on a hillside in the 4th to 5th centuries CE. There are several caves and the best known largest one has four stories with a huge recreated statue of Vishnu in a reclining posture, sculpted from a single block of granite inside the second floor. Many Buddhist artifacts and stupas in Andhra were converted into Hindu temples and deities and Undavalli is an example. It was originally a Hindu cave resembling the architecture of Udayagiri and Khandagiri. It is estimated that these caves were sculpted sometime in 4-5th century CE during the reign of Vishnukundina kings The main cave is one of the earliest examples of Gupta architecture, primarily primitive rock-cut monastery cells carved into the sandstone hills. Initially, the caves were shaped as a Jain abode and the first-floor abode still retains the Jain style; the vihara exhibits Jain monastics and includes tirthankara sculptures. This first level of the cave is a carved vihara and includes Buddhist artwork. The site served as the Bhikkhu monastic complex during ancient period. The walls of the caves display sculptures carved by skilled craftsmen.