There are three caves in the western side of the lower hill, with rock-cut images dedicated to Shiva in each of them. The first two caves have intricate rock-cut sculptures and Dvarapalas on either side of the sanctum, while the third one is plain. There are various sculpted images of Vishnu, Durga, Lingodbhava, Harihara. The image of Dvarapalas in the caves, on either sides of the sanctum, with each leaning in the direction facing the sanctum, are found to be the earliest representation of the images. These are not found in Pallava architecture, which precedes the Pandyas. The caves are considered one of the major specimens of rock-cut architecture of the Pandyas, counted along with Vettuvan Koil, Thirumalaipuram and Thiruparankundram.