 Portuguese rule saw a decline in the Hindu population on the island and the abandonment of the Shiva cave ( main cave ) as a regular Hindu place of worship , though worship on <unk> , the festival of Shiva , continued and still does . The Portuguese did considerable damage to the sanctuaries . Portuguese soldiers used the reliefs of Shiva in the main cave for target practice , <unk> only the Trimurti sculpture . They also removed an inscription related to the creation of the caves . While some historians solely blame the Portuguese for the destruction of the caves , others also cite water @-@ logging and dripping <unk> as additional damaging factors . The Portuguese left in 1661 as per the marriage treaty of Charles II of England and Catherine of <unk> , daughter of King John IV of Portugal . This marriage shifted possession of the islands to the British Empire , as part of Catherine 's dowry to Charles . 
