They were evicted from the British overseas territory in the 1960s to allow the UK government to lease Diego Garcia to the US to use as an airbase.
Many residents of what is the largest of the Chagos Islands ultimately resettled in Crawley, West Sussex.
The petition demands the islanders' right to return home is respected.
TV presenter Ben Fogle, patron of the UK Chagos Support Organisation, handed in the petition, which had been signed by more than 2,500 people.
"Diego Garcia, which is one of the main prohibiters to the islanders returning, is up for renewal in the next year or so," Mr Fogle said.
"We're asking this government to think about whether these islands should belong to the people who inhabited them for a very long time, or to the US government who have used it for renditions - for effectively torture."
He said he had visited the islands and found "houses still as they were left" and "the graveyard strangled by vegetation having been left untended".
"For me, being a Brit, it was probably one of the things I'm most ashamed about, that I'm part of a country that forcibly evicted these people and is now refusing their right to return," he said.
Mr Fogle added: "How wrong is that, that I've been able to be there and they can't?"
Henry Smith, the Conservative MP for Crawley, said: "I think we can't turn back time but what we can do is rectify the problem as soon as possible."
A Foreign Office spokesman said: "The government is committed to its ongoing review of resettlement policy towards the British Indian Ocean Territory."
A petition calling for residents of the Chagos Islands to be allowed to return to their Indian Ocean homeland has been handed in to Downing Street.