The inmate, from Romania, had been serving a 30-month sentence for sexually assaulting a girl under the age of 13.
He was due for release from HMP Littlehey in Cambridgeshire on 6 January, but was deported on Friday morning.
The Ministry of Justice (MoJ) said an investigation is now under way.
A spokesman said the man had not been released out on to the streets, but placed in an immigration removal centre.
"He was placed in an immigration removal centre before being removed from the country the following day," he said.
"Incidents like this are extremely rare."
An investigation was under way to establish the circumstances of the man's release, he added.
A sex offender was mistakenly released from prison a month early, the BBC has learned.