Illegal immigrants and foreign offenders facing deportation are languishing in detention for months and even years because no new travel documents have been obtained for them, an official watchdog has found.

John Vine, the independent chief inspector of borders and immigration, says a sample group of 27 foreign offenders whose cases were examined in detail by inspectors had each been locked up for an average of 563 days – more than 18 months – beyond the end of their sentences.

In one case, Vine found that a detainee had been held for 1,288 days – more than three and a half years – because there was no passport to send him home with. The chief inspector said that detention had so far cost the taxpayer £211,032, at the rate of £164 a night.

He called for an urgent review of all the long-term immigration detainees for whom a lack of travel documents was the primary barrier to their removal. He said the Home Office could not even tell him the overall scale of the problem and only had detailed figures for foreign national offenders, not all illegal migrants facing removal.

The report is one of two highly critical immigration inspection papers whose publication has been delayed by at least two months by the home secretary, Theresa May, who determines their publication date.

The second Vine report reveals that immigration enforcement officers have been involved in the widespread abuse of their powers to carry out raids on restaurants and other business premises and make arrests without a warrant. The chief inspector said that in two-thirds of the case files he examined, the use of the power had been abused, and in six cases the raids had been carried out unlawfully.

The first inspection into getting travel documents for those facing deportation was undertaken between May and September last year and the report was submitted to the home secretary on 17 December.

Vine said it appeared the Home Office was keeping foreign offenders and illegal immigrants facing deportation locked up in the hope that they would eventually comply with attempts to get them new travel documents and co-operate with their removal.

"I was concerned to find that the Home Office was keeping foreign criminals, who had completed their prison sentences, in immigration detention for months or even years in the hope that they would eventually comply with the re-documentation process. Given the legal requirement only to detain individuals where there is a realistic prospect of removal, this is potentially a breach of their human rights. It is also very costly for the taxpayer," he said.

In late 2012, the inspectors made a detailed study of a sample of cases of those facing deportation who refused to comply by giving evidence of their identity or nationality. They found that nine individuals had been removed by 15 June 2013 and a further 16 released on bail. But 27 individuals who remained in detention had been held for an average of 563 days. The shortest period was 176 days and the longest 1,288. In cases where no travel documents had been obtained because the country to which they were being sent back refused to co-operate, the average period of detention was 755 days – more than two years.

The inspectors found that in 13 of the 27 cases of prolonged detention, no application for the individual's travel documents had actually been lodged with his or her embassy; in a further six cases, previous applications that had been rejected had not been renewed. In only seven of the 27 cases was an application for new travel papers under consideration by the relevant embassy.

Vine said the Home Office had been successful in securing emergency travel documents for a second, much larger group of foreign nationals with no right to be in Britain, but had made little effort to remove them.

The chief inspector said the Home Office had a stock of several thousand emergency travel documents that had already been agreed by foreign embassies, some dating back more than 10 years. But many of those cases were not being worked on and 15% of the people had actually been given permission to stay in Britain and should not have remained in this pool.

"The Home Office has no clear picture of the scale of non-compliance by individuals and had no effective strategy for tackling this," Vine said.

The second report revealed serious failings in the use of immigration powers to carry out raids on business premises where swift action was required, usually because of an immediate threat of absconding by immigration offenders.

The inspectors found there had been a big increase in the number of times the "warrantless" powers were used in the last two years, rising from 843 in 2012-13 to 1,049 in the first six months of 2013-14. Over the same period, the number of raids carried out with search warrants fell from 5,850 to 3,046.

There were also wide variations in the use of the powers: the south London immigration enforcement team used them in more than two-thirds of its illegal working operations, while east London used them only in 3% of such operations.

"There was widespread non-compliance with the Home Office's own guidance in relation to the use of this power and a lack of management oversight," reported Vine. "In almost two-thirds of cases [of the sample 60 examined], I disagreed with the decision made by the assistant director to authorise the use of this power. In particular I was very concerned to find six cases where the power appeared to have been used unlawfully." He said that in these cases either the authorising officer was not senior enough or the power was not used within the timescale set out by law. He said the average time taken to prepare the raids was 13 days, which was ample to apply for a search warrant.

The chief inspector said it was apparent during the inspection last autumn that a significant number of staff and managers were either ignorant of the Home Office guidance or were choosing to ignore it. He found that once his inspection got under way, senior managers responded by introducing a range of measures to improve performance and compliance.

A Home Office spokesman said: "We are building an immigration system that is fair to British citizens and legitimate migrants and tough on those who abuse the system or flout the law. Those with no right to live in the UK should leave at the earliest opportunity and we make no apologies for enforcing our immigration policy."

He said the letters authorising raids on business premises without a warrant and emergency travel documents played a vital role in tackling and removing immigration offenders: "Work was already under way to improve their use before the inspections were conducted and we will continue to make improvements in the way we work."

He added: "The immigration bill will make it easier than ever to remove people with no right to be here. It will cut the number of appeal rights from 17 to four, allow us to remove people while their appeals are ongoing and restrict how often individuals can apply for immigration bail without a change in circumstance."