The DUP leader said he was not prepared to remain as first minister in a power-sharing government "kept in the dark" about such an important matter.
He was speaking after the trial of Donegal man John Downey collapsed.
Mr Downey denied killing four soldiers in the 1982 IRA Hyde Park bombing.
The case collapsed because he was mistakenly told in a letter in 2007 that he was no longer a wanted man, despite the fact that police in Northern Ireland knew he was still being sought by Scotland Yard.
Although police soon realised they had made a mistake, the assurance was never withdrawn.
It said a total of about 200 such letters were distributed - mostly under the previous Labour government - to suspects "on the run".
The Northern Ireland Office (NIO) has confirmed that all the recipients were republicans.
Prime Minister David Cameron told the Commons on Wednesday that Mr Downey should never have been sent the letter and that it had been a "dreadful mistake".
Mr Downey's lawyer said his alleged offences had been categorised as one of the on-the-run cases that would no longer be pursued in the light of progress in the Northern Ireland peace process.
Attorney General Dominic Grieve told the Commons on Wednesday that the judgement would not be appealed.
Mr Robinson called for all letters sent out to be rescinded and "full disclosure" of what had happened.
"I am not prepared to be kept in the dark by Her Majesty's government about matters relevant to Northern Ireland," he told the BBC.
"I want a full judicial inquiry to find out who knew, when they knew and what they knew. I want to know who they are and what crimes they are believed to have committed."
He said he felt deceived by the government, and if he and former DUP leader Ian Paisley had known about this, they would not have entered into power-sharing government with Sinn FÃ©in in 2007.
"I am not prepared to be a stooge for Westminster who keep secrets on matters which are now devolved to Northern Ireland," he said.
Anyone already convicted of paramilitary crimes became eligible for early release under the terms of the Northern Ireland Good Friday agreement of 1998.
The agreement did not cover:
Mr Robinson said he would discuss the matter with Northern Ireland Secretary Theresa Villiers at a meeting later on Wednesday.
Responding to Mr Robinson's comments, Martin McGuinness, Northern Ireland's deputy first minister, tweeted: "My unionist colleagues need to calm down. We've all come a long way. No sensible person will thank anyone for threatening the institutions."
Denis Bradley, a former vice chairman of the NI Policing Board and who co-chaired a group set up to deal with Northern Ireland's past, said he was surprised when unionists said they were not aware of the issue.
"I don't know who kept who in the dark because it was very much in the public situation," he said.
"It was well briefed at the Policing Board at the time, it may not have been at full board, but it was certainly at a subcommittee which means it would be in the records, in the minutes of some of those board meetings.
"It's in the Eames-Bradley report, it's there to be read, we referred to it, we addressed it, so I don't know who is being kept in the dark."
Jim Allister, leader of the Traditional Unionist Voice party, which is opposed to power-sharing, said that maybe the DUP "did not want to know" about the letters.
He said: "It seems to be a failure on their part to fully interrogate the issues that lay at the heart of the negotiation to get both Sinn FÃ©in and DUP into government.
"Maybe it was a bit of we don't want to know because we're very anxious, as the DUP was, to get into government with Sinn FÃ©in and now it's all coming home to roost."
Ms Villiers admitted that the fall-out from the case had made progress on the Northern Ireland talks chaired by US diplomat Richard Haass more difficult.
The talks - on parades, the past and flags - broke up without agreement in December.
"There was some very important work that has been going on in recent weeks between the party leaders; some real dedication to trying to find a way forward," she said.
"There is no doubt that finding that way forward will be more difficult now given the events of the last 24 hours, but I continue to encourage the parties to do so."
She will meet Northern Ireland Justice Minister David Ford on Thursday.
Mr Ford told the BBC he only became aware of the letters following the outcome of the court case, and it had nothing to do with his department.
He said Ms Villiers "may wish to dump responsibility for the scheme onto her predecessors in the Labour government, and indeed it was them who set it up, but she cannot dump responsibility for it on the Department of Justice which was told nothing about it".
"I am concerned about the whole issue of what this so-called scheme amounts to," he added.
The Alliance leader said that while his party accepted that the issue of on-the-runs had to be dealt with, "what we needed was an open and transparent process".
He said he did not think the government had been honest about the issue.
The NIO has said 38 of the letters were issued by its officials to "individuals who were already under consideration" since the coalition government came into power in 2010.
The last letter was issued in December 2012.
Ulster Unionist leader Mike Nesbitt said he had spoken to his party's former leader Lord Trimble earlier on Wednesday.
"David Trimble had no knowledge of these letters until yesterday, he would never have supported an amnesty," he said.
"Dealing with the past is off the agenda for the party leaders' talks until this matter is sorted."
Meanwhile, BBC News Channel's chief political correspondent Norman Smith has said some Conservative MPs want the threat of prosecution of paratroopers involved in Bloody Sunday to be lifted following the collapse of the Downey case.
He said one MP told him: "I'm damned if they should be given an amnesty and former soldiers left hanging there; uncertain over whether they might face prosecution."
Thirteen civilians were killed during the Bloody Sunday shootings in Londonderry in 1972. A 14th died later.
It is understood a letter is to be written to the prime minister calling on him to lift the threat of any criminal prosecution against the paratroopers.
Northern Ireland's First Minister Peter Robinson has threatened to resign unless there is a judicial inquiry into secret letters given to more than 180 Irish republican paramilitary suspects.