Mohammed al-Barghathi earlier said he had decided to step down in the wake of the recent sieges by armed groups of the justice and foreign ministries.
But he was subsequently told by Prime Minister Ali Zeidan not to present his resignation at a cabinet meeting.
The gunmen had demanded parliament pass the Political Isolation Law, which bars Gaddafi-era officials from office.
The General National Congress did so on Sunday despite criticism from human rights groups, who said the legislation was too vague because it potentially barred anyone who worked for the authorities between 1969, when Muammar Gaddafi came to power, and 2011, when he was ousted.
Despite this, some gunmen have refused to withdraw from the ministries.
Addressing a news conference in Tripoli on Tuesday afternoon, Mr Barghathi cited the sieges as the reason for offering his resignation.
"I will never be able to accept that politics [can] be practiced by the power of weapons," he was quoted as saying by the Reuters news agency.
"This is an assault against the democracy I have sworn to protect."
However, Mr Zeidan's office later issued a statement saying: "The cabinet would like to clarify that the resignation was not officially presented to the prime minister.
"The prime minister has asked the defence minister to renounce it and [he] has confirmed that he will continue his work and expressed his understanding of the present circumstances in the country."
The BBC's Rana Jawad in Tripoli notes that the Political Isolation Law will apply to Mr Barghathi when takes effect. He was commander of the air force in Benghazi before retiring on a government pension in 1994,
The law could affect other senior members of the government, including the head of the General National Congress Mohammed al-Megaryef.
He defected from Gaddafi's government in the 1980s after serving as Libya's ambassador to India.
Gunmen who have remained at the foreign ministry - the first to be targeted on Sunday 28 April - said they were prepared to go but have not done so as yet, our correspondent says.
The gunmen outside the justice ministry, which was surrounded last Tuesday, have said they are waiting to be replaced by official security forces, she adds.
The various militias around the ministries have different allegiances.
The Higher Revolutionary Council, which claims to represent various ex-fighters and is believed to control some of the armed groups, told the BBC it wanted to see the prime minister removed from office.
Our correspondent says Mr Zeidan, who it seems will not be affected by the new law, recently described those behind the armed protest as the "losers" in last year's nationwide elections.
Libya's Defence Minister has withdrawn his offer to resign, hours after saying he was leaving his post.