The comments came ahead of his Budget speech, in which he cut the country's growth forecast for 2016 to 0.9%, down from 1.7%.
He conceded the economy was struggling with shrinking growth, 25% unemployment, and widespread poverty.
The South African currency, the rand, which has halved over the past five years, fell after the speech.
It dropped 2.25% to make one rand worth around $0.0639.
In a briefing ahead of the Budget, Mr Gordhan said: "There is no doubt about the fact that we are in crisis."
The measures announced in Parliament were aimed at stopping the country falling into recession and to appease the rating agencies who have threatened to downgrade South Africa to junk status, which would raise borrowing costs for the country.
Mr Gordhan unveiled government spending cuts, a civil service job freeze and some moderate tax rises.
These tax increases affect property sales, fuel, sugary drinks, alcohol, tobacco and and capital gains, as well as environmental levies, which are expected to bring in an extra 18bn rand ($1.18bn; Â£840m).
He had been expected to announce plans on privatising state assets - he fell short of that but said the government was looking at the possibility of merging the loss-making national carrier, South African Airways, with the state-owned SA Express airline "with a view to engaging with a potential minority equity partner".
Privatisation has long been resisted by sections of the ruling African National Congress.
Mr Gordhan was blunt on his outlook for the country.
"We cannot spend money we do not have. We cannot borrow beyond our ability to repay. Until we ignite growth and generate more revenue we have to be tough on ourselves."
Maike Currie, investment director at Fidelity International, said she was unsure if the measures announced would stop the rating agencies cutting their outlook for the country, as they have warned: "There was no increase in VAT, which has remained unchanged at 14% for two decades now, or income tax hike as the finance minister looked to moderate the impact of tax increases on struggling South African households amid a testing economic backdrop.
"Whether this will be enough to appease rating agencies and businesses, however remains to be seen".
She added that it was interesting that Mr Gordhan had focused on cutting down government spending without enforcing austerity measures on everyday South Africans, such as raising income tax.
In his Budget speech, South Africa's finance minister Pravin Gordhan was cautiously optimistic and realistic about the challenges facing the country.
He spoke about the need to tackle wasteful expenditure and was candid about corruption.
The most blatant example of misuse of public funds is the Nkandla scandal where officials used taxpayers money to renovate the president's private country home.
His  strategy seems to be that of spending less in order to save more.
The minister said that his focus will be to reduce the government deficit to 3.2% this year, and over the next three years.
The rand weakened after he had finished speaking which suggests the markets are not convinced that the minister has a recovery plan that will lift growth and convince the ratings agencies before they take a decision around May or June.
The ruling ANC party faces municipal elections later this year and much was riding on this Budget to turn the economic situation around. Although the country is rich in mineral resources it has been hit hard by the fall in global commodity prices. South Africa's farmers have also been hit by the worst drought in more than a century.
Last December, President Jacob Zuma shocked the nation and the business community by suddenly replacing the well-respected finance minister Nhlanhla Nene with an ally and backbencher David Van Rooyen.
The markets took a dim view, with foreign investors withdrawing money and there were complaints within his own ruling African National Congress party.
Four days later President Zuma did a U turn and re-appointed Mr Gordhan who had served as finance minister for five years to 2014.
The South African economy is 'in crisis' says the country's finance minister Pravin Gordhan.