But the former Scotland midfielder insists that he can turn the Cumbernauld outfit around despite only one win in 12 games.
"Of course I can," the 39-year-old manager told BBC Scotland.
"But, if we have any aspirations about getting into the play-offs, we need to win on Saturday."
With Forfar Athletic eight points clear of Arbroath at the top, Clyde slipped 15 points behind third-placed Elgin City when they were trounced 4-1 away to the Moray outfit on Saturday.
The Bully Wee are also 11 points behind this weekend's hosts, who sit in the final play-off spot.
Ferguson's side do have three games in hand over Annan, but Montrose and Berwick Rangers have also overtaken Clyde during their run of nine league games without a win.
"It is tough," admitted Ferguson, who previously had a spell as caretaker manager with Blackpool. "Listen, it's easy to walk away, but I am there fighting and we'll get into the play-offs."
Clyde have played fewer games as a result of a run to the fifth round of the Scottish Cup that ended last midweek after they lost 2-1 in extra-time at home to Championship outfit Ayr United.
"We had a long game on Tuesday night - went into 120 minutes and into extra-time, but I'm not going to use it as an excuse," said the Bully Wee boss, whose side were praised for their display against the full-time visitors.
"We try to do everything properly, the recovery - we went and got protein shakes, recovery shakes, we took them up the night before, which we normally don't do because it is a bit of a cost to the club.
"But it just wasn't good enough. I'm not going to mix my words. I told them after the game."
Ferguson, who had a long playing career in the top flight in Scotland and England, has often expressed his frustration at managing part-time players.
"On Tuesday night, Ayr United scored the winner in extra-time," he said. "I had four guys who were going straight from that game to night shift for eight or nine-hour shifts.
"I take my hat off to them as a realise now how difficult it is.
"But that's why we do a pre-season and we should should be fit enough to play three games in a week."
Clyde's Barry Ferguson admits it is make or break for his side's promotion play-off chances when they travel to face Annan Athletic on Saturday.