Their last home game, a 4-0 win over Droylsden, attracted 142 supporters and after staff and players were paid, the Tigers were left with £21 profit.
Last week, the club issued an appeal for £25,000 so they could "stay in a solvent position" until the summer.
"There are fans out there but if they don't come back then we're not going to have a team," said Hughes.
The club's owner, Eamonn McGurk, has supported the club financially since they struggled to pay a tax bill of more than £80,000 in the 1990s.
It is estimated his investment runs to more than £1m, in addition to funding the planning costs for a new ground.
The Blue Square Bet North club have been without a home for six years after their Meadow Park ground was destroyed by flooding and their nomadic status has played a major part in their financial difficulties.
They have shared with local clubs Forest Green Rovers, Cirencester and most recently Cheltenham Town - where they have been based since 2010.
But Hughes believes their move out of the city, and the historical rivalry the Tigers have with Cheltenham, has caused fans to abandon their local club.
"If I had £10 for every time I've spoken to someone who has said 'I used to support Gloucester but I'm not going to come back until you've returned to Gloucester', I'd probably be able to sustain the club," he told BBC Gloucestershire.
"There's always been this rivalry across the divide but Cheltenham are a Football League club now and that's what we've always wanted to aspire to.
"We've been fortunate that Cheltenham allowed us to have a ground share - they could have said no."
Gloucester pay £40,000 a year to play at Whaddon Road, but were recently threatened with eviction after failing to meet payments.
The issue is close to being resolved and Gloucester hope to have a deal in place next week to remain in Cheltenham for next season's campaign.
Plans for a new stadium at the Meadow Park site were submitted to Gloucester City Council in 2011, but have yet to be approved.
"It has left me so frustrated," Hughes added. "There's been a lot of money spent on reports etc [for new stadium] and it's been draining.
"It's been difficult. We're tired and we just need something we can hang our hat on. We need something that will attract investors and keep us going."
Gloucester City chairman Nigel Hughes has urged absent fans to return to the struggling non-league club.