Thousands of fans celebrated on the pitch at full-time after the Seagulls beat Wigan.
The promotion came 34 years after the club reached the FA Cup Final but was relegated.
The club's chief executive Paul Barber said a 25,000-strong crowd remained at the stadium after the game on Sunday.
Virgo said the scenes of euphoria at the Amex stadium followed the heartache and "dark, dark times" since 1983.
From brink of disaster to the Premier League
"Nothing will ever top this success in the history of the football club," he said.
"They're not just going into a big league, they're going into a worldwide league and the club now will be seen all over the world."
Championship leaders are promoted to the Premier League
Mr Barber said: "We stayed in the ground for quite a long time because we had about 25,000 people we had to look after.
"We can never encourage people to go on the pitch, but we knew they would and we knew that people had waited a very, very long time for this moment and we certainly weren't going to spoil it."
Describing the scenes as "unbelievable", he added: "They'll last in people's memories for a lifetime."
Setting out the club's plans before the next season, he said managers had to look at players' contracts and the stadium needed infrastructure improvements of up to Â£5m, including floodlights, media facilities and improvements to the players' area.
"We're going to have some of the biggest clubs in the world visit the city and no doubt all of them will be bringing huge numbers of fans as well - so this is a special time, not just for the football club but for the whole city."
The club last played in the old First Division in 1983 - they reached the FA Cup Final that year but got relegated.
In 1997, the club had to sell the Goldstone Ground to pay off some of its debts.
It spent two seasons 70 miles away at Gillingham's Priestfield Stadium.
The club then moved into the Withdean Stadium in 1999 - a council-owned athletics track on the suburbs of Brighton.
Chairman Tony Bloom's arrival eight years ago paved the way for their new permanent home at the Amex stadium and the success that followed.
DJ and Albion fan Norman Cook, better known as Fatboy Slim, who lives in Hove, said he was going to enjoy the moment "from now until the middle of August".
"I'm going to make the most of it and enjoy it because reality will kick in.
"In the middle of August we could get pasted, but it's going to be part of a learning curve," he said.
Lifelong Albion fan Attila the Stockbroker - aka John Baine - said: "It's an incredible journey. Twenty years ago, bottom of the league with no ground. Now we're going into the Premier League."
Ingram Losner, who has supported Brighton since he was a child and travelled from the US to watch the match, said: "The atmosphere was just building up throughout the day, and the actual game and the celebrations after were just nothing like I'd ever experienced."
On Twitter, Brighton council's leader Warren Morgan tweeted the club's chairman Tony Bloom should be given the freedom of the city.
Hove MP Peter Kyle posted that he was "bursting with pride".
Nothing will ever top Brighton and Hove Albion's promotion to the Premier League, the club's former defender Adam Virgo has said.