After taking part at London 2012, the four national football associations could not agree on sending a British women's team to the Rio Games.
A place was earned by England finishing third in the 2015 World Cup.
"To turn it down is a great shame. It's something we may look at in the future and try to compete in," Allardyce told BBC 5 live's Sportsweek.
"When you see the delight on Justin Rose's face after winning the gold medal in golf it shows what it all means."
England's Football Association had put forward the idea of sending Great Britain teams to the Olympics, but Fifa said it would need the agreement of the ruling bodies in Scotland, Wales and Northern Ireland, who were against it.
The other home nations fear such a move could affect their independent status within the sport's world governing body, Fifa, and at international tournaments such as the World Cup.
Team GB entered men's and women's sides at London 2012, with Stuart Pearce coaching the former and Hope Powell the latter. Both teams were knocked out in the quarter-finals.
For European nations, qualification for the Olympic football competition is via the Uefa Under-21 Championships for men and the World Cup for women.
England were the only home nation to compete at the Under-21 Championships, but failed to secure the required semi-final berth.
British Olympic Association vice-chairman and former sports minister Sir Hugh Robertson believes politics in particular has denied the women's game a perfect platform to build on the interest it received at London 2012.
A total of 154,998 people turned out to watch the four Team GB women's matches four years ago.
"From the British Olympic Committee's perspective, we would love to see Team GB football," Robertson told Sportsweek.
"It is particularly a powerful tool to promote the women's game.
"The tragedy is that the politics of football administrators impact on the athletes because women football players would want to be at the Olympics."
England manager Sam Allardyce would welcome Team GB entering football teams at future Olympics.