More recently, women's football soared in popularity after the nation's female team won the World Cup in 2011 and Olympic Games silver in 2012.
Now rugby union is looking for a similar breakthrough into the national consciousness, when its World Cup is hosted by Japan in 2019.
With the Olympic Games being held in Tokyo a year afterwards, the end of the decade is shaping up to be a major sporting time for Japan.
"These two events coming together means the Japanese citizens' interest in sport has already gone up, and should increase rapidly nearer the events," Koji Tokumasu, general manager of the Japan 2019 Rugby World Cup Organising Committee, tells BBC News.
He was recently in the UK for his nation's friendlies against Scotland and Gloucester, where Japan played with tenacity but lost both matches.
Japan - known as the Cherry Blossoms or Brave Blossoms - are traditionally the strongest rugby union power in Asia.
But Mr Tokumasu says in Japan, where baseball is the most popular sport and football second, rugby union lags behind in about 10th spot.
"This is something we have to work on over the next six years," he says.
"We want to approach our existing fans - we do already have some rugby fans in Japan, and activate them to spread the World Cup message."
He says this hardcore fan-base can use digital media to publicise the rugby experience and the excitement surrounding the sport, including the thrill of match day.
They will also be able to explain the intricacies of the sport, including the scoring system.
"We want to raise interest and passion in the sport by bringing quality teams like New Zealand to our fans, who can then pass on the message to their family, friends and colleagues," says Mr Tokumasu.
When Japan hosted world champions the All Blacks earlier in November, the 25,000 tickets were sold out after half an hour. The host nation, not surprisingly, lost that encounter, but there has been playing success this year.
Japan won this year's HSBC Asian Five Nations tournament, and also beat a Wales second string team in the summer. That victory was their first win over a major rugby nation in eight decades of trying.
To show it was no fluke, they have also beaten the US and Canada this year.  In addition, this month Japan beat Russia, thrashed Spain, and also won the 2013 HSBC Asian Rugby Sevens.
The first stage of 2019 preparation is already under way, with the release of the criteria for would-be World Cup host cities and stadiums.
Organisers want to select between 10 and 12 match venues before March 2015, for the 48-game tournament.
"The reason we have chosen that date is because we want the successful host venues to then be able to attend that year's World Cup in England to see and experience how the event is staged," says Mr Tokumasu.
"At present we are visiting cities and telling them how exciting hosting the 2019 World Cup can be for them."
In Tokyo, a showpiece 80,000-capacity Olympic Stadium is being built in time for the 2019 rugby event, and in other match venue centres local government will pay for stadiums.
In many cities stadiums have remained underused since the 2002 World Cup. Mr Tokumasu says: "We would like it if we could use some of the those stadiums, but we cannot force those cities to take part - they have to express an interest."
As well as increasing awareness, another major pre-2019 aim is to increase playing participation.
"This is a major aim, and our government supports this, they think that sport is good for health," he says.
"We are going to take rugby into primary schools, first going into 5,000 schools, and by 2019 hoping to have spread the word in all 22,000 primary schools.
"Boys and girls can learn together and we will be introducing tag [touch] rugby, so that teachers won't be worried about children getting hurt.  So by 2019 a lot of young people will know all about rugby."
That work among the young should widen the base of the next generation of rugby players in a country where 95% of participants are university graduates.
Rugby playing is also very intertwined at present with the top end of the corporate workplace,
In the premier domestic competition, the Top League, the clubs are owned by large Japanese corporations such as Toshiba or Panasonic, and when a player's career comes to an end, they then usually take up a management role with the parent company.
The government also wants to see people taking part as event volunteers, following a successful programme at the 2002 global football event, and ahead of Tokyo 2020.
But he stresses again that the 2019 organisers want to bring rugby to a wider audience.
"In Japan people tend to stay at home to watch the very best of global or national sports on TV or the internet, rather than actually attending live events - we want to overcome this and attract people to rugby games," he says.
"We are going to produce more opportunities for people to attend a match - we really want to make the best use of the next six years "
Football - or "socca" as the Japanese call it - was given a boost in the baseball-mad land of the rising Sun when the World Cup was held there in 2002.