In the 1960s and 70s, the spirits of scores of convicted Japanese war criminals were "enshrined" there.
The most controversial were the 14 "Class A" war criminals, including wartime leader Hideki Tojo, who were "enshrined" in the late 1970s.
These men were the ones who ordered and oversaw Japan's brutal war in China and South East Asia.
Yasukuni Shrine
It was a war in which millions died, in which there were widespread massacres of civilians, in which rape was routinely used as a weapon and where Japan used chemical and biological warfare against civilians.
So if the shrine is so offensive to China and South Korea why did Mr Abe go?
Firstly, because he wanted to.
Close observers of the Japanese prime minister say he is at heart a nationalist and a historical revisionist.
He believes the trials that convicted Japan's wartime leaders were "victors' justice".
His own grandfather Nobusuke Kishi served in the war cabinet and was arrested by the Americans on suspicion of being a Class A war criminal. He was later released without charge.
But the stain of association with Japan's war crimes in China never completely went away.
Secondly, Mr Abe's support base comes from the right wing of the Liberal Democratic Party.
According to Professor Jeff Kingston of Temple University in Tokyo, Mr Abe is "showing he is a tough guy", that he is not afraid of China. It is something that plays very well to his base.
But there is perhaps a bigger goal that Mr Abe has in mind.
He wants to radically revise Japan's post-war constitution.
This, too, is a long-held dream that started with his grandfather in the 1950s.
Mr Abe believes he is the man to complete the historic task of getting rid of the hated "peace constitution".
Like many on the right here, Mr Abe believes that constitution was forced on Japan by America and is a humiliation.
It imposes not only pacifism, but also Western notions of human rights and civil liberties. It rejects Japan's uniqueness in favour of "universal values".
Mr Abe would like to change a lot of this. But it will be very hard. And so he will need some help.
"Abe has provoked China, and China has reacted just as Abe wanted it to," says Prof Kingston. "There is a shrewd political calculus at work here."
What he means is that having an external threat in the shape of big and frightening China may be just what Mr Abe wants to help push through his controversial nationalist agenda at home.
Whatever Shinzo Abe says, any visit to the Yasakuni shrine by a Japanese prime minister is deeply political and sure to cause offence.