All Japan's nuclear plants were gradually shut down after a series of meltdowns at the Fukushima plant sparked by the tsunami and earthquake.
But after passing stringent new safety tests, Kyushu Electric Power restarted the number one reactor at its Sendai plant on Tuesday morning.
There is still strong public unease about a return to nuclear power.
Protests have been taking place outside the Sendai plant and at Prime Minister Shinzo Abe's residence in Tokyo, about 1,000 km (600 miles) away.
The BBC's Rupert Wingfield-Hayes in Tokyo says that after being told a disaster like Fukushima could never happen, public confidence has been shaken.
A total of 25 plants have applied to be restarted, he says, but all are facing legal challenges from concerned locals.
What is happening inside the Sendai reactor?
Kyushu said reactor No.1 at Sendai began operating again at 10:30 local time (01:30 GMT).
TV images showed the plant's control room as workers turned the reactor back on.
Kyushu Electric spokesman Tomomitsu Sakata said the reactor had gone back online without any problems.
It will be about 24 hours before a full reaction takes place, and the plant is expected to start generating power by Friday. It will reach full capacity some time next month.
Prime Minister Abe said on Monday that the reactors had passed "the world's toughest safety screening".
"I would like Kyushu Electric to put safety first and take utmost precautions for the restart," he said.
Since shutting down all nuclear plants, Japan has been relying on imported fossil fuels for its energy, at huge expense. The government has said nuclear power must resume to cut both import bills and growing CO2 emissions.
Experts have also warned that reactors left idle for years tend to experience teething problems and that such a mass restart of dormant reactors has never been attempted before, says our correspondent.
Japan's Nuclear Regulation Authority approved two reactors at the Sendai plant last September under stricter safety rules. The second reactor is due to be restarted in October.
More than $100m (Â£64m) has been spent on fitting new safety systems at the Sendai plant.
But local residents say the new safety regulations are not stringent enough - they are worried about potential dangers from active volcanoes in the region.
Protesters were rallied by Naoto Kan, prime minister at the time of the Fukushima crisis, who told the crowd: "We don't need nuclear plants."
He said the Fukushima disaster had "exposed the myth of safe and cheap nuclear power, which turned out to be dangerous and expensive".
One of the most powerful earthquakes ever recorded struck off the coast of Japan in March 2011, triggering a huge tsunami which damaged the Fukushima nuclear plant, leading to meltdown.
Almost 16,000 people died and more than 2,500 are still listed as missing, though none of the deaths has been linked to the nuclear disaster.
Some 160,000 people were evacuated from the surrounding areas in the following weeks - continuing high radiation levels mean most have never been able to return home.
Japan has restarted its first nuclear reactor under new safety rules following the 2011 Fukushima disaster.