Screen grabs showing the website with messages from gay activists are being circulated on social media sites.
In one, the prime minister apologises to all homosexuals living in Uganda and gives his support to a gay pride march.
Homosexual acts are illegal in Uganda and gay people have faced physical attacks and social rejection.
Earlier this year, a controversial anti-gay bill, which proposes to increase the penalties for homosexual acts from 14 years in jail to life, was re-tabled in the Ugandan parliament.
The bill was first introduced in 2009 but never debated - and the MP backing the legislation says a clause proposing the death penalty will be dropped.
It originally said those found guilty of "aggravated homosexuality" - defined as when one of the participants is a minor, HIV-positive, disabled or a "serial offender" - would face the death penalty.
The Uganda Justice Law and Order Sector website was also reportedly taken over by hackers this week.
Ambrose Ruyooka, commissioner for the Information Communications Technology ministry, told the BBC the problem had now been rectified.
The government has created a directorate of security for all government websites and tightened cyber laws to counter rising cyber crime, he said.
An activist named @DramaSett3r on Twitter is said to be behind the attacks.
The operations were carried out by a group known as The Elite Society and the Anonymous hacktivist community, the activist said.
On Wednesday, the official website of Prime Minister Amama Mbabazi quoted him as saying: "We have got to expel the narrow mindedness from this country, and begin afresh, starting with a full and formal apology to all homosexuals living in Uganda today," screen grabs on Twitter and Facebook show.
A message allegedly posted by the hackers on the prime minister's site on Tuesday said: "Your violations of the rights of LGBT [lesbian, gay, bisexual and transgender] people have disgusted us.
"All people have the right to live in dignity free from the repression of someone else's political and religious beliefs."
The East African nation is a largely conservative society and many people condemn homosexuality both as unAfrican and unChristian.
Western donors have recently said they could withdraw aid from those countries, including many in Africa, which do not recognise gay rights.
The Ugandan prime minister's website was attacked by hackers on Tuesday and Wednesday, a government official has confirmed to the BBC.