He has written to the parliamentary speaker criticising her for passing it in December without a quorum.
Homosexuals were "abnormal" or were so for "mercenary reasons" and could be "rescued", a local paper quotes his letter as saying.
The bill provides for life imprisonment for homosexual acts and also makes it a crime not to report gay people.
The promotion of homosexuality - even talking about it without condemning the lifestyle - would also be punishable by a prison term.
The BBC's Catherine Byaruhanga in the capital, Kampala, says the president is aware that if he signs the bill there will be an international outcry, which could see some countries suspend aid to the country.
His spokesman told the AFP news agency that Mr Museveni believes that gay people are sick but this does not mean they should be killed or jailed for life.
"What the president has being saying is that we shall not persecute these  homosexuals and lesbians. That is the point," said Tamale Mirudi.
By Catherine ByaruhangaBBC, Uganda
Uganda already has legislation banning gay sex, specifically between men, lesbianism wasn't considered under a provision of the 1950 Penal Code. The new law would add female-to-female sex to banned practices.
The Penal Code also never made it an offence for someone to identify himself or herself as a homosexual. It was the act that was illegal. Gay activists have been able to state their sexuality in public and advocate for their rights without being prosecuted.
This, legislators felt, endangered Uganda's culture and family structure, centred around marriage between a man and woman.
There's been a battle here, well captured in the international media, between gay activists and Evangelical Christians over the rights and wrongs of homosexuality.
So what MPs are trying to do is to create the "idea" of homosexuality in the law. Once you specify that homosexuality is wrong, you then ban its promotion.
If the law is passed, standing up saying "I am gay" would become illegal.
Citizens would also have to report anyone who they believe is gay to the police. And it would be illegal to provide advisory services to homosexuals.
He denied that the president had changed his mind under pressure.
"The president's position has been the same for a long time, nothing has changed,"  he added.
Our reporter says Mr Museveni is trying to reach a compromise with MPs, because if he refuses to sign the bill, parliament can still force it through with a two-thirds vote.
But in contrast to Nigeria, where earlier this month the president signed a bill banning same-sex marriages, gay groups and shows of same-sex public affection, Mr Museveni is politically strong and so more able to resist pressure from conservative groups, she says.
Mr Museveni said the bill was forced through despite his advice to shelve it until the government had studied it in depth, Uganda's private Monitor newspaper reports.
"Even with legislation, they will simply go underground and continue practicing [sic] homosexuality or lesbianism for mercenary reasons," he is quoted as saying.
The president's eight-page letter to speaker Rebecca Kadaga said they could be "rescued" by improving the economy.
He also disputed the view that homosexuality could be described as an "alternative sexual orientation".
"You cannot call an abnormality an alternative orientation. It could be that the Western societies, on account of random breeding, have generated many abnormal people," he said.
He said another reason women became lesbians was because of "sexual starvation" when they failed to marry, the Monitor reports.
Ugandan gay rights activist Pepe Julian Onziema told the BBC's Focus on Africa programme he had mixed feelings about Mr Museveni's comments.
"Him not assenting to the bill makes us happy but him calling us 'abnormal', 'nothing-doers', 'sexually starved', that is so derogatory," he said.
"It encourages the community to attack people like me."
There is meant to be a caucus meeting of ruling party MPs later this month to discuss the bill.
The government will try to persuade them to reject it, but some have already said they would go against their government's wishes, our correspondent says.
Human rights activists say the bill highlights the intolerance and discrimination the gay community faces in Uganda.
One gay activist was killed in 2011, although the police denied he was targeted because of his sexuality.
The bill has been condemned by world leaders since it was mooted in 2009 - US President Barack Obama called it "odious".
The private member's bill originally proposed the death penalty for some offences, such as if a minor was involved or the perpetrator was HIV-positive, but this clause has been dropped.
Correction 19 January: An earlier version of this story referred to Pepe Julian Onziema as a woman. The BBC is sorry for any offence caused.
Uganda's President Yoweri Museveni has refused to approve a controversial bill to toughen punishments for homosexuals.