The sculpture of Themis - the goddess of justice - wearing a sari was less than six months old, but Islamist groups demanded its removal by Friday.
They claimed it hurt the religious sentiments of Muslims and it prompted mass protests in the capital, Dhaka.
PM Sheikh Hasina agreed to its removal, but secular groups opposed it.
Workers came with equipment and a crane at midnight to uproot the controversial statue, the BBC's Bangla service reported.
The statue is being removed to maintain peace, said its creator Mrinal Haque.
Analysts say this is a sign of the rising tension between Islamic conservatism and liberal values in Bangladesh.
Protesters have long asserted that the figure, which held the familiar sword and scales of justice in her hands, amounted to idolatry.
In February, conservative Islamist group Hefazat-e-Islam led protests and threatened to spread the demonstrations across the country if the statue was not removed.
Many of those protesting will have been followers of Hefazat-e-Islam, but observers have also pointed to increasing conservatism among the general public.
The tension between such forces and secular voices has been one of the defining themes in Bangladesh over the last few years.
Workers have begun to dismantle the statue of a Greek goddess from Bangladesh's Supreme Court complex, after an outcry from Islamists.