The Lesedi la Rona, almost the size of a tennis ball, was unearthed in Botswana in November.
The rough diamond is 1,109 carats and believed to be more than 2.5 billion years old.
It had been expected to sell for more than £52m ($70m) but the highest bid was about £45m ($61m).
The auction at Sotheby's was the first time a rough diamond of such a size has gone on public sale.
Why have so many huge diamonds been found recently?
The auction house had called its discovery "the find of a lifetime" and earlier said that "every aspect of this auction is unprecedented".
The Lesedi la Rona, which means "our light" in the Tswana language spoken in Botswana, was discovered by Lucara Diamond Corp's Karowe mine.
After it failed to sell, the Canada-based company - which has retained possession of the diamond - saw its stock fall more than 14% on the Toronto Stock Exchange.
According to a study by the Gemological Institute of America, the rough diamond's colour and transparency "exemplify" type IIA diamonds.
Stones in this group are said to be "the most chemically pure and often show extraordinary optical transparency".
In terms of its size the rough is exceeded only by the Cullinan Diamond, mined in South Africa in 1905 and presented to King Edward VII.
That 3,106-carat diamond was cut into nine stones, many of which are in the British Crown Jewels.
The world's largest uncut diamond has failed to sell at a London auction after bids did not reach the minimum reserve price.