The six male athletes also said they engaged in disruptive behaviour.
The admission follows a report that assessed Australia's poor swimming performance at the London 2012 Games which pointed to a "toxic" team culture.
Swimming Australia said the swimmers will face an inquiry panel.
Stilnox, used to treat insomnia, is not considered a performance enhancer and is not a banned substance, but the Australian Olympic Committee (AOC) put it on the prohibited list ahead of the Games.
In their statement, the six members of the men's 4x100 freestyle relay team said the prescriptions "were filled in Australia before their departure to their staging camp in Spain and prior to the AOC's announcement the drug was (prohibited)".
But they "acknowledge that at the time Stilnox was consumed it had been recently prohibited for use".
"We stand here collectively to confirm that we did take part in a bonding exercise during which members of the relay team took Stilnox," the six swimmers said.
"We acknowledge by our recent action in continuing a recent tradition in the Australia swim team, we have let ourselves down and the people who have supported us."
All but one team member took the medication, they said. Then they engaged in pranks they described as "childish" and "stupid", such as knocking on the doors of other athletes and making phone calls. They said they were asleep by 22:30.
"I think one of the reasons I agreed to go along with it was all the pressure I was under," one of the swimmers, James Magnussen, said. "Completely inappropriate in hindsight."
The relay team, nicknamed Weapons of Mass Destruction before the Olympics, came in fourth in the event, in which they had been expected to deliver a medal.
Magnussen, a favourite to win the men's 100 metres freestyle, won a silver.
Their admission follows the release of a review of the sport after the country's worst Olympic swimming performance in two decades.
It found that amid a lack of leadership a "toxic" team culture developed that led to bullying and misuse of prescription drugs, and that standards and discipline were "too loose".
Australia's swimmers won just one gold medal at London 2012, far short of previous medal hauls at other recent Olympics.
Members of Australia's Olympic swimming team said they used the sleeping medication Stilnox during a "bonding" session before the Games last year.