They have been taken to UN centres in Dohuk in Kurdish northern Iraq.
It is unclear whether they escaped in Iraq or were freed, as the UN declined to give more information to avoid jeopardising future releases.
IS killed and enslaved thousands of Yazidis after seizing the northern town of Sinjar in 2014.
Kurdish Peshmerga forces regained control in 2015 but many Yazidis were held captive by IS elsewhere as the group took over large swathes of northern Iraq.
The 36 survivors include men, women and children who were enslaved, the UN Office for the Co-ordination of Humanitarian Affairs said.
They reached Dohuk two nights ago where they are being housed in centres established by the UN Population Fund.
There, they are being reunited with family members and offered care including clothing and medical and psychological aid, the UN said.
The women and children are being cared for at dedicated service points and will be referred for more specialised treatment.
The UN's humanitarian co-ordinator for Iraq, Lise Grande, said the UN was going to do "everything possible" for the rescued Yazidi.
"What these women and girls have endured is unimaginable," she said.
Who, What, Why: Who are the Yazidis?
The UN - which has accused IS of committing genocide against the Yazidi - estimates that 1,500 women and girls are still being held and could be suffering protracted sexual abuse.
A UN report released last year said the extremists had subjected captured Yazidis to the "most horrific of atrocities", killing or enslaving thousands, warning IS aimed to completely erase their way of life.
In March. lawyer Amal Clooney urged the UN to back a UK-led investigation into the crimes committed by IS.
IS has been under increasing pressure in Iraq, losing much of the territory it seized in 2014.
Iraqi forces have recaptured most the city of Mosul from IS, but they are still trying to push the group out of the Old City district in the west.
Thirty-six members of the Yazidi religious minority are free after nearly three years in the hands of so-called Islamic State (IS), the UN says.