The Lord Advocate said he had informed First Minister Nicola Sturgeon of his intention to quit the post after five years in the job.
He said it had been a "real privilege" to lead the prosecution service in Scotland and to provide legal advice to the Scottish government.
The law officer added that it was time to "step down and do other things".
In a statement, Mr Mulholland said: "In recent years the Crown has embedded specialisms in the way it does its job.
"Our expertise in handling offences including rape, domestic abuse, serious organised crime, counter terrorism and cold cases has helped us become one of the most effective prosecution services in the world and given victims greater confidence to report crimes.
"It's been an honour to do this job working with so many dedicated and talented people to deliver justice in some of the most demanding and challenging of cases."
Nicola Sturgeon described Mr Mulholland as "an outstanding Lord Advocate".
In a statement, she said he had carried out his role with "dedication, energy, integrity and intellect".
She highlighted his work in leading Scotland's first successful "double jeopardy" murder prosecution, against the Worlds End murderer Angus Sinclair.
The first minister also praised his role in establishing a National Sexual Crimes Unit and appointing Scotland's first specialist prosecutor to deal with domestic abuse.
"It is clear that he has worked to bring about change to ensure that the system makes a real difference to people's lives, and his dedication to the law and his compassion for others has been behind that drive," she added.
"Frank has made a substantial contribution to the law and Scottish society."
Mr Mulholland served as Solicitor General until 2011, when he succeeded Dame Elish Angiolini as Lord Advocate.
He has faced some criticism during his tenure, most notably over the Crown's decision not to press charges over the Glasgow bin lorry crash.
The families of some of the victims of the 2014 tragedy are now pursuing a private prosecution against the driver.
It is understood that he will formally step down when his successor is appointed by the Queen.
Scotland's top law officer, Frank Mulholland, will step down after the Scottish Parliament elections in May.