Justice Secretary Michael Matheson has directed Her Majesty's Inspectorate of Constabulary in Scotland to investigate the actions of undercover officers.
A review is ongoing in England and Wales, but the UK government has refused to extend it to Scotland, leaving Mr Matheson "disappointed".
Victims of police spying had called for a Scottish inquiry to be set up.
Holyrood's parties had been united in calling for an extension to the Pitchford Inquiry, which was set up in England and Wales to investigate allegations of misconduct by undercover officers.
There are allegations that undercover Metropolitan Police officers had fathered children and caused miscarriages of justice, with claims of this activity extending to Scotland.
Mark Kennedy, one of five officers known to have had relationships with women while undercover, is alleged to have infiltrated environmental groups ahead of the G8 summit at Gleneagles in 2005.
Labour MSP Neil Findlay wrote directly to the policing minister, Brandon Lewis, but was told that it was "not possible" to extend the probe north of the border for "a number of reasons".
Mr Matheson said he was "firmly of the view" that extending Pitchford would be the "right thing to do", and said he was "disappointed" with the UK government's refusal.
He said: "Given the operational extend of the units involved, the Scottish government believes that a single coherent inquiry is the most effective approach to provide a comprehensive investigation into these activities.
"However, in light of the Home Office's decision I have today directed Her Majesty's Inspectorate of Constabulary in Scotland to undertake a strategic review of undercover policing in Scotland."
The justice secretary said this investigation would cover "the extent and scale of undercover policing in Scotland" dating back to 2000, as well as operations carried out by the Met Police units under investigation by Pitchford.
He added: "This review will also give independent assurance on the operation, procedures and safeguards in place by Police Scotland in relation to undercover policing."
The Scottish Liberal Democrats welcomed the news, with Liam McArthur describing the probe as "nothing less than required".
But Mr Findlay said it was "minor progress", calling it "the police reviewing the police" rather than the public inquiry he had called for.
Victims who are participating in the Pitchford Inquiry had hit out at the decision not to extend it to Scotland, calling on Holyrood to order its own probe.
A group of 17 victims issued a joint statement after Mr Lewis refused to extend the inquiry saying they were "extremely frustrated", saying it "sets the inquiry up to fail before it even begins".
One of the group, Merrick Cork, added: "It's outrageous that Theresa May thinks victims of police abuse deserve justice in England but not in Scotland.
"The police admit English officers committed human rights abuses against citizens on Scottish soil. It's absurd to expect public trust in police when abuses of power are swept under the carpet, and it's a snub to Scotland to say it doesn't matter there."
Mr Lewis defended the limited scope of the inquiry, saying it was "extensive and complex" already, and needed to "proceed swiftly" without further delays.
He added: "The Home Office is confident the inquiry can both gain an understanding of historical failings and make recommendations to ensure unacceptable practices are not repeated without a need to consider every instance of undercover policing, wherever it was taken."
The Scottish government has ordered an independent review of undercover policing in Scotland.