Her Majesty's Inspectorate of Constabulary in Scotland (HMICS) will conduct the joint inspection with its counterpart in England and Wales.
From February, officers will analyse BTP's efficiency, legitimacy and leadership of the service.
They will also examine strategic issues caused by the proposed transfer.
BTP's inspection was commissioned by the parliamentary secretary of state at the Department for Transport (DfT) under the Railways and Transport Safety Act 2003.
A single joint report will be produced by inspectors following the examination which will be submitted to the DfT.
Derek Penman, HM Chief Inspector of Constabulary in Scotland, said: "In addition to assessing the performance of British Transport Police in Scotland, it is my intention to use this joint inspection to identify any strategic issues arising from the proposed transfer of its Scottish operations to Police Scotland and the Scottish Police Authority under the Railway Policing (Scotland) Bill.
"Our report and any recommendations or findings could inform future discussions about the implementation of this transfer."
BTP has about 230 officers in Scotland and a network of 11 offices within major railway stations.
The force currently polices all of Britain's railways, providing a service to rail operators, staff and passengers.
Police Scotland is the largest force in the UK outside London, with more than 17,000 officers.
Scotland's Justice Secretary Michael Matheson decided in 2015 that Police Scotland should take over railway policing when the function is devolved.
The plans were criticised by rail unions who said skills and expertise in dealing with the specialist policing needs on the railways would be lost if the move went ahead.
A police watchdog will inspect British Transport Police (BTP) ahead of a proposed transfer of its Scottish operation to Police Scotland.