Abdul Nabi, 30, is described as black - of Asian appearance - about 5ft 7in tall and skinny, with short curly hair.
Anyone who spots him has been asked to call 999.
Officers were searching land around Midpark Hospital and Maidenbower near the Crichton university and business complex. Premises in the immediate area were alerted.
The search subsequently moved on to the Calside and Georgetown areas of the town and pupils at Georgetown Primary School had their sports day interrupted.
Police dogs and the police air support unit also joined efforts to try to trace Mr Nabi.
"This is a missing person inquiry and we want the public to be alert, however not alarmed," said a spokesman.
"We are asking that if he is seen he is not approached and you contact the police."
Insp Rory Caldow said officers had been carrying out a large-scale search since about 12:30.
"We did have early concerns that he may have been in possession of a knife, however we understand that this is not now the case," he said.
"Mr Nabi is a vulnerable missing man and our concern for the safety of the public centres on the uncertainty of his reaction should he be approached.
"We ask the public to be vigilant and report any sightings, without making an approach to him, on the 999 emergency number."
Police have warned the public not to approach a man reported missing from a psychiatric hospital in Dumfries.