The 53-year-old man, identified as Ramu Chavan, was accosted by traffic police after they received multiple calls from people who saw him.
Police told BBC Hindi that Mr Chavan had confessed to beheading his wife but did not divulge a motive.
Police suspect he is mentally unstable and are seeking more information.
He worked as a watchman at a building currently under construction, police said.
Videos and images of Mr Chavan walking around the city with the severed head are being circulated on Indian social media.
Police told BBC Hindi that he appeared to be walking towards the police station even before he was accosted by officers.
Meanwhile the relatives of the murdered woman, identified as Sonubai, have arrived in the city, and police are trying to get more information about Chavan from them.
A man in the western Indian city of Pune has been held after carrying his wife's severed head down a busy road, police say.