David Mundell argued that it was time for Holyrood to "end its dominance" over Scottish life.
And he said Westminster was "setting the pace" in breaking up the "central government monolith".
The Scottish government said it had a "dynamic agenda of shifting power".
It also said Mr Mundell had "no credibility" when it came to devolution, given his opposition to amendments to the Scotland Bill.
And it urged Mr Mundell to look at how Westminster was handling local authority affairs south of the border before criticising the situation in Scotland.
Mr Mundell was speaking in Glasgow days after council leaders warned that Scottish finance secretary John Swinney's latest budget plans would be "catastrophic" for jobs and services in Scotland's local authorities.
Mr Swinney has allocated Â£10.3bn for local government as part of his spending plans for 2016, which local government body Cosla said amounted to a Â£350m cut.
There is almost an iron rule of local government - when in opposition people call for more decentralisation, and when in government they tend not to decentralise as much as they claimed they would.
It strikes me that in their different ways, although both parties are in government, they are playing out that kind of line.
Both Mr Mundell and Mr Swinney are, in a sense, talking up what they have done that they think fits with the agenda of localism.
So David Mundell, and indeed the chancellor and the government in England, are talking about devolution to local authorities.
But there was no mention of the requirement, for example, that Manchester should have a mayor before it got those accumulated authorities for Greater Manchester.
That is throwing aside the democratic decision of people who lived in and around Manchester when they actually voted on having a mayor a while ago.
Similarly, Mr Swinney is talking about about giving the power to local authorities to "vary" business rates.  A more accurate word would have been giving them the power to "reduce".
If you were genuinely giving power and authority and accountability to local authorities, you would enable them to reduce and/or increase business rates.
So (both governments) tend to play up what they are doing well and be quiet about these areas where centralisation and control sits at the heart of what is going on.
The Scottish secretary argued that councils' concerns about the future mean it is time for an "honest and frank" debate about the way forward for them.
His proposals would see local authorities given greater control over areas such as health, transport and policing, with towns and cities also having more say over their own affairs.
Mr Mundell said: "The interminable debate about Scotland's constitutional place within the UK has drowned out debate about how power and responsibility is distributed within Scotland.
"The referendum was decisive and an obsession with independence can no longer be an excuse to ignore this issue. It's time for Holyrood to step up and send real powers to the people.
"There is a revolution going on in local government across the rest of the United Kingdom, with local areas regaining power and responsibility at an unprecedented rate. Scotland cannot afford to be left behind."
Mr Mundell stressed that he was not advocating a reorganisation of local government, but rather bringing an end to what he sees as a "one-size-fits-all" approach for councils.
He urged the Scottish government to "follow the lead of the UK government" in devolving powers to cities, counties and towns.
Mr Mundell said that "the Northern Powerhouse is breaking new ground" and the "Midlands Engine is gaining pace" south of the border.
He added: "On the crucial issue of breaking up the central government monolith, it's now Westminster which is setting the pace and leading the way.
"There is now real risk that Glasgow, Edinburgh, Aberdeen, Dundee, and indeed the towns and counties of Scotland as a whole, will be left behind - stuck in a 1990s time-warp of centralised, Holyrood-dominance."
The Scottish government has said that it takes a "partnership approach" to local government and has a "commitment to subsidiarity".
It points to legislation such as the Community Empowerment Act and Land Reform Bill as evidence that it is giving local communities a bigger say in their own affairs.
Mr Swinney told the BBC's Good Morning Scotland programme: "Just in the last few weeks I took a decision to enable local authorities improve their business rates performance by reducing business rates, making their areas more competitive and attracting more business into their locality.
"I took that decision and implemented it in October. That decision will not be implemented in the rest of the United Kingdom until 2020.
"So I think before Mr Mundell starts giving us a lecture in Scotland about how we are exercising these responsibilities, perhaps his own government should get its own house in order."
But David O'Neill, president of council body Cosla, said that Scotland had become "one of the most centralised countries in Europe" over the past 50 years.
He said that this had led to "big and expensive inequalities in Scottish society".
Mr O'Neill added: "This top down approach has had its time, and it's good news that political parties of all kinds are now waking up to the need for a more local future.  David Mundell's comments are further evidence of the growing calls for change."
Scottish cities, towns and villages risk being left behind other parts of the UK unless they are given greater control over their own affairs, the Scottish secretary  has claimed.