The painting, which had been on loan at The National Gallery for 30 years, was at risk of being sold abroad.
"It's one of the quintessential images of 19th century British art and it's worth every penny," Tate Britain's Director Nicholas Serota told the BBC.
The work will be shown at Tate Britain until the end of the year before travelling around the UK.
"This is the one of Constable's most important paintings," Sir Nicholas told the BBC's Will Gompertz. "He regarded it as one of his masterpieces and always wanted it to be in the national collection."
Salisbury Cathedral from the Meadows was acquired for £23.1m, but could have sold for as much as £40m at auction.
By Will GompertzArts editor
£23.1m might be a record for a Constable but it's worth every penny: a bargain compared to what's being paid for some second rate modern art.
It is arguably the most accomplished work by one of the finest, most exciting painters of the Romantic period.
Painted shortly after his wife's death, the "iconic" image was secured through major grants from the Heritage Lottery Fund, the Art Fund, The Manton Foundation and Tate Members.
Sold by the children of the late Lord Ashton of Hyde, the work was at risk of going abroad had the Tate not been able to put the money together.
"I know there was a major American museum that asked for first refusal," said Sir Nicholas.
The acquisition was part of a partnership between five national and regional galleries:  Amgueddfa Cymru - National Museum Wales; the National Galleries of Scotland; Colchester and Ipswich Museums; Salisbury and South Wiltshire Museum; and Tate Britain.
The partnership will enable the work to go on "almost constant" view across the UK.
Jenny Abramsky, from the Heritage Lottery Fund, said they were "proud" to be a major funder of the masterpiece with a £15m investment.
"It is unimaginable that this particular painting might have ended up anywhere other than in a UK public collection," she said.
Constable's The Lock became one of the most expensive British paintings ever sold when it fetched £22.4m at auction in July 2012.
Salisbury Cathedral from the Meadows is one of a series of monumental six-foot canvases painted by the artist, a scale he reserved for his finest compositions.
John Constable's 1831 masterpiece Salisbury Cathedral from the Meadows has been bought by the Tate for £23.1m.