King Salman's decree also means Prince Mohammed bin Salman, 31, will become deputy prime minister while continuing as defence minister.
Prince Mohammed bin Nayef, 57, has been removed from his role as head of domestic security, state media say.
He has pledged allegiance to the new crown prince, his younger cousin.
Saudi Arabia has typically been ruled by kings in their 70s or 80s.
Prince Mohammed bin Salman's rapid ascent is seen by the younger generation as a sign that things are changing.
Before his latest promotion, he was responsible for leading Saudi Arabia's war in Yemen, overseeing the kingdom's energy policy and economic reform.
He must have already ruffled a lot of feathers in a royal family that was used to being presided over by a succession of elderly men, the BBC's Middle East editor Jeremy Bowen reports.
His father King Salman is 81 and reportedly not in the best of health, our correspondent adds.
Prince Mohammed bin Salman could, potentially, lead Saudi Arabia for decades, heralding a long period of internal stability.
However, the new crown prince is close to US President Donald Trump, and could ratchet up pressure against Iran - which would raise tensions in the region.
The rise of Prince Mohammed bin Salman has been meteoric.
When I met him in Jeddah in 2013 he described himself as simply "a lawyer". Today he is a heartbeat away from ruling the most powerful country in the Arab world.
Despite being the driving force behind Saudi Arabia's damaging and inconclusive military campaign in Yemen, he is largely popular at home, especially with younger Saudis.
He has swept away many of the ineffective timeservers in government offices and replaced them with young, Western-educated technocrats. He has set out a possibly over-ambitious development plan, "Vision 2030", and announced plans to sell off part of the vast state-owned oil company, Saudi Aramco.
He has also forged ties with Washington and the Trump administration.
But his biggest and most risky move may yet prove to be his bid to curb the power of the conservative religious establishment. Washington likes this move, but others closer to home do not.
The founder of Saudi Arabia, King Abdulaziz (known as Ibn Saud), had dozens of sons, and there are as many as 15,000 princes and princesses in the royal family.
In 2006, a decree was passed setting up a committee responsible for electing the king and crown prince. The committee is made up of senior princes of the Al Saud royal family and is known as the Allegiance Council.
Mohammed bin Salman's new promotion to crown prince was approved by 31 out of 34 members of the council.
Analysts say those in power tend to appoint their sons to roles that will enhance their chances of being promoted - so when monarchs and crown princes change, sons can come into and out of favour.
King Salman, a son of King Abdulaziz, acceded to the throne in January 2015 after the death of his half-brother Abdullah bin Abdul Aziz.
He launched his first major cabinet reshuffle a few months later, promoting Mohammed bin Nayef to crown prince and Mohammed bin Salman, who was relatively unknown at that time, to deputy crown prince.
Experts say the second reshuffle has reduced uncertainty over the line of succession, and means the king has averted a potential power struggle between his son and his nephew.
The decree promoting Mohammed bin Salman includes a clause preventing him from appointing one of his own sons as his heir.
It is the most influential Arab country - and a key ally of Western powers including the US and the UK.
The kingdom is also the world's second largest oil producer - and the Saudi royal family is thought to be one of the richest ruling families in the world.
It is guardian of the two most holy sites in Islam, and regards itself as leader of the Sunni community worldwide.
The kingdom is a key player in Sunni attempts to block Shia Iranian influence in the Middle East.
It has launched air strikes on Yemen to fight Houthi rebels, who Saudi Arabia believes are backed by Iran. However, the campaign has been criticised for human rights violations - including the killing many civilians.
Recently, Saudi Arabia also joined efforts to impose a blockade on Qatar.
Prince Mohammed bin Nayef was the kingdom's security chief for many years and was known for his strong stance against jihadist militants.
As deputy prime minister and interior minister, he headed up the counter-terrorism portfolio, and also had good relations with Saudi Arabia's Western allies.
State media broadcast footage showing Mohammed bin Nayef meeting Mohammed bin Salman after the succession was announced.
The former crown prince said "I am content" and "I am going to rest now. May God help you", while the new heir replied "May God help you. I will never do without your advice".
Reaction to the appointment of the new crown prince was largely (and unsurprisingly) celebratory, although Saudi Arabia's regional rivals took the opportunity to criticise the kingdom.
Saudi-funded Al-Arabiya TV ran blanket coverage and emphasised the positive effect the appointment had on the country's stock exchange.  Similar reports appeared in other friendly Arab states.
But in Iran - Saudi Arabia's bitter rival - several news outlets described the appointment as a "soft coup".
Similar language was used by the Pro-Qatari HuffPost Arabi, which called the appointment "a coup in the royal palace".
Syrian state media's largely neutral report came with a considerable sting in the tail.
Coverage ended with a reminder to viewers of a "tyrannical regime" which "lacks the most basic democratic principles" and "supports terrorism".
Jihadists on social media celebrated the reshuffle, anticipating that the changes would pave the way for internal strife within the House of Saud and eventually bring about its downfall.
Saudi Arabia's king has appointed his son Mohammed bin Salman as crown prince - replacing his nephew, Mohammed bin Nayef, as first in line to the throne.