Police in Madhya Pradesh state told reporters sedition was "hard to prove".
The men have been charged with "disturbing communal harmony" instead.
They were arrested after their Hindu neighbours complained they had burst firecrackers during the game, which saw Pakistan beat India by 180 runs.
Senior police officer RR Parihar said that an additional charge of conspiracy against the men would also remain.
"It's difficult to prove the sedition charge. Moreover, none of them has a criminal background," he told reporters.
The accused were sent to jail in the city of Khandwa on Tuesday.
Meanwhile, Amnesty International has called for the immediate release of the men, calling the arrests "patently absurd".
Sedition is one of the most serious charges under the Indian penal code.
People charged with sedition have to surrender their passports, are not eligible for government jobs, must appear in court as and when required, and spend money on legal fees.
If found guilty, they could be sent to prison for life.
The India Today website quoted police as saying that the men were charged because of the anti-India slogans and not because they were cheering for Pakistan.
However family members of the accused have denied the charges against them.
"We don't know who burst crackers in support of Pakistan. All these charges against my son are fabricated," Gulzar Tadwi, the father of one of those arrested, told BBC Hindi.
Sikhander, who uses one name, said police had come in the night and taken away his sons without even telling them why. He also denied that anyone in his house had cheered for Pakistan.
This is not the first time Indian Muslims have got into trouble for cheering for the Pakistan cricket team.
In 2014, 66 Muslim students from Indian-administered Kashmir were kicked out of their university in the northern state of Uttar Pradesh and charged with disturbing communal harmony.
And in 2016, police were sent into a university in Indian-administered Kashmir after clashes between students from the state, the only Muslim majority one in India, and other parts of the country.
Indian police have dropped sedition charges against 15 Muslim men arrested for allegedly shouting "anti-India and pro-Pakistan" slogans during the Champions Trophy cricket final.