Prime Minister Narendra Modi described the accord between his government and the National Socialist Council of Nagaland as "historic".
The rebels in Nagaland state have been demanding an independent homeland for two million Naga tribespeople.
The group, in India's north-east, have been fighting for more than 60 years.
"I have the deepest admiration for the great Naga people for their extraordinary support to the peace efforts," Mr Modi said, after the agreement was signed at his office in Delhi on Monday.
"Our oldest insurgency is getting resolved, it is a signal to other smaller groups to give up weapons," he added.
The prime minister earlier said that developing India's north-east was a priority for his government. Talks between India's government and the rebels had been taking place since 1997.
Correspondents say many hope the deal will pave the way for similar agreements with other rebel groups in the region.
The agreement was signed by its leader Thuingaleng Muivah, who once led rebel raids from neighbouring Myanmar in what is often described as India's first ethnic rebellion.
The terms of the peace agreement were not immediately known.
The Naga mainly live in Nagaland and also in the states of Manipur, Assam and Arunachal Pradesh.
India's government has signed a peace deal with a leading Naga separatist group, bringing to an end one of the country's oldest insurgencies.