The children were taken in a cross-border raid in the Gambella region last Friday, in which 208 people died.
The government has said members of the Murle community were responsible.
Flags have been flying at half mast in Ethiopia as the country mourns those who were killed.
Africa Live: News updates from around the continent
A government official in Gambella said that the abducted children would soon be rescued, the government-affiliated Fana Broadcasting Corporate reports.
South Sudan acting Foreign Minister Peter Bashir Gbandi is set to travel to Ethiopia's capital, Addis Ababa, on Friday to coordinate the mission, the AP news agency reports.
It says that South Sudan does not want Ethiopian troops to go deeper into the country.
The area where the children are being held "is full of jungle", AP quotes Mr Bashir as saying.
Ethiopia shares a long border with South Sudan and cross-border raids are not uncommon.
Hours after news of Friday's attack emerged, Prime Minister Hailemariam Desalegn said the government had requested permission from South Sudan to enter its territory to pursue the attackers.
The South Sudanese ambassador to Ethiopia also said his country was "cooperating and helping" the Ethiopian army to track down the raiders and rescue the abducted children.
Ethiopia's request to enter South Sudanese territory seems to have been approved quickly.
Before South Sudan's civil war began in 2013, joint security operations between the two countries were common as communities on both sides of the border were often involved in cattle raids.
Residents of Gambella town held a demonstration on Thursday demanding "justice for what happened" and calling for better security.
A mother whose husband was killed and three of her children abducted by the attackers earlier told the BBC that she had no hope of seeing her children again.
"I don't know if they were killed during the crossfire," Chol Malual said. "The fighting was intense and if they survived, they will be probably be killed by the Murles."
The targets of the raid were members of the Nuer ethnic group who live in both South Sudan and Ethiopia, the AFP news agency reports.
Ethiopia's army has surrounded the area in neighbouring South Sudan where it believes more than 100 abducted Ethiopian children are being held, local media report.