Marshall Leonard is being held by police after throwing a bomb into one of the department stores in Tupelo, Mississippi, early on Sunday.
Mr Leonard is known in the city for the 4ft (1.2m) flag he flies over his car.
Nobody was hurt or injured by the improvised device that produced only a loud bang but no damage.
"An employee was sitting in the vestibule taking a break. He told the employee to run - that he was going to blow the place up," said Police Chief Bart Aguirre.
The suspect then threw the newspaper-wrapped package into the store - around 0130 local time (0630 GMT).
Walmart, along with several other retailers, pulled the battle flag and other memorabilia from their shelves following the killing in June of nine black worshippers at a church in Charleston, South Carolina.
The man charged with the killings was pictured holding the flag.
Explosives experts said that the package contained enough explosive material to damage the store, but its poor assembly rendered it ineffective.
He was arrested by police about a half-hour later when his silver Mazda adorned with stickers of battle flag and the Mississippi state flag - which itself features an inset Confederate battle flag - was seen running a red traffic light.
"He's a strong supporter of keeping that flag flying," the police chief said. "This is his way of bring attention to that".
Mr Leonard, who lives alone and is unemployed, does not currently have a lawyer, but will be appointed one by a judge if it is deemed he cannot afford one.
The Confederate battle flag became a potent symbol for the southern states fighting the Civil War as they sought to break away from the union.
It is seen by some as an icon of slavery and racism while others say the banner symbolises their heritage and history.
A man apparently angered by Walmart's decision to stop selling Confederate battle flags has been accused of bombing one of the retailer's stores.