The Union troops were in the center of Baton Rouge, while the Confederates were lined up in two divisions, north of the city.
The colonel led a retreat back to prepared defensive lines near the Penitentiary, under the protection of the Union warships.