According to the 2010 census, Brownstown has a total area of , of which (or 99.38%) is land and (or 0.63%) is water [REF].
Brownstown lies on the East Fork of the White River, bordered by the Jackson-Washington State Forest and near the Hoosier National Forest.