Born near Staunton, Virginia, Brown attended Washington College (later Washington and Lee University) in Lexington, Virginia, and the College of William & Mary, Williamsburg, Virginia.
Brown commanded a company of sharpshooters in an expedition against the Indians in 1789.