Brown attended Florida A&M University Developmental Research School, before moving on to Prairie View A&M University.
He missed his senior season, after being dismissed from the school following an incident of burned buildings, in response to the killing of halfback Wesley Davidson.