Gilbert grew up in Detroit, Michigan with parents Leroy and Ann Brown, older brother Wes, and younger siblings, Sheri, Tommy and Anna (all of whom went to college and became athletes).
During his career at Mackenzie High School, he recorded 189 tackles and 19 sacks.