Pennybacker was born at Pine Forge, near New Market, Shenandoah County, Virginia on September 3, 1805.
He attended Washington College (now Washington and Lee University) and the Winchester Law School, run by Henry St. George Tucker, Sr.