He served as federal judge from September 5, 1883, until his death in Harrisonburg, Virginia, on November 1, 1901.
His son, John Paul Jr., also became a lawyer, and served as U.S. Representative and judge of the United States District Court for the Western District of Virginia.