Neumann was sworn into the 104th United States Congress on January 3, 1995, when the Republican Party gained control of both houses for the first time since the 1950s.
In September 1995, Neumann failed to pass an amendment requiring congressional approval of troop deployment to Bosnia.