Why does a two-port network have two degrees of freedom?
I apply an input voltage it is the only independent factor and all else is dependent on the input voltage?


You have to also consider what you connected to port 2. If you connected a voltage or current source, that obviously also sets one of the variables on port 2. If you connected nothing at all, that means you set \$I_2=0\$. If you connected a short, then you set \$V_2=0\$. 

If you connected a resistor, you set \$I_2 = -V_2/R\$, which doesn't force either variable to anything in particular, but it does remove one degree of freedom from your original equations.