How do you determine the input impedance for an inverting amplifier?
I had thought the input impedance would be the R1||R2 (or whatever
  else would go to the node for V− which in this case is just R1 and R2)
  but I am doubting myself.
  
  Can anyone clarify what this input impedance is actually referring to?


For an ideal opamp, there is no current into the input terminals.  Thus, the voltage across \$R_3 \$ is \$v_{R3} = 0 \rightarrow v_B = 0 \$.

Since there is negative feedback, \$v_A = v_B = 0\$.

So, the entire input voltage, \$v_{in}\$, appears across \$ R_1 \$.

Thus, the input resistance must equal the value of \$R_1 \$.

\$R_{IN} = \dfrac{v_{in}}{i_{in}} =  \dfrac{v_{in}}{v_{in}/R_1} = R_1\$

Update due to edit of question:

I suspect you might be confused by two very different resistances.

The input resistance is simply the ratio of the input voltage to the input current:

\$R_{IN} = \dfrac{v_{in}}{i_{in}} \$

The resistance seen by (looking out of) the inverting terminal is \$R_1 || R_2 \$.

This is why \$R_3 =  R_1 || R_2\$ if you want the resistances attached to the input terminals to be equal.


  I can't believe that R2 wouldn't matter in this, so if anyone can
  clarify that, it would be useful.


Why?  It's basic opamp theory.