Did I damage my Arduino?
Yes, it sounds like you fried something.

I can't quite make out what it was you did from your description (I'm sure it makes perfect sense to you with the hardware in front of you), but the symptoms do sound somewhat fatal.

What actually happened depends very much on the Arduino you have - if it's a genuine one or a cheap Chinese clone. Either way the USB circuitry seems to not be functioning. Whether that is because the USB chip (varies depending on if it's a real Arduino [ATMega16U2] or a clone [CH340G]) or one of the voltage regulators is unknown.

You should power the board up normally and test the voltages at the 5V and 3.3V pins. Also if a component is still getting hot you should note which component it is, and that will give more of a clue as to what went wrong.