What would happen if you hypothetically happened to touch an aurora
Auroras typically happen pretty high. About a century ago, Norwegian scientist triangulated more than 12,000 auroras, and this is the distribution they found:



Figure 1 from Störmer, 1946.

So the fist problem will be to "touch" anything at an elevation of 100 km, where the atmospheric pressure is 0.00006% of the sea level pressure and 0.0002% of the pressure at the top of Mount Everest (pressures estimated using CalcTool).

So let's imagine you use very thin space suit that stop your body from blowing up, and keep you alive.

Even in that case, the air is so thin that its heat capacity and conductivity would be extremely low. Therefore, no matter if the air is at -100°C or +300°C, you wouldn't feel it, and the temperature of your fingers would be dominated by the heat loss through radiation (i.e. your hot fingers would radiate infrared radiation and become very cold pretty soon).

Visually they are diffuse objects, so it will be as disappointing as "touching" a cloud. Where you never really feel you ever get to the "edge" of the cloud and touch it. You just feel that you become shrouded in an increasingly thick fog. That's is at least my experience climbing mountains that from below look like they have their top covered in thick clouds. On airplanes the experience is quite similar. So I think you will just not realize you are at the edge of an aurora and you will just see a subtle glow in the air.

Another important effect would be the radiation. You would get a big dose. I mention a bit of this in my answer to this question, however the formula there is not valid at 100 km. Because radiation increase exponentially the first 10-15 km as you go up, and then flattens (Stozhkov et al, 2001). Under normal conditions, at 100 km you would be receiving approximately 500 times more radiation than in the surface (ref1,ref2), which is pretty much the same dose of outer space. That is not a big deal if you don't stay there very long. But auroras don't happen under normal conditions. However, radiation at surface levels during auroras go up but not by a large amount, so I don't think radiation will kill you.

So, if you get up there with a space suit that keep you alive, I think your quest to touch the aurora will be rather disappointing. However, the view from up there would be amazing.

You might want to see this video of a weather balloon sent up under an aurora. With cameras and radiation sensors. It is pretty cool, but it only got about a fifth of the way towards the aurora. And you can't get much higher with balloons, the world record (unmanned) is 53 km, which is just half way there. These are a couple screenshot from that video: