Should I be worried about my iPhone overheating?
Yes and no.

The newer iOS devices have internal temperature sensors and will let you know when the internal temperatures are so high that damage is of concern. You basically get locked out of the device until it's cool enough to run again. 

As long as the sensor is working, it is safe to use the device up until it shuts itself off. This protection should cover you from charging or operating the device and causing harm to the components in the short term as well as drastic long term damage to the battery.

If you are concerned about the long term health of the battery - then yes, do worry about temperature. At 0°C - you can expect to lose between 2 and 6% of the capacity per year depending on the average charge in the battery (the lower is at 40% average charge - the higher is 100% average charge). If the temperature is 25°C - these losses jump to 4% to 20% capacity loss. If you store/use the phone for one year at 40°C which most people would consider very hot to hold - the losses in capacity are 15% to 40% annually. Charging the battery this hot or hotter is even worse for the health of the battery.

So, a few times of limited use under 40°C shouldn't be of concern, but leaving it in a very hot car (> 50°C) can be very damaging if it's charging or there for hours or days.

Whenever you can keep it cooler, and not fully charged for long periods of time, your battery will degrade less over time. You can weigh the need to use the battery when warm with the eventual replacement cost of $79 if you should ever just need a battery swapped out by Apple.