Do action potentials cause the neuron to heat up as well as its environment?
While it is possible that action potentials themselves result in waste heat, there are other processes within the neuron likely to contribute much more to energy lost as heat. The action potential itself uses energy that's been stored in chemical and electrical gradients that are the result of numerous sodium-potassium pumps which require the energy in ATP. There would be some loss of energy in the operation of these pumps.

But far more heat is likely generated by the mitochondria within the neurons. These are the structures that convert most of the energy in the breakdown products of carbohydrates and fats into the ATP the Na-K pumps require. Generous estimates put the efficiency of this production at roughly 50%. That means about half the energy from the food-stuffs used to create the ATP is lost as heat.