Alcohol (ethanol) distillation: mystery congener
People turn apple cider all the way to apple cider vinegar, so during the process, there will be a time when the amount and ratio of ethanol and acetic acid are optimally balanced for making ethyl acetate - but it doesn't seem to happen. However, given enough time (and oxygen and bacteria), maybe different things happen. Interesting that the expired cider didn't have the nasty taste, but the concentrate did. I wonder if you actually made the ethyl acetate by your heating and distillation procedure.

Ethyl acetate presence gives nail polish remover/ solvent flavor/aroma. This is not related to volatile acidity. It's known to be produced by Pichia (and probably other microbes) in the presence of oxygen.
Source: https://www.homebrewtalk.com/threads/need-a-little-help-with-cider-flaws.666728/

The CRC Handbook shows a ternary azeotrope of water (9.0%), ethanol (8.4%) and ethyl acetate (82.6%) that boils at 70.23 °C vs the water-ethanol azeotrope that boils at 78.17 °C (96% ethanol). The boiling points are sufficiently close that a one-plate column might not do a significant separation, but the heads still should taste worse.
Suggestion: if the apple cider is acidic from a little acetic acid, neutralize to pH 7 or 8 before distillation ($\ce{NaHCO3}$). This might reduce the possibility of ethyl acetate formation. Another experiment would be to take fresh apple cider and see if you can get good tasting alcohol from it.
TLC of volatile solvents is difficult, but you might be able to detect different spreading patterns for the heads, the nasty heart, and some other heart (which tastes OK).