ISA Model and GFS data
I would agree with your hypothesis, but I must point out that it will likely break down further and further as you approach the tropopause. Whether the ISA model is actually a average, I cannot say. However, the idea that the ISA model is a horizontal average is plausible.