the reason llms seem to reason is the general fact that lemons are sour is a little encoded in the frequency with which s follows l in the whole corpus of language, and the same is true for anything else we've reasoned about at scale.