A lot of people don't know the current wave of large models can effectively be traced back to one dude. At the time OpenAI was mostly doing RL. All evidence was that scaling models via parameter counts was pointless. The consensus view in the field BY FAR was that gradient descent was unlikely to lead to anything crazy on it's own and a new paradigm likely required. The order of events I've heard was that Alec Radford basically went off and did the first GPT paper by himself in a corner at Open AI. I think that work was as important as the attention paper. This is also the why the whole 'google invented attention why didn't they capitalize' argument is very wrong - almost no one in the field felt that scaling those methods would work.