Conjecture: The majority of improvements in ChatGPT (all models) has been from RL on user conversations. If you've ever said to it "that's stupid," or "that's wrong," you're helping improve its understanding.