How do we know if GPT-2 is a better language model?
You have experimented with a small model (117M parameters). OpenAI has now released the medium sized model (345M parameters). Note that the full model has 1.5B parameters. Also note that GPT-2 has been trained on a tiny fraction of all available text. It's almost guaranteed that a larger model trained on more text will generate better text.

I have experimented with the 345M model, and I agree with The Guardian assessment. See for yourself:

Prompt: 


  I want to generate beautiful music using machine learning. Do you
  think a computer will ever be able to compose something as novel as
  creations of Mozart, Bach, or Chopin?


Results (first four samples, not cherry picked):


  Yes, computers will probably be able to do it by 2020. I think the key
  for us will be to make music that is interesting and beautiful, so we
  will never really know. My hope would be that computers will become
  far more efficient over time – something that is already happening in
  a lot of ways like in big data – while still being able to do more
  things that they are trained on, like find patterns. I think that's
  where I'd want to use neural networks in the future.





  We don't know, but it will probably happen. That's an interesting
  question to ask. Of course we have AI, which can do interesting things
  without human input. It can recognize the world, find objects and
  images, it understands language, we can make games for it. But if you
  imagine an AI being able to write music, will it even be good enough?
  How much better than human players? How much better can someone else
  do? AI will never really be at its best because it will always have to
  come up with creative ways to use and play it. To ask a question like
  it is good enough is to ignore that it is human. You are only allowed
  to ask good questions





  The software will help, as it already can do. But it will also allow
  more human creators, which is what I think it should do more of to
  bring music to any genre. In terms of composing, it's a matter of
  creativity in humans in general. But how does it achieve that? Let's
  just say that there are fewer humans composing today than ever before.
  Do you remember the day when, by chance, a computer could write lyrics
  to Mozart's Sonata No. 4, or the day when a human wrote lyrics to
  Mozart's La Résistance du Ciel No. 2? That was back in 1960, and it's
  just not true anymore. As we move forward, we are not going to create
  music that human musicians cannot compose. What is your goal as a
  computer music creator?





  At the core of this idea is that music is a collection of sounds – it
  was the very first art form, it was the first music that humans
  evolved, a collection of sounds. Music is not an abstract thing of
  sound, but a collection of sounds. You can't write music the way we
  write and compose books and films, because we write and compose with
  sounds in mind. This is where machine learning comes in. You can read
  some books about how to get better music writers or how to improve the
  quality of their music, but you cannot make a difference with this
  kind of information at all. What does music teach you? To write and
  perform beautifully, to convey emotion accurately, and to create
  lasting and rich relationships with