What are the most recent and influential breakthroughs in NLP? I'm looking at the history of NLP, starting in the 1950s, with the Georgetown–IBM experiment.

What are examples of the most recent (e.g. in the last 5-10 years) and influential breakthroughs in natural language processing?
I suggest you take a look at the syllabus of http://cs224d.stanford.edu/syllabus.html and see what was invented in the last ~20 years. They focus on deep learning methods, and they include there almost everything important.

Among the things, which I would include into the review are:


Word2Vec models
Seq2Seq models


Concerning what happened in the last 5 years, i.e. since 2013, I suggest you mentioning super human performance on the SQuAD 1.1 dataset: https://rajpurkar.github.io/SQuAD-explorer/