What are the most recent and influential breakthroughs in NLP?
I suggest you take a look at the syllabus of http://cs224d.stanford.edu/syllabus.html and see what was invented in the last ~20 years. They focus on deep learning methods, and they include there almost everything important.

Among the things, which I would include into the review are:


Word2Vec models
Seq2Seq models


Concerning what happened in the last 5 years, i.e. since 2013, I suggest you mentioning super human performance on the SQuAD 1.1 dataset: https://rajpurkar.github.io/SQuAD-explorer/