At the same time, some information about inflected word forms in the context can be useful, but it is lost during lemmatization, and this leads to the decreased score. Arguably, this means that lemmatization brings along both advantages and disadvantages for WSD with ELMo.  
 Question: Do the authors mention any downside of lemmatizing input before training ELMo?
Yes