How can I get probabilities of next word with ELMO?
ELMO is not a language model. As specified in the paper:


  We introduce a new type of deep contextualized word representation that models both (1)
  complex characteristics of word use (e.g., syntax and semantics), and (2) how these uses
  vary across linguistic contexts (i.e., to model
  polysemy).


ELMO representations (embeddings) are indeed calculated by looking at the context of the word. But the output is a fixed size vector that contains/represents the contextualised meaning of that word. This can be compared to other word embeddings such as word2vec, glove or fasttext but with the novelty of having different representations of the same word based on their context within a sentence.

I suggest you do some reading about the differences about language models and word representations.