Throughout all our experiments, we used BERT-Base BIBREF2 to provide the state-of-the-art contextualized modeling of the input text.

Semantic Retrieval: We treated the neural semantic retrieval at both the paragraph and sentence level as binary classification problems with models' parameters updated by minimizing binary cross entropy loss. 
 Question: How do they model the neural retrieval modules?
BERT-Base BIBREF2 to provide the state-of-the-art contextualized modeling