Algorithm for document retrieval in QA system
I am still not quite sure how you are solving in the problem with tf-idf for a QA system. However, there have been lots of improvements that has been done in the QA domain over the years, with the usage of deep learning for natural language processing.

I would urge you to look at the following approaches that might help you to reach the accuracies you are looking for:


Bidaf Model that was used to solve SQuAD dataset.
BERT model that was used to solve the SQuAD dataset.


You have multiple open source implementations to train the network and use it for prediction. You could  also learn about them in the papers published by the creators of these networks.