What's beyond topic modeling?
You could use doc2vec to create vector representations of each document. Once you have all the vector representations you can use standard unsupervised clustering techniques like k-means, hierarchical clustering, or K-SOM.

The doc2vec model you create will be able to compute cosine similarity between two documents and also find the n most similar documents to a given document and provide a similarity score for each.