Alter the path of the more link from Views
If I have understood your question, I have been needing the same thing.
I solved it by means of:

1) add the field 'link to taxonomy term' and exclude it from visualization

2) format the description field as 'clipped version' (I am using right now a Spanish version of Drupal 8, not sure what the English option name is called)

3) rewrite the output for the description field as '{{ description__format }} {{ view_taxonomy_term }}'

That did the trick.

I hope that that's what you had asked, and that someone can take advantage of it.

EDIT: After re-reading your question, I realize that what I was trying to solve is different: a taxonomy terms view, which returns a list of terms, the description shortened to 160 chars, with a 'read more' link based in the taxonomy term's custom path (/path instead of /taxonomy/term/2). 

Since I haven't been able to find an online answer to that, and given that google drove me to this question while trying to find an answer to mine, I think I could still leave this answer here. However, if the OP or some mod tell me to remove this answer, I gladly will.