How to change Android's Google Chrome's default downloader? (not under app - set default)
You'll have to first disable ES Downloader (download manager) from ES explorer settings:


Open ES explorer and navigate to Settings → (Tools settings) → Download manager
Tap on Download manager and check Disable ES Downloader
Now open your preferred browser (Chrome) and open a downloadable link, it should now prompt you as before and select the suitable handler.
Essentially you would've cleared the default handler for downloading links.


 

Note: I have also observed a similar bug in Android 7, where the defaults are not simply cleared the conventional way (i.e by going to Settings → Applications → Clear defaults, for now this is the better way to do it whenever ES explorer is involved.

This may also depend on version of ES explorer in use. I tested with version 4.0.4.7 

Hope this helps