Refresh Downloads list
This is a correction for Akhil's answer. I have tried this on Android Lollipop and it works :

Settings > Apps > All > Download Manager (NOT Downloads) > Clear Data and Clear Cache.