Removing the USB storage is removed notification
Yeah this happens on Moto X too. You can long press on the notification and click on App Info. Then force stop the application that posted the notification (System UI). Doing so doesn't seem to affect anything else. Annoying we have to do that but better than restarting phone.