How do we use signal::filter and signal::butter in R for EEG data?
I think your problem is that you want to use signal::filtfilt rather than signal::filter for the cut-offs you set to map on to the half-amplitude cut-off (otherwise, with the signal::filter function, these are the half-power cut-offs).

See my answer here for example code showing how to use the W parameter in signal::butter and double-check that your filter is working appropriately:

https://stackoverflow.com/a/49905340/9665680