Is it necessary to downsample data for filtering of low frequencies?
Yes, you can get much better results if you downsample your data first. The problem occurs because the width of your transition band relative to the original data sample rate is very small. This can cause numerical problems when designing or implementing digital filters. The problem only gets worse as the filter order increases, especially with IIR filters. 

With that said, in some cases, you might have more success (i.e. better numerical stability) if you convert the filter to second-order sections. However, for this case, I think the right answer is to downsample the data first. 

The problem likely occurs because of the very small cutoff frequency on the low end of your bandpass filter. There is very little room for the transition band (less than 0.1 Hz, which is 1/320000-th of the sample rate), which is difficult to implement. If you downsample the data as much as possible, then you can at least increase this ratio by a good amount, which should give you better results.

I would also try using FIR filter implementations if possible, as they typically have superior numerical stability due to their lack of feedback.