Downsampling lowpass filter for audio: FIR or IIR?
If linear phase is a requirement, that will probably steer you toward an FIR implementation. It is possible to build IIR filters that have approximate linear phase, but it is easy to design a linear-phase FIR. 

If you're concerned about latency, forward-backward filtering as in filtfilt isn't really a good option. In general, it's really meant to be used an offline process, since to implement the technique exactly, you have to run the entire signal through forward, then do the same in reverse. That implies that you have access to the entire signal at once, which is not commensurate with low delay.

In general, an FIR filter will require a higher order for a given set of performance requierments. However, FIR filters bring some real advantages, such as guaranteed stability, lower susceptibility to roundoff errors (since the quantization error doesn't get fed back through the filter, although you can compensate for this with some increased complexity), and simply-achieved linear phase response. In addition, efficient FIR filter implementations are available for many processor architectures, mitigating the cost of the extra taps somewhat.

Another way to mitigate the extra-tap cost for FIR filters in your situation is to take advantage of efficient multirate signal processing techniques. Specifically, you can use a polyphase decimation approach to significantly reduce the number of computations that you perform in the process of decimating your signal. This has the effect of decreasing the number of effective taps (in terms of computational complexity) in the decimation filter. In addition, if you need to decimate by a large factor, then multistage approaches can help further reduce your load. Lyons' introductory DSP book has some good easy-to-read material on these topics.

Given more specific parameters of your system, one could make more pointed recommendations. What are your filter design requirements? What sort of computational capabilities does your platform have? What sample rate will you decimate to?