Can I use electrolytic capacitors in an oscillator?
Yes, you can use electrlytics in a Colpitts.

A Colpitts oscillator has places where you can use polarised capacitors, and for the places that experience bipolar operation, you can use two in series with a Mohm bias resistor to the centre point to a high voltage to keep them both properly biassed.

Use the choke from a flourescent fitting as the inductor, and you might get operation way down in the low audio. I don't normally recommend mains transformers as inductors, and certainly they don't ever make good power inductors, but here they would be a good source of large low frequency inductance. Excellent learning opportunity. Stability, tolerance, Q, suitable values, well, there's a reason electrolytics don't get used much.

You will probably find it won't work with a small inductor and big capacitor. The sqrt(L/C) gives you an impedance. Any given design of sustaining amplifier will only work well over a range of tank impedances. Too low an impedance and the active device gets excessively loaded, and the losses in the components drop the Q values too low to resonate. Keep the impedance up with a large inductor. That's part of the learning, but unfortunately an oscillator that just 'doesn't go' can be a bit of an enthusiasm damper.

I will agree with the comments that if you just want a quickly working low frequency oscillator, then go RC. If you want to learn, then find a big inductor to go with the big caps.