How do I extrude perpendicular to a plane NOT oriented along X, Y or Z?
Press F to fill the face with an n-gon and then extrude.
Once a face exists the extrusion can happen along the normal of that face.
If you then want to move the face along that axis again you can change the transformation widget to the 'Normal' axis:

Allowing you to move like so either by using the widget pictured...

or by pressing G,Z,Z.
You can then delete the face after the extrusion is complete via X.