Extruding outwards from a cylinder
It is most likely an issue with the normals of your faces (the way that blender thinks the faces are pointing). You also appear to have some duplicate vertices that are throwing things off.

I would do these steps (starting from before you extruded the faces shown in the image):

-Select everything with A

-Press W and select Remove Doubles

-With everything still selected press Ctrl+N which will force blender to recalculate the normals of the faces.

-Select the faces you need again and extrude with E and it should work.

Duplicated vertices are often caused when you extrude a face and then cancel the extrusion with right click. Despite it appearing that the extrusion has been cancelled, extra vertices will have been created that are in the exact same location as other vertices. This can get Blender confused about which way is out for some faces.