Why is this plus sign in a yellow circle appearing on the selected face?
Check your Tools panel, on the left of your 3D View (displayed with T), you must have activated the Extrusion tool, select another one if you want. To use the Extrusion tool, you can either click on it on the Tools panel, or activate it with the shortcut E, in that case you won't see the yellow point.