Is not using soap when washing cast iron pans safe?
That residual fat is leftover from what you cooked in there before, so it would only be an issue after cooking something with a very low smoke point, washing the pan,  then cooking something with a high smoke point.

Wiping the pan with a paper towel will get rid of the vast majority of this, and if you dry it over heat you can carry on to cook off the last little bit, like a sort of half-baked or top-up seasoning. Doing this before storage gives the smoke smell time to clear before you next use it, but this should be minimal if you cook it off gently. I recommend turning on the cooker hood extractor if you have one. 

I like the heat-retention of cast iron but I've gone over to enamelled (le Creuset). I still use a carbon steel wok that needs reseasoning after using the traditional metal spatula, so I actually rinse, dry with heat, wipe over with oil and heat.