Is this natural gas shut off valve behaving correctly?
When the red lever is in line with the pipe then the valve is open. When the red lever is across or at 90 degrees to the pipe it is shut. This is a very simple visual system.

I think the red lever has lost the pin or protruding part that would act as the stop for normal operation. For your safety, and others, it would be a good idea to get it replaced - if just the lever needs replacing then that is easy. But if the complete assembly needs replacing then in some places you will need a qualified gas fitter.