Why IC 4081 is taking 2-5 seconds to set the output to low
When you leave a CMOS input open anything can happen, including receiving mystical message in morse code.

Seriously, if you want a CMOS circuit to behave you MUST connect the input to a well-defined 0 or 1 level. Leaving it open is not OK.