Nexus 5 in Recovery mode not detected in ADB but another Nexus 5 gets detected
I got it to work. The problem was with my USB cable. It intermittently disconnected my phone from the laptop and that ended up in windows thinking of my phone as an unrecognized device.

I manually installed the drivers for ADB Composite Device and then fired "adb devices" and it showed up.

Thanks Firelord for your help.

Really appreciated. :)