How to remove pattern lock on android 5.0 on mu Nexus 4
Go to Settings >> Security >> Device Administrators. See if anything other than Android Device Manager is checked. If something is checked, it's possible it's forcing a higher level of security. 

Uncheck it if there is, and check the screen lock options again. 

Also try Settings >> Security >> Clear credentials.