Disconnect a Google/Android account from a phone number
Instead of deleting your phone number I advice adding an additional recovery email.
But if you want to delete your phone number from your Google account, you have to go to myaccount.google.com and then select access and security, then select recovery phone. Finally enter your Gmail password, press edit and delete the phone number.