How to remove an installed AE certificate from server using PowerShell Scripts?
It is just Remove-Item to remove the certificate, and then include -DeleteKey to also drop the private key.

Based on your command it should be something like this as you have to pass in the thumbprint of the certificate to the remove command, so we need to look it up first:

Get-ChildItem -Path Cert:\LoaclMachine\My -Recurse -DocumentEncryptionCert | 
    Where-Object Subject -eq 'AlwaysEncryptedCert' | Remove-Item -WhatIf