How to delete Grammarly extension from Safari?
After some trying, I figured it out.

Close all the apps that might be using Grammarly and delete it from finder. 

No need to remove the extension from Safari after that.