MySQL Error: Authentication plugin 'caching_sha2_password' cannot be loaded
So, I found the running process of mysqld by sudo lsof -i tcp:3306 then I killed it using sudo kill -9 &lt;PID&gt;.
After this I tried $ mysql_secure_installation again but ran into a new error Error: Can't connect to local MySQL server through socket '/tmp/mysql.sock' (61)
After trying a few fixes for getting mysql.sock to work, I started the MySQL server with
$ sudo mysql.server start
then went ahead with $ mysql_secure_installation to set password for the root user. 

This worked for me finally.

Note: Homebrew asks you to first do mysql_secure_installation before starting the MySQL server for the first time but that just made me run into endless loop of errors.