MySQL incorrectly saying serverID non unique when adding an instance to the InnoDB cluster
Did you restart the server after changing the server-id? Despite server-id being dynamic, there is the following caveat on the manual:

If the server ID is set to 0, binary logging takes place, but a master
with a server ID of 0 refuses any connections from slaves, and a slave
with a server ID of 0 refuses to connect to a master. Note that
although you can change the server ID dynamically to a nonzero value,
doing so does not enable replication to start immediately. You must
change the server ID and then restart the server to initialize the
replication slave.

I am giving a wild guess that may be your issue, because it is a very common problem that people suffer once when setting up replication.
Edit: Re &quot;Interestingly mysql is ignoring the server_id in the my.cnf file and choosing its own.&quot;
Then you have a configuration problem in which your server id is not properly read, please use --print-defaults to debug which files and options are being applied live, and check the paths for correct permissions, file existence and avoid duplicate config definitions/unintended includes.