SSH connection using public/private key authentication fails only for port 22
The problem here seems to be that you have "something" interfering with your network traffic that is specifically blocking out your attempt to authenticate via keys on port 22.

This "something" could be so called "endpoint protection" software (software firewall), it could be a network device on your network (i.e. router with deep packet inspection, firewall, etc.) or it could be a network device on the network path or at the server end.

This seems to be the most logical conclusion as this can explain why it doesn't work on port 22, but do work on other ports.

I.e. if you hadn't included the port information, I would have looked at the authorized_keys file on the server. It is possible to specify that a specific key can only be used for authentication when connecting from a specific host - or that it must not be used when connecting from a specific host. If you had set that up to allow the Ubuntu host and not the Mac host - that could explain parts of what you're seeing. However, this method does not include matching again the port number - so it cannot explain that.

This is all assuming that you're running the exact same configuration for port 22 and port 2222 (which you stated you are), and that you're running standard OpenSSH software on the server. If you're not, that software might include configuration to discriminate based on ports - but I don't think it is likely.