Is changing the SQL Server port really that much safer?
It helps against common port scans that can be initiated through port scanning websites. But it won't help against a committed attacker. It's just another layer, but doesn't add much over the firewall, as you mentioned.