Cannot connect remotely to sql server 2012 instance using SSMS
You should not need to specify the port for a named instance (that's kind of the point of the name - and since it's dynamic you won't want to hard-code a port into your connection strings etc. unless you want to change your config files every time the port changes). Try connecting to:

123.123.123.123\SQLEXPRESS


Also make sure that your server is exposed to the Internet on that port (firewalls may be preventing this) and that the SQL Browser service is running.

PS Ping means nothing. You need to try to telnet to the port that SQL Server is exposed on.