Having a PostgreSQL database exposed to the internet
It's not really about it being a bad idea in general, more that there are very few instances where you really need to expose your database to a public interface.

Much of security is about reducing the number of attack surfaces you expose. If your application does not absolutely have to expose your database to the public then you can hide it away behind a firewall and have another level of protection from any potential exploit that might appear in the future.

I would not consider "remote administration" a good enough reason to expose postgres. As you say; you already need SSH access to your box so you could easily keep postgres behind a firewall and gain access via that without any problems.

RE: SSL connections. This would be a must unless you were happy with third-parties just reading your data while you send it over the wire. The considerations here are the same as any traffic over the public internet.