Proper remote access control management in PostgreSQL
Just change pg_hba.conf from 

host all all 0.0.0.0/0 md5


to rules you need. Columns are :
type (leave host),
database,
username,
IP,
access method.
For example, if you need to allow only 1.2.3.4 and 5.6.7.8 to connect any database using any username:

host all all 1.2.3.4/32 md5
host all all 5.6.7.8/32 md5
host all all 0.0.0.0/0 reject


Use
    service postgresql reload 
(or smt like that, depending on your system) to apply changes.