Some SQLs sometimes take all available RAM
The main problem is with the shared_buffers = 32MB parameter. It is 2014 not 2000. You cannot expect any kind of decent performance on a 6Gb machine, if you do not increase that to at least 512Mb. I believe that you are also using PostgreSQL 64bit (if not make the change now). Also, you say that your dataset = 1.5 GB. Are you sure? 50 million records fit on 1.5GB on disk? Have you run SELECT pg_database_size('myDbName'); Does it still show 1.5GB?

I also see you do not have an index at status_razsmernika field and you do a join on that, which leads to a sequential scan. You also have (razsmernik.status_razsmernika!=6) and (razsmernik.status_razsmernika !=0). So, you really need to add a b-tree index to status_razsmernika.

Also you are using the "updated" field for order in descending value. If this is always the case, specify the order in the related index as well. http://www.postgresql.org/docs/9.3/static/sql-createindex.html