drupal site is really slow on VPS server I have a drupal 7 website on a VPS server with the following specifications: 

Processor Name: Intel(R) Xeon(R) CPU E31270 @ 3.40GHz

Processor Speed (MHz):  3392.294

Total Memory:   3924628 kB

Free Memory:    276096 kB

Total Swap Memory:  4063228 kB

Free Swap Memory:   3894104 kB

Apache 2.2.29:  Running

DirectAdmin 1.47.0: Running

Exim 4.84:  Running

MySQL 5.5.31:   Running

Named 9.8.2rc1: Running

ProFTPd 1.3.5:  Running

sshd:   Running

dovecot 2.2.16: Running

Php 5.3.29: Installed

With one GB of ram, I thought that the slowness of my website might be because of the heaviness of tables. (Number of tables: 381, DB size: 60 MB) 
But by searching the net, I found that my DB is not that heavy. I also noticed that in my services, the memory usage of httpd is 650 MB. I think this also might have something to do with slowness of my website. 
Can anybody help me with this issue?
Thanks.
Isn't it closer to 4Gb RAM based on your posted stats?
A server with only 1Gb RAM will most likely not work properly with a full application stack. Apache and MySQL require memory to be performant. The least I'd recommend based on personal experience is 2Gb RAM.

What you can also do is perform traces on your site with xhprof to see what the slowest items are, and improve those.
APC and memcache can also help to boost speed. Or if you only have anonymous users, using the boost module can be a very quick fix, although it's not maintained actively anymore for Drupal 6 it seems.