How to increase the performance of my website in Drupal 7
You can guess all you want and experiment, but the only way to actually achieve meaningful results is to profile the system to figure out where the bottlenecks are.   There are many ways to accomlish this.

You can use the Performance module to identify slow pages.

You can use xhprof and the Devel module to identify memory problems, and show slow functions and queries.

Are you running APC?  If not, you should.  Stick the apc.php file somewhere safe and check the statistics to make sure you don't have cache churn.

Install DB Tuner and see what the mysqltuner results and recommendations are, particularly with regards to buffer sizes and query cache size.

You can also read up on profiling and optimizing Apache

You could also experiment with Zend Server CE as it includes a PHP optimizer in addition to opcode caching.