iTunes can only perform one task at a time
First off, maybe you should check in Activity Monitor if it is iTunes that is the cause of the slow performance or if it is some other process that is taking up a lot of memory/CPU. 

If it is indeed iTunes that is causing the problem, try this:


Turn off iTunes Match and/or Genius as they are known to take up a lot of memory and downgrade the responsiveness and performance of iTunes. 
You could rebuild your library by following the steps here (offical Apple document): http://support.apple.com/kb/HT1451. This excerpt was taken from there and is why I am recommending that you rebuild your library:


  Your iTunes Library file may become damaged and cause iTunes to become unresponsive.

I doubt this will make a huge performance difference, but you could try disabling smart playlists and reducing the number of criteria by which your sort the songs (e.g. In the music tab of iTunes, don't sort your songs by name, album, time, bitrate, artist, date-added, date-modified, etc. - only pick a couple criteria).


I hope some of these suggestions helped you!