Safari is using too much RAM. How can I reduce its memory footprint?
So it appears that Flash is still the main culprit - if you enable any flash in ClickToFlash it stays resident in memory.

Following Gruber's advice on Daring Fireball, I've disabled Flash entirely on all browsers except Chrome and this seems to have helped immensely