Change plist and make it stick
Well that was quick, I found the answer. Apparently in Mavericks the OS caches the preferences so you have to read them out after you change them. In my case I opened Terminal and typed defaults read com.apple.Safari

Read this article for more info