Mac OS X (10.9) and 8192 bit certificates - Error -67762
The answer, as discovered by shizmob, is that Apple moved the location of this preference in Maverics to /Library/Preferences/com.apple.security. So what you need to run is

sudo defaults write /Library/Preferences/com.apple.security RSAMaxKeySize -int 8192