How to prevent dictation from accessing/sending your contacts?
This is just a work around and I don't own Mountain Lion yet, so I can't test it myself, but I bet it will work. In all likelihood, each user on a given computer can turn Dictation on or off independent of the other users and Dictation only sends information from whichever user account is currently active.

When I get Mountain Lion, I will set up a second "dummy" user, that will only be used for dictation/transcription. I won't use any user info-based applications (iCal, Address Book, etc.) in this secondary account. When I want to dictate something, I'll use the handy fast user switching command to jump over to my dummy account (also copying over any needed information or documents pertinent to the dictation), dictate the content, and when it comes back from Apple I'll copy the transcription back to my main account. Hypothetically, then, the only info Apple will get from me will be fairly standard hardware/OS stats, my Apple ID (which they, of course, already have), and the contents of the dictation/transcription.