What I'd like to do is have my Google Voice contacts available on my iPhone, not the other way around. I recently had to restore the phone to factory defaults and it's a pain to manually enter all them all.

When I make a new GMail e-mail account on my iPhone it won't let me import contacts from my Google account, but even if it did I don't want every single contact in my phone. Google for some reason adds every single person I've ever sent e-mails to into my contacts list, which as you can imagine is quite a large list by now.

Does anyone have any suggestions for how to do this?