How to disable AddressBookSourceSync in El Capitan
Not sure if this is really a good idea, but you can run this (as yourself, not root) in Terminal:

launchctl unload -w /System/Library/LaunchAgents/com.apple.AddressBook.SourceSync.plist


FWIW, I had the same problem and even after following the instructions in Tetsujin's link, turning off all Contacts accounts in System Preferences, logging out of iCloud, blowing away the AddressBookSourceSync directories in ~/Library/Caches and ~/Library/Application Support, and of course many reboots, I could not get AddressBookSourceSync to stop blocking sleep. This command “fixed” it. AddressBookSourceSync no longer runs and pmset -g reports nothing preventing sleep.

You can undo the effect with this command:

launchctl load -w /System/Library/LaunchAgents/com.apple.AddressBook.SourceSync.plist