Change the Nexus 4 voice dialer
I solve this on the Nexus 5 using Tasker with AutoVoice plugin.

Once you get both installed, add a profile that reacts to the "AutoVoice BT Pressed" event with the action for running the activity for Google:Voice Search (long clicking on the Google Search app and selecting VoiceSearchActivity from the activities presented.