Set an app to be default without clearing old first
Updated answer: Try Better Open With app, I see they added support for dialer (I can chose from Contacts or Phone).

Old answer: Go to Settings
Go to Apps
Select the app that is currently the default launcher for contacts.
Scroll down to “Launch By Default”.
Tap “Clear Defaults”