Check constraint that exactly one in group has bit column set as true
You can define a filtered index on the table which will do this for you, there is no need to use a check constraint or an additional table for this. 

CREATE UNIQUE INDEX IX_Address_UniquePrimaryAddress 
ON dbo.Address(CustomerId) 
WHERE IsPrimary = 1;


Here is a good link of filtered indexes that should help you:

https://msdn.microsoft.com/en-us/library/cc280372.aspx