Custom Pluging Entries FieldType
You can use a Relation Field for each of your element types and include a custom js rule so if one of them has a value, the other one can't be selected. 

The more difficult way would be to extend the field and the ElementSelectModal and the ElementIndex to select multiple element types but this is going to be a little bit more tricky since you'll need a custom controller to fetch all elements and can't rely on the ElementsController