Custom entity - BaseFieldDefinition for entity reference field
The easy solution is to create the field via the interface instead of creating it as a base field. Drupal core still have some issues with basefields not working exactly the same way as config fields (fields created as config via interface). Views is missing various integration, the same goes with date fields.

In your case, where you have a multi valued field, it would be store the same way, the only difference is the table name (ENTITY__FIELDNAME vs FIELDPREFIX_FIELDNAME).

If you really want to use base fields, you can make up from the missing integration by handling it yourself in a custom EntityViewsData class for your entity. Having done this in the past, my advice would be to just create a config field, since that is working pretty well, and doesn't change anything on the way the field is store in the database anyways.