Is there a way to get a value of a multi value field of an entity reference (reference a field value of a referenced term)?
You can do this in a custom field type based on the core reference field, adding  a second property and dynamically filling the options of the select list depending on the first property. 

For the multi-value part of your question a non-coding solution would be a paragraph containing both fields. Adding the dynamic behavior can then be done in a form alter hook or you can try Business Rules and adjust this recipe to your requirements: https://www.drupal.org/docs/8/modules/business-rules/advanced-usage/dependent-field