Feeds mapping to Entity ref by Entity ID not working
Might be important: Are the product displays (the albums) already in the site or are you importing those as well? 

Entity IDs are generic unique identification numbers that entities use to reference each other. Nodes are a certain kind of entity and their id is "Node ID" but you are looking for the Product ID, which, Kickstart 2 does a really good job of hiding from the site builder. Try going to admin/commerce/products/variations to see all your products and when you click edit on one, the number in the url is your entity id for that product.

So, when you create an album with mp3s, the structure looks like this:

Album (node id: 123)
  |__ commerce_product (field), Value: 23,24,25

MP3-1 (product id: 23)
MP3-2 (product id: 24)
MP3-3 (product id: 25)


All of the "id"s are entity ids.

Regardless, you can use the entity field query finder function of the feeds_tamper module to locate an entity reference while importing a CSV using Feeds.