Representation Learning of Music Using Artist, Album, and Track Information
Jongpil Lee Jiyoung Park Juhan Nam


[
Representation Learning of Music Using Artist, Album, and Track Information


Jongpil Leekaist
Jiyoung Parknaver
Juhan Namkaist


kaistGraduate School of Culture Technology, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, South Korea
naverNAVER Corp., South Korea

Juhan Namjuhannam@kaist.ac.kr

Representation Learning, Artist label, Album label, Track label, Music Metadata

0.3in
]

  


Supervised music representation learning has been performed mainly using semantic labels such as music genres. However, annotating music with semantic labels requires time and cost. In this work, we investigate the use of factual metadata such as artist, album, and track information, which are naturally annotated to songs, for supervised music representation learning. The results show that each of the metadata has individual concept  characteristics, and using them jointly improves overall performance.




§ INTRODUCTION



Representation learning of music has been recently performed by supervised deep learning using semantic labels such as genres, moods and instruments <cit.>. However, annotating music with such semantic labels requires significant time and cost and the labels are often ambiguous, resulting in disagreement among annotators <cit.>. Meanwhile, metadata such as artist labels require no cost and they are factual information with no ambiguity. We recently investigated the possibility of using artist information for representation learning of music and evaluated it in transfer learning settings <cit.>. The results showed that the learned representation is comparable to those using the semantic labels. In this work, we extend the use of music metadata to album and track information, which are more specific levels than the artist information. We use a similarity-based learning model following the previous work and also report the effects of the number of negative samples and training samples.






§ MODELS


Figure <ref> illustrates the overview of representation learning model using artist, album, and track information. Following the previous work, we use a Siamese-style Convolutional Neural Network (CNN) with multiple negative samples[In this work, we used twice the number of filters for all layers.]. We build one large model that jointly learns artist, album, and track information and three single models that learns each of artist, album, and track information separately for comparison. The single model basically takes anchor sample, positive sample, and negative samples based on the similarity notion. For example, in the artist similarity concept, positive and negative samples are selected based on whether the sample is from the same artist as the anchor sample. We should note that the model takes a segment of audio (e.g. 3 second long), not the whole chunk of the song audio. Thus, in the track similarity concept, positive and negative samples are chosen based on whether the sample segment is from the same track as the anchor segment. Finally, we construct a joint learning model by simply adding three loss functions from the three similarity concepts, and share model parameters for all of them. 

 
 


§ EXPERIMENTS AND EVALUATIONS


















The four models are trained with Million Song Dataset (MSD) and its artist and album metadata <cit.>. We first build two splits based on each artist and album information. The artist split is the same as the previous work, which has 20 songs for each artist. For the album split, we selected 10 songs for each album and used twice as many albums to match the number of training samples of artist. Then, 10 songs of one album are divided into 8 songs, 1 song, and 1 song for training, validation and testing. The artist split is twice these numbers. For the validation sampling of artist or album concept, the positive sample is selected from the training set and the negative samples are chosen from the validation set based on the validation anchor's concept. For the track concept, it basically follows the artist split, and the positive sample for the validation sampling is chosen from the other part of the anchor song. 

The evaluation is conducted in two ways: 1) hold-out positive and negative sample prediction and 2) transfer learning experiment. The hold-out positive and negative sample prediction was designed to see how well the models distinguish each concept. The evaluation is conducted on the test set of the above splits. For the artist and album concept, the positive sample is selected from the validation set and the negative samples are from test set based on the anchor's concept. In this evaluation, the random guess is 20% when the model uses 4 negative samples. The transfer learning experiment is performed on three external genre classification datasets including GTZAN (a fault-filtered version) <cit.>, FMA small <cit.>, and NAVER Korean <cit.>. In this experiment, the learned representation is extracted and injected into a linear softmax classifier. This experiment was designed to see the generalization ability of the learned representations. For both evaluations, we used a model trained with 5000 artists (or/and 10000 albums) with 4 negative samples. After grid search, the margin values of loss function were set to 0.4, 0.25, and 0.1 for artist, album, and track concepts, respectively.




§ RESULTS


The result of hold-out positive and negative sample prediction is shown in Table <ref>. We can see that each of the models performs best when the concept matches between the training set and test set. Also, the jointly learned model achieves good performance for all concepts.

The transfer learning experiment result is shown in Table <ref>. The artist model shows the best performance among the three single concept models, followed by the album model. This is probably because the genre classification task is more similar to the artist concept discrimination than album or track. The jointly learned model slightly outperforms the artist model. Finally, we included the baseline results obtained by performing genre classification directly without transfer learning. The results show that transfer learning using large music corpora with the factual metadata is highly effective in the GTZAN and FMA datasets, but not in NAVER dataset. This was due to the cross-cultural differences between the source and target datasets when looking closely at class-wise performances. 

The effects of the number of negative samples and the number of training samples are shown in Table <ref> and Table <ref>, respectively. We can see that increasing the number of negative samples and the number of training songs improves the model performance as expected.










icml2019



