Combining heterogeneous data sets for more powerful machine learning Suppose we have two data sets of movie reviews; one from IMDB and one from Rotten Tomatoes (RT). Each entry has a written-review and a score attached to it. The concatenated datasets might look like 

Movie|Score|Review|Site

The Lion King|9.8|"This is one of the best movies ever!"|IMDB

The Lion King|4.5|"This move absolutely made my childhood"|RT

etc. 

Our task is to predict scores from reviews. This is similar to the Kaggle IMDB sentiment analysis challenge, except we're trying to predict a continuous variable from two data sources. For a single data set, we might train a model in Keras with an embedding and lstm layer. In R:

model &lt;- keras_model_sequential() %&gt;%
   layer_embedding(input_dim = features, output_dim = 32) %&gt;%
   layer_lstm(units = 32) %&gt;%
   layer_dense(units = 1, activation = "linear")


where features is the number of words in our corpus. 

How do we combine the IMDB and Rotten Tomatoes data sets to maximize our predictions?

My first thought is to include the site column as a categorical variable and then let the model learn how to best combine them. Would this cause a skew in the predictions? 

We could also try multi-task learning, but most of these methods seem to be designed to predict repeated observations. Such as 5 different people reviewing the same 100 movies. 

Some things to consider:


IMDB and RT don't have all the same movies (i.e. not a repeated observation problem)
There are words local to each dataset. i.e. the word "rotten" might be a frequent word on RT, but never appears in the IMDB dataset. 
The outcome measures are different. RT uses a 5 star scale, whereas IMDB uses a 10 star one. 


How do we deal with this heterogeneity?
Following your considerations:


  IMDB and RT don't have all the same movies


I would build a larger dataset of movies, aggregate all observations available, and run a model on that.




  There are words local to each dataset. i.e. the word "rotten" might be
  a frequent word on RT, but never appears in the IMDB dataset.


I think this doesn't constitute a problem once you aggregate data in a larger dataset. At that point you can run some NLP method on the whole aggregated corpus. Let me know what you think.




  The outcome measures are different. RT uses a 5 star scale, whereas
  IMDB uses a 10 star one. How do we deal with this heterogeneity?


You just need to uniform the scales on a given interval like [0, 1]. Then all votes could be compared.