Best way to scale across different datasets
Your observation stems from two facts


Model performance deteriorates when data distribution changes (shifts). For example, when model is trained on Boston dataset but tested on Paris dataset, or trained on summer dataset but tested on winter dataset.
Model evaluation (wrongly) shows an improvement when information leaks from test set. For example, when test set is scaled with must-not-be-known statistics of the test set like (min, mean, std, max).


If a data set for other cities like Paris is not available at the training time (1), obviously there is nothing we can do. In real world cases, when we see a noticeable difference between validation score and test score, we hypothesize that the test set has a different distribution than the training set, thus, we try to incorporate the new data and re-train (probably not a full re-training in an online learning setting). Otherwise (2), we can


Combine all the cities into one dataset, then split it into train and test sets (later, validation set will be peaked from train set), i.e. 

  train = A_train U B_train U ...
  test = A_test U B_test U ...


This way, the information from all cities will be used for scaling. We may under- or over-sample some cities to have a data set with equal representatives from each city.
Or, build a separate (specialized) model for each city (or a group of similar cites) which bypasses the "train on A_train, and  test on B" problem all together.