App Store Optimization: Individual localization for English (U.S.), Australia, Canada and U.K.?
After almost a year of observation, it appears that it does not have a negative impact on ASO.

This is because my primary language English (U.S.) is used for Australia, Canada and U.K. as well.

So the other localizations were redundant, given that they were a copy of English (U.S.) anyway.