ServiceProviderBase override "winner" - can this be done with module weight? It is noted that modules implementing ServiceProviderBase to alter defined services can 'lose' to other modules overriding the same service as there is no way to know which one should overtake in that situation.

Is this 'fixable' by making your modules weight higher than the others?
Yes, the serviceProviderClasses are discovered in DrupalKernel::discoverServiceProviders(), I've stopped the code there and the list was sorted by module weight and modules with the same weight alphabetically.