r2.15 cherry-pick: a1fd78b23b1 "Potential fix - try deleting old DeviceCompiler if new PjRtClient found for TPU."
Refer to the original commit: https://github.com/tensorflow/tensorflow/commit/a1fd78b23b15e78272f636449cadde2921936fe7