Why are Python 2 and 3 so incompatible with each other?
The saying goes, "In C++11, 11 is the number of legs the stapled onto the dog to make a better octopus".

As with basically everything in Computer Science, this is all about tradeoffs.

If you focus on backwards compatibility, you have potentially conflicting features interacting with each other. You're stuck with the design choices that were initially made, when you didn't have the knowledge and data that years of production use provide.

Breaking changes, conversely, have the disadvantage of breaking old code, though with Python 2, usually files could be converted mechanically.

Now, there are also questions of "what makes a language?"

From a semantics point of view, every time you add a new feature to a language, you have effectively defined a new language with different semantics. There are nice formalisms for dealing with this: for example, some extensions to languages are conservative, meaning you're only expressing features that the start language could express.

Python 2 to 3 was not an extension, let alone a conservative one. So from a theory point of view, they're different languages, that happen to share a name, most syntax, and a real-life community.

Java broke backwards compatability when they decided to be a different language from C++ or SmallTalk or Eiffel. At the end of the day, the lines between languages and dialects and versions can be blurry, and the decision will come down to what is viewed as the most pragmatic for developers.