Why is novelty mandatory for a Ph.D. degree?
Novelty is a basic requirement in research because without novel results, the work is of at best limited usefulness. It's not useless - review papers are useful - but it's less interesting to the people at the frontier. In the case of a PhD, it is supposed to signify a student is now capable of doing research. Research (as opposed to literature review) necessarily involves new and novel results. Therefore novelty is a requirement.

If I understand your last paragraph right, you're wondering about open problems which might be too complex to solve in one go. But in that case, you still get new and novel results - they might be partial results, but they're still results. The same goes for research using new methods that turn out not to work: knowing where not to look is not as useful as knowing where to look, but by narrowing the parameter space, it's still useful.