Criticism by referee
Insufficient novelty is definitely good enough reason to reject a paper - otherwise for example I can just repackage the proof of Pythagoras's theorem from the nearest mathematics textbook I can get my hands on, and publish that (I can publish that multiple times too).

To publish your results you need to convince the editor that the result actually matters. You write that they "significantly advance our understanding of certain mathematical objects". That's the key point you want to elaborate on. How exactly do they advance our understanding? Do they show that certain approaches to the open questions in the field are more likely / less likely to be successful? Do they lead to any partial results?

It's impossible to tell how other (not necessarily lesser) journals will react, since peer review is an inherently stochastic process, but if I'm the author I would look carefully at elaborating on the above questions before submitting it to another journal.