Phone only works with busted battery
So I found out why my new battery isn't working, it had a transparent sticker on the power pins (and only on the pins). It was virtually invisible in broad daylight and there was no indicator on the package anywhere that there is such a sticker there (much less that it needs to be removed). It was only when I looked on the battery at a certain angle with electric light that I noticed it.

I'm not an expert on mobile phones and I change phone batteries pretty much never, so maybe that sticker there is common knowledge for everybody else, but the funny thing is, the guy at the shop where I bought the new battery didn't notice it either (and he should have been an expert).