The bass riff at 0:50 in "I just did a bad thing" sounds SO GOOD and I can't get it out of my head, what makes it sound so good?
well, it's not behaving as a bass, at that moment. it's behaving as a lead melody riff, to supplement the vocals/lyrics. one weird trick i've learned about songwriting is that non-lyrics make the lyrics stronger, and lyrics make the non-lyrics stronger. Also not just the lyrics stops for that riff, the whole arrangement stops for it. so this is certainly the reason you have noticed it, and are thinking about it