Inner tube blows up due to rim holes
I figured out a solution. The thing that is supposed to stop the inner tube from expanding into spoke holes is called a rim tape. The tape that came with the bike is really flimsy. I'll purchase and apply a high pressure rim tape. 

Hopefully that would fix the problem.

Update:
I've decided to go for a cotton rim tape as they're more durable than rubber ones. And after installing the tape, hey presto, the problem is gone.