Boot from usb take forever
OP here, it was actually pretty simple to fix. I just hard reset it and try again. Had to hard reset twice before the setup succeed.

I don't know why but my guess is that El Capitan is quite newer than my laptop so it takes a few tries to iron out all the conflicts.