Is it common to buy a gift for a supervisor when he invited me to a dinner?
No, there is no reason for buying a gift. The most you might do is send a follow-up email thanking your supervisor for the dinner.

Having lunch/dinner with colleagues isn't uncommon, and they usually aren't all that formal unless there is a special occasion (e.g., it is banquet organized by the department/university).