How to deal with racial insensitivity in an academic environment?
In your question, However, he realized it and walked away immediately. and But after few hours, he came back and he was conversing with me like nothing happened

indicate to me that he felt sorry and didn't know exactly how to deal with it.

Professors are also human. They make the same mistakes others do.

I would suggest you to talk to him in person about this incident politely and seriously. Tell him that you are upset with the remarks he made earlier and explain to him why. Don't be emotional.

Whether or not he will apologize is up to him. If he doesn't offer apology to you after you talk to him, then talk to his superiors if you want the apology.