Should I share my external grant with my supervisor? He reasons that he supported me earlier
Take into account that ethically (and often legally) speaking you cannot be reimbursed more than once for a conference trip.

I am assuming that the ACM grant was a travel grant to cover the expenses for the conference, right?

The most reasonable option, given that you found external funding, is to return the money from the university grant + your professor's, using that same ACM grant.

It's not your professor's business to decide what you do with the remaining money from that grant, if there is any.

Your professor should actually be happy that you were able to get external funding so you could attend at 0 cost for the University and his team.

Saying that you have to pay back more to him than he "loaned you" because "he supported you" tells a lot about your professor already: he does not support or help students because he considers it's his duty or job, but in exchange for future rewards.

Unfortunately, this bullying, favor/grant-exchanging, mob-behavior is not new to many of us at all.