Limit the groups exposed in Group profile field
One way is to change the visibility of the groups to User and User Admin Only to hide the group display from all the public pages or profile.

If you don't want to do that, you might want a developer to add a js file on the profile rendering page and hide extra group values using javascript.