View will not group the taxonomy terms
What do you have the "sort" criteria set to? Also, I find the default Views grid-view to be quite awful. I use the Views Fixed Grid module for any grid views. It will give you better output and greater control in my humble opinion.