What is the meaning of an icon that appears next to the Location field in Calendar app?
You can find what the exclamation point means in Calendar Help>View events>Calendar symbols:



The explanation "the item isn’t recognized as a valid entry by the server" is somewhat misleading. Even when the event is created using Google's web interface (so you would expect the server validated your location information) Calendar complains about the location and shows the dreaded exclamation point.

There seems to be some kind of mismatch between the information Google calendar exports through CalDAV and what Calendar expects. This is probably also the reason why location information in an imported Google calendar is shown inside a bubble:



That's not the case for iCloud Calendar events.

A Google search shows many posts of other people who have the same problem. Nobody seems to have found a solution, though. I don't think you can do much about it.