Structuring groupcontent with taxonomy
If the number of group types is relatively small and consistent, you might consider adding several taxonomy term fields to the base groupcontenttype, each referencing the relevant grouptype specific vocabulary, and then hiding the fields that are not relevant to a group's type.

If you hid the field on the node edit page using a switch statement (on group type) in hook_form_alter, then ensured that empty fields were not displayed on the node, then both visitor and admin would not see irrelevant fields.