Add field to vocabulary itself
So the basic problem is that a vocabulary is not fieldable. In theory this is easy to fix, but I don't know if they lack this functionality because it would cause technical difficulties, or to avoid confusion users.

To make a vocabulary fieldable, implement hook_entity_info_alter:

function hook_entity_info_alter(&amp;$entity_info) {
  $entity_info['taxonomy_vocabulary']['fieldable'] = TRUE;
}