Programmatically adding fields to specific vocabularies
It turns out that with taxonomies, vocabularies are bundles of taxonomy_term entities.

So, it turned out to be something like this:

Assuming a taxonomy machine name of example_taxonomy

  $field_example = array(
    'field_name' =&gt; 'field_example',
    'cardinality' =&gt; 1,
    'type' =&gt; 'text',
  );
  field_create_field($field_example);

  $field_example_instance = array(
    'field_name' =&gt; 'field_example',
    'label' =&gt; st('Example Text Field'),
    'bundle' =&gt; 'example_taxonomy',
    'entity_type' =&gt; 'taxonomy_term',
    'widget' =&gt; array(
      'type' =&gt; 'text_textfield',
    ),
  );
  field_create_instance($field_example_instance);


So, yeah, it turns out that taxonomy_terms are the entities and vocabularies are the bundles...