Programatically adding Taxonomy text field
The only way to achieve adding a text field to a Taxonomy Vocabulary is to place it in the hook_enable() function of your module. 

If it's outside of that the "Text" module won't be loaded and you'll receive the error above.

If anyone has a way around that please let me know.