How do I alter a function? I want to track taxonomy term tagging count in the database; for each taxonomy term, I want a term tagged count that stores the number of nodes using the taxonomy term. I know I can retrieve it with a query on taxonomy_index but it is too time consuming for me.

I've found taxonomy_build_node_index($node) and taxonomy_delete_node_index($node), which are called in taxonomy_node_insert(), taxonomy_node_update(), and taxonomy_node_delete() (node hooks).

I know a form can be altered with hook_fom_alter(). Is there a way to alter these functions as well?
Drupal doesn't have a way to generally replace a function in a module with a function implemented by another module.

In the case of hooks, you could alter the output of a hook with another hook, but in your case you don't even need this. Just implement hook_node_insert(), hook_node_update(), and hook_node_delete(), and add the code necessary for counting the tags used by nodes.

As said from Jaypan, you don't say in which way querying the taxonomy_index table would be too expensive. I am not sure it would be that expensive, since the table saves the taxonomy term IDs and the node IDs: You just need to count the table rows with a specific value for tid.