Is it bad practice to track all in the configuration module
I have tried it in Drupal7 (not on Drupal8).

Logically it should be good practice to track all the configurations. However, not all the "configurations" are actually configurations. For example there are persistent variables or cached values that have nothing to do with the configuration. So, tracking all of them would undermine the aim of configuration tracking.

Besides this, the UI becomes infinitely slow and unresponsive when you select all the configurations. I guess it is because it tries to find out dependencies of each configuration and does it in an inefficient way. You can try to handle them from the command-line (with drush) but I doubt that it would be more efficient.

So, I think that the best option is to cherry pick the relevant configurations. But once you start cherry-picking them, you might as well use features, configuration management loses its meaning.

This is basically what I did on a project of mine: I started with configuration management, and ended up using features, which by the way was a tedious process.