Tactics to avoid feeling overwhelmed by machine learning
We have to climb up a steep learning curve when we learn about machine learning. Your question is quite general: One of the tactics I use when learning is divide and conquer. Get some coarse overview about the whole area, then pick some particular area and dig deeper only there.
Perhaps the question is too general, the best tactic may vary and depend on the area you address. 
But I am not sure if learning the math is always helpful (although it may always be interesting for those who care). 

The algorithms can often be applied in a black box approach, and it may be sometimes not necessary to understand an algorithm in math terms (white box), but sufficient to know it's function, strengths and weaknesses (black box).  

You may be the first one that tests that algorithm for the domain, so pure experimentation is useful in the end.