What are some good resources for strengthening my theoretical foundation for machine learning?
I suggest 
Probability and Computing: Randomized Algorithms and Probabilistic Analysis by Mitzenmacher and Upfal. Probability is at the foundation of machine learning and it's one of the weakest points for many beginners, in my experience.