How can I fill in the missing background I need to be a good computer scientist?
Let me give you a bit of perspective, as well as assurance. In mathematics it is no longer possible for a person to be knowledgeable in every aspect of the art. That possibility ended early in the 20th century. Henry Poincaré is often given as the last person who was conversant in everything mathematical. Math has evolved a huge amount in the interim. So, even Poincaré didn't know all that we now know about math. 

In any field there is a tendency toward specialization. Research normally requires this. Over time, being a good researcher requires less and less of the whole both because of expansion of the field and the much narrower needs of a specific sub-topic. 

While it is good to know a lot and your goal of expanding your horizon is a good one, don't be discouraged if others, especially other researchers have different ideas, use different tools, and especially utilize different thought patterns than you do. 

Probably the most effective means, given that you like your research area and that it rewards you properly, is to branch out from there, studying similar things that you don't explicitly need, but which might come in handy. But rather than focus on the details of the related topics, pay special attention to whether the thought patterns you hear discussed are somewhat different from your own. Think Different was a slogan (Apple Inc.), but it has a deeper meaning. 

Another way to expand your horizon from a given base is to study something very different from what you know, but to which your current knowledge might be applied. 

I don't know of CS has reached the stage yet where you can't know everything, but I suspect we are pretty close to the boundary. You can have a broad education, and you should. You can also go deep into one or a few areas, as you also should. But there is (likely) too much there to do it all. 

Modern Academia tends to reward specialists, as I'm sure you are already aware.