Avoiding conferences: Potential effects on an academic career and mitigation strategies
The primary argument against your strategy arises when you go up for promotion: if senior people in your field can't really "tie" you to your work—in other words, if you're someone who has published some really good papers, but are otherwise "invisible" in the community, it will be hard to write the positive letters of reference needed for a successful tenure case.

However, that is not to say that the strategy of focusing on disseminating research is not important—indeed, the two strategies need to go hand-in-hand! So I would argue that networking will become more critical once you have independent work of your own to share.