Is a deep technical understanding of neural networks required outside of research?
It depends on what exactly you want to be. You don't need to be mathematician if you just want to run neural networks. Most data scientists don't understand the mathematics, but they know how to run machine learning frameworks. Generally, only PhDs understand the mathematics.

In the industry, most job positions in machine learning (e.g software engineer, big data engineer etc) don't require advanced mathematics. But you should be very comfortable in programming, and understand simple things like matrix manipulation. Data engineers in the industry generally have good programming skills, but they can't read mathematical equations (or just simple equations).

However, quantitative positions in machine learning do require significant mathematics. That happens both in research and industry.

Good news --- if you don't have a PhD (e.g statistics) nobody would expect you understand the mathematics.

Summary:


If your job is more like a development position, mathematics is an advantage but not absolutely necessary
If your job is research, you do need advanced mathematics understanding (e.g. PhD)
If your job is not research but still quantitative based. Mathematics is very important.


PS: I work in research. Everybody I know who works in machine learning has a PhD.