Is my general understanding of finding weights correct?
This is interesting !?!
The way you think about it is as solving a linear system equation, which is not exactly the case.
Imagine if you have thousands of data points , and you try to find the set of weights that solves ( satisfies) these points, what if an exact solution is not exist. What if there is no set of weights that exactly solves all these points. It becomes an optimization problem. What is the set of weight that minimizes the error. In this case the error is the difference between the actual output and the output you get by using the current weights. 
In fact, in current machine learning technology, it is difficult even to construct such a linear system.
We use optimization techniques to solve this and find the optimal set of weights