Is my general understanding of finding weights correct? I started a course in Deep Learning. I'm trying to make an example in order to explain to myself how the weights are found mathematically.
If what I wrote below is nonsense I'll be glad to hear an explanation. Thanks.

So, for a given image we do WX+b. We get some vector Y and then we compare it to a desired label vector L according to . I'm assuming that we calculate D with "Cosine Similarity". For simplicity S(Y)==Y. So what we're trying to do is to calculate  so it will be one.
Let’s say we have image X of the letter “a”  and two labels (“a”, “b”). Then  . We want to calculate W and b for which we will get such vector  that when we’ll insert it into  we’ll get zero. We convert X to a vector. Since we have 2 labels and size of the X is 9, the W and b are the following: .  So, we get: . This gives us the following system of equations: . So, now we need to solve the following .

If what I wrote above is not nonsense, I don't quite understand where finding minimum is applied?
This is interesting !?!
The way you think about it is as solving a linear system equation, which is not exactly the case.
Imagine if you have thousands of data points , and you try to find the set of weights that solves ( satisfies) these points, what if an exact solution is not exist. What if there is no set of weights that exactly solves all these points. It becomes an optimization problem. What is the set of weight that minimizes the error. In this case the error is the difference between the actual output and the output you get by using the current weights. 
In fact, in current machine learning technology, it is difficult even to construct such a linear system.
We use optimization techniques to solve this and find the optimal set of weights