Purpose of weights in neural networks
The reason for weights in machine learning is actually a lot easier than it seems. It's the way by which our model learns some underlining function and performs the classification or regression. We tune these weights in order to model some underlining function which can map our input to a desired output. Either a class in classification, or a range of values in a regression.

Let's look at an easier machine learning model so that we can understand why weights are needed. 

Linear Regression

This is just a straight line which splits data into two sections. Let us apply this model in 2D, where we have two features $x = [x_1, x_2]$, for example weight and height. And labels $y$ which will split our data into $y \in \{men, women\}$.

A random line in this space is defined as

$0 = -2x_2 + x_1 + 1$.

Assume this is our boundary line. Let's just assume that women fall above this line and men fall below it. Like this



Now if we get a new data point $x_{new} = [2, 8]$ we will label this as being a woman. So the entire decision is based on the numbers $1, -2, 1$ from our linear equation. We need to tune these values using the training data. We usually call these trainable parameters the weights $w$ associated with the features $x$ and we also add a bias $b$. In general the linear separator in 2D is 

$0 = w_1x_1 + w_2x_2 + b$.

Our predicted label is $\hat{y} = w_1x_1 + w_2x_2 + b$. If $\hat{y} &gt; 0$ then woman, else man.

In $n$ dimensions this can be written as a matrix multiplication as 

$\hat{y} = w^Tx + b$

Obviously, a linear separator is not sufficient for most classification tasks. Things are not always linearly separable. So we use more complex models. 

Neural networks

In neural networks each node is associated with a function much like the linear separator. However, we will use the sigmoid function

$\sigma(w^Tx) = \frac{1}{1 + exp^{-(w^Tx + b)}}$.

The weights here have the same effect. They will modulate the input values $x$ such that we are able to learn some classification or regression. 



How do the weights affect the decision boundary. We want the two different classes, circles and x's, to be on opposite sides of this boundary in order for us to be able to correctly classify them.

The weights are trained iteratively using gradient descent. We can see that the decision boundary starts off terribly and then gets progressively better.

$0 = -1.0 - 1.0x_1 - 1.0x_2$



$0 = -16.0 - -39.0x_1 - 71.0x_2$



$0 = -36.0 - -94.0x_1 - 61.0x_2$



$0 = -83.5 - -134.0x_1 - 76.0x_2$



$0 = -88.5 - -114.0x_1 - 98.5x_2$



As you can see we changed the weights until we were able to find this ideal boundary between the two classes. If you want to find our more about how the gradient descent algorithm works to tune these weights you can look here.