Purpose of backpropagation in neural networks
Is backpropagation just fancy term for weights being optimized on every iteration?


Almost. Backpropagation is a fancy term for using the chain rule. 

It becomes more useful to think of it as a separate thing when you have multiple layers, as unlike your example where you apply the chain rule once, you do need to apply it multiple times, and it is most convenient to apply it layer-by-layer in reverse order to the feed forward steps.

For instance, if you have two layers, $l$ and $l-1$ with weight matrix $W^{(l)}$ linking them, non-activated sum for a neuron in each layer $z_i^{(l)}$ and activation function $f()$, then you can link the gradients at the sums (often called logits as they may be passed to  logistic activation function) between layers with a general equation:

$$ \frac{\partial L}{\partial z^{(l-1)}_j} = f'(z^{(l-1)}_j) \sum_{i=1}^{N^{(l)}} W_{ij}^{(l)} \frac{\partial L}{\partial z^{(l)}_i}$$

This is just two steps of the chain rule applied to generic equations of the feed-forward network. It does not provide the gradients of the weights, which is what you eventually need - there is a separate step for that - but it does link together layers, and is a necessary step to eventually obtain the weights. This equation can be turned into an algorithm that progressively works back through layers - that is back propagation.


  To be more precise, what's the point of automatic differentiation when we could simply plug in variables and calculate gradient on every step, correct?


That is exactly what automatic differentiation is doing. Essentially "automatic differentiation" = "the chain rule", applied to function labels in a directed graph of functions.