Speed decay proof for L2 regularization and non-normalizied weight initiation I'm working through http://neuralnetworksanddeeplearning.com/chap3.html,
textbook by Michael Nielsen.

While most problems are rather straightforward, I'm very stuck on these two questions:

Sketch a heuristic argument that: (1) supposing $\lambda$ is not too small, the first epochs of training will be dominated almost entirely by weight decay; (2) provided $\eta\lambda \ll n$ the weights will decay by a factor of $ \exp(−\eta\lambda/m)$ per epoch;

To give a self-sufficient context, the weight decay function is:

$w \rightarrow \left(1-\frac{\eta \lambda}{n}\right) w -\frac{\eta}{m}
  \sum_x \frac{\partial C_x}{\partial w}$,

and cost function is

$C = -\frac{1}{n} \sum_{xj} \left[ y_j \ln a^L_j+(1-y_j) \ln
(1-a^L_j)\right] + \frac{\lambda}{2n} \sum_w w^2$

(Classic logloss)

Weights in the network are initiated as $N(0,1)$.

My intuition for (1) goes as follows: As weight initialisation is non-normalized by size of previous layer, activation values are likely to be either $1$ or $0$ with equal probability (which is pointed out by the author). $y_j$ has $0.1$ probability of being 1. Given that $\frac{\partial C_x}{\partial w} = a^{L-1}_k(a^L_j - y_j)$ for logloss and sigmoid neurons, expected value of $\frac{\partial C_x}{\partial w}$ on the first iteration is $0.2$. 
Given that $\lambda$ is arbitrary, we can be sure that 
$|\frac{\eta \lambda}{n} w|$ can be larger than $|\frac{\eta}{m}
  \sum_x \frac{\partial C_x}{\partial w}|$

Despite questions being rather informal, I don't think this solution is a good one.

As for the (2), I lack any ideas whatsoever, except driving $\frac{\eta \lambda}{n}$ to zero, but that seems to have zero chance of yielding the required result.

I hope someone can give me a hand in this.
Thank you!

P.S. There is a chance I'm going to the wrong overflow, but I used
https://meta.stackexchange.com/questions/130524/which-stack-exchange-website-for-machine-learning-and-computational-algorithms
question as a guidance for overflow choice.

A note: I had asked similiar question here, but that might not be the best overflow for this question. I'll put a link to whichever get's answered first :)
Ok, I have found the source of confusion, thanks to @Emre for the insight.
The task talks about epochs of training, while I was trying to solve one batch update. I think this confusion is statistically possible enough to validate existence of this question :)