Why are the initial weights of neural networks randomly initialised?
You shouldn't assign all to 0.5 because you'd have the &quot;break symmetry&quot; issue.

http://www.deeplearningbook.org/contents/optimization.html
Perhaps the only property known with complete certainty is that the initial
parameters need to “break symmetry” between different units. If two hidden
units with the same activation function are connected to the same inputs, then
these units must have different initial parameters. If they have the same initial
parameters, then a deterministic learning algorithm applied to a deterministic cost
and model will constantly update both of these units in the same way. Even if the
model or training algorithm is capable of using stochasticity to compute different
updates for different units (for example, if one trains with dropout), it is usually
best to initialize each unit to compute a different function from all of the other
units. This may help to make sure that no input patterns are lost in the null
space of forward propagation and no gradient patterns are lost in the null space
of back-propagation.