HOW TO: Deep Neural Network weight initialization
As far as I know the two formulas you gave are pretty much the standard initialization. I had done a literature review a while ago, I copied it below if interested.



[1] addresses the question:

First, weights shouldn't be set to zeros in order to break the symmetry when backprogragating:


  Biases can generally be initialized to zero but weights need to be initialized carefully to break the symmetry between hidden units of the same layer. Because different output units receive different gradient signals, this symmetry breaking issue does not concern the output weights (into the output units), which can therefore also be set to zero.


Some initialization strategies:


[2] and [3] recommend scaling by the inverse of the square root of the fan-in
Glorot and Bengio (2010) and the Deep Learning Tutorials use a combination of the fan-in and fan-out:


for sigmoid units: sample a Uniform(-r, r) with $r=\sqrt{\frac{6}{\text{fan-in}+\text{fan-out}}}$ (fan-in is the number of inputs of the unit).
for hyperbolic tangent units: sample a Uniform(-r, r) with $r=4 \sqrt{\frac{6}{\text{fan-in}+\text{fan-out}}}$ (fan-in is the number of inputs of the unit).

in the case of RBMs, a zero-mean Gaussian with a small standard deviation around 0.1 or 0.01 works well (Hinton, 2010) to initialize the weights.
Orthogonal random matrix initialization, i.e. W = np.random.randn(ndim, ndim); u, s, v = np.linalg.svd(W) then use u as your initialization matrix.


Also, unsupervised pre-training may help in some situations:


  An important choice is whether one should use
  unsupervised pre-training (and which unsupervised
  feature learning algorithm to use) in order
  to initialize parameters. In most settings
  we have found unsupervised pre-training to help
  and very rarely to hurt, but of course that
  implies additional training time and additional
  hyper-parameters.


Some ANN libraries also have some interesting lists, e.g. Lasagne:

Constant([val]) Initialize weights with constant value.
Normal([std, mean]) Sample initial weights from the Gaussian distribution.
Uniform([range, std, mean]) Sample initial weights from the uniform distribution.
Glorot(initializer[, gain, c01b])   Glorot weight initialization.
GlorotNormal([gain, c01b])  Glorot with weights sampled from the Normal distribution.
GlorotUniform([gain, c01b]) Glorot with weights sampled from the Uniform distribution.
He(initializer[, gain, c01b])   He weight initialization.
HeNormal([gain, c01b])  He initializer with weights sampled from the Normal distribution.
HeUniform([gain, c01b]) He initializer with weights sampled from the Uniform distribution.
Orthogonal([gain])  Intialize weights as Orthogonal matrix.
Sparse([sparsity, std]) Initialize weights as sparse matrix.




[1] Bengio, Yoshua. "Practical recommendations for gradient-based training of deep architectures." Neural Networks: Tricks of the Trade. Springer Berlin Heidelberg, 2012. 437-478.

[2] LeCun, Y., Bottou, L., Orr, G. B., and Muller, K. (1998a). Efficient backprop. In Neural Networks, Tricks of the Trade.

[3] Glorot, Xavier, and Yoshua Bengio. "Understanding the difficulty of training deep feedforward neural networks." International conference on artificial intelligence and statistics. 2010.