Guiding principle of Neural Network structure building I study Neural Networks and I pretty much understand the logic of the structure with layers, activation functions and connections. But a fundamental question is not clear: how does one put together the actual structure from the several possible combinations? It seems me that most people just combine these quite randomly without any proper reason (or with brute force "grid search" in better cases).

I've read several tutorials but nobody explained the reason of choosing "sigmoind" activation function over "tanh" for instance, not to mention which one to pick in case of different activation functions in different layers. I stress that I understand the working of these functions by themselves, but the logic of the order of them is quite a puzzle.
You would use a sigmoid if you want your activations to be between zero and 1 and tanh between -1 and 1, this is important for the final layer, but for a hidden layer there is not much difference between them. 

Here is an article on how to build a neural network in a more systematic way.

The basic principle is to start with only one layer, and make it bigger and bigger until you see satisfactory results, if you don't, add another layer and start over. Oh and make sure you use batch normalization, and don't use sigmoids or tanh's, use PreLUs or ELUs:
https://www.linkedin.com/pulse/keras-neural-networks-win-nvidia-titan-x-abhishek-thakur?trk=prof-post

In general:

Sigmoids: use for output layer which are probabilities with logloss loss function, and for gates in RNNs. 

Tanh: use for updating the state in RNNs (linear-like activations might make it unstable), or for an output layer with variables between -1 and 1.

Rectifiers: use for activations in hidden layers of feed-forward network.