How to create Partially Connected NNs with prespecified connections using Tensorflow? I'd like to implement a partially connected neural network with ~3 to 4 hidden layers (a sparse deep neural network?) where I can specify which node connects to which node from the previous/next layer. So I want the architecture to be highly specified/customized from the get-go and I want the neural network to optimize the weights of the specified connections, while keeping everything else 0 during the forward pass AND the backpropagation (connection does not ever exist).

I am a complete beginner in neural networks. I have been recently working with tensorflow &amp; keras to construct fully connected deep networks. Is there anything in tensorflow (or something else) that I should look into that might allow me to do this? I think with tf, I should be able to specify the computational graph such that only certain connections exist but I really have no idea yet where to start from to do this...

I came across papers/posts on network pruning, but it doesn't seem really relevant to me. I don't want to go back and prune my network to make it less over-parameterized or eliminate insignificant connections. 

I want the connections to be specified and the network to be relatively sparse from the initialisation and stay that way during the back-propagation.
So I've done some more research after posting this and it looks like treating things as separate neural nets and connecting them later in subsequent hidden layers is a good idea. If anyone has other ideas that might be helpful, I'd love to hear, though!