Sparse connections in feedforward network tensorflow or pytorch? I want to create sparse feed-forward networks in Pytorch and Tensorflow, i.e., say each node is only connected to k number of neurons of the next layer where k is strictly less than the total number of nodes in the next layer. But all the tutorials/examples I have seen so far are for fully connected feed-forward networks. Is there any way to construct such sparse networks in these tools?
We don't use sparse connections in feedforward to make use of an efficient matrix product operations.
But If you want to model that aspect, see dropout