How to build neural network in this structure？with different nodes connects to different number of nodes in next layer
I only know how to use the built-in network like RNN of LSTM in PyTorch. But they tend to deal with every node in the previous layer that will give information to all nodes in the next layer.
I want to do something different but don't know how to code it myself. Like in this figure: the node a maps to all [d, e, f]  three nodes in layer 2, while node b maps to [e,f] and node c only maps to [f]. As a result,  node d will only contain information from a,  while e will contain information from  [a, b]. And f will contain information from all nodes in the previous layer.
Does anyone know how to code this structure? PLS give me some insight I'll be very grateful :D
Structure

When you have a layer that looks like Fully-Connected layer but with custom connectivity, use a mask with proper structure.
Let's say x = [a, b, c] is your 3-dim input and W denotes the connectivity matrix.
&gt;&gt; x
tensor([[0.1825],
        [0.9598],
        [0.2871]])
&gt;&gt; W
tensor([[0.7459, 0.4669, 0.9687],
        [0.9016, 0.4690, 0.0471],
        [0.5926, 0.9700, 0.5222]])

then W[i][j] points to the connecting weight between jth input and ith output neuron. To build the structure similar to your toy example, we would make a mask like this
&gt;&gt; mask
tensor([[1., 0., 0.],
        [1., 1., 0.],
        [1., 1., 1.]])

Then you can simply mask the W
&gt;&gt; (mask * W) @ x
tensor([[0.1361],
        [0.6147],
        [1.1892]])

Note: @ is matrix multiplication and * is pointwise multiplication.
