Understanding the parameters of a simple neural network in Pytorch
I have defined a simple NN as follows in Pytorch: 

my_model = nn.Sequential(
            nn.Linear(1, 5),
            nn.Tanh(),
            nn.Linear(5, 1)) 


I then iterate through the parameters and check their sizes:

[parameter.shape for parameter in my_model.parameters()]


I get:

[torch.Size([5, 1]), torch.Size([5]), torch.Size([1, 5]), torch.Size([1])]



I'm confused as to why the last Size is 1. Shouldn't there be 5 bias values as well, going out from the hidden layer into the output layer? 

Um, so it looks like you are defining a two layers network.

The input size is [1], and the hidden layer size is [5], and next, the hidden layer is connected to the output layer, the output layer size is [1].

It should look like:

       *   &lt;- input
   / / | \ \
   * * * * *  &lt;- hidden layer
   \ \ | / /
       *   &lt;- output


So for the hidden layer, there are 5 values as bias, for the output layer, there is only one value as bias. Sounds reasonable?
