Adding bias in deconvolution (transposed convolution) layer
We are going backwards in the sense that we are upsampling and so doing the opposite to a standard conv layer, like you say, but we are more generally still moving forward in the neural network. For that reason I would add the bias after the convolution operations. This is standard practice: apply a matrix dot-product (a.k.a affine transformation) first, then add a bias before finally applying a non-linearity.

With a transpose convolution, we are not exactly reversing a forward (downsampling) convolution - such an operation would be referred to as the inverse convolution, or a deconvolution,  within mathematics. We are performing a (transpose) convolution operation that returns the same input dimensions that produced the activation map in question, with no guarantee that the actual values are identical to the original input.

You can see from the animations of various convolutional operations here, that the transpose convolution is basically a normal convolution, but with added dilation/padding to obtain the desired output dimensions. The trick is to retain the mappings of localisation between the pixels.

In the paper from which those animations are taken, they explain how a transpose convolution is essentially the convolution steps performed in reverse:


  ..., although the kernel w defines a convolution whose forward and
  backward passes are computed by multiplying with $\textbf{C}$ and $\textbf{C}^T$ respectively, it also defines a transposed convolution whose forward and backward passes are computed by multiplying with $\textbf{C}^T$ and $(\textbf{C}^T)^T = \textbf{C}$ respectively.


One other source to back up my opinion: in the PyTorch implementation it seems the bias is added the output of the convolution's result.