Append feature maps to intermediate layers of network in PyTorch
I have the following PyTorch network architecture shown below. It takes in the input of size 16x8x2048, and gives an output of size 256x128x3. I have a feature map that I want to append along the channel dimension after every upsampling layer in the below architecture. I would be scaling the feature maps to the appropriate resolution before appending. How can I append these feature maps in the intermediate layers of the network?

model = []
model += [Conv2dBlock(2048, 256, 3, 1, 1, norm='bn', activation=activ, pad_type=pad_type)]
model +=  [nn.Upsample(scale_factor=2, mode='bilinear')] 
model +=  [Conv2dBlock(256, 128, 3, 1, 1, norm='bn', activation=activ, pad_type=pad_type)]
model +=  [nn.Upsample(scale_factor=2, mode='bilinear')]
model +=  [Conv2dBlock(128, 64, 3, 1, 1, norm='bn', activation=activ, pad_type=pad_type)]                        
model +=  [nn.Upsample(scale_factor=2, mode='bilinear')] 
model +=  [Conv2dBlock(64, 32, 3, 1, 1, norm='bn', activation=activ, pad_type=pad_type)]
model +=  [nn.Upsample(scale_factor=2, mode='bilinear')] 
model +=  [Conv2dBlock(32, 32, 3, 1, 1, norm='bn', activation=activ, pad_type=pad_type)]                        
model += [Conv2dBlock(32, 3, 3, 1, 1, norm='none', activation=activ, pad_type=pad_type)]                        
model = nn.Sequential(*model)


So the solution was just changing the way in which the network is written as pointed out by https://stackoverflow.com/users/10229754/mercury. The following change shows a way in which the network can be written so that is possible to append additional input. In the forward() function, the input can be directly appended using torch.cat()-

conv1 = Conv2dBlock(2063, 256, 3, 1, 1, norm='bn', activation=activ, pad_type=pad_type)
up1 = nn.Upsample(scale_factor=2, mode='bilinear')
conv2 = Conv2dBlock(256, 128, 3, 1, 1, norm='bn', activation=activ, pad_type=pad_type)
up2 = nn.Upsample(scale_factor=2, mode='bilinear')
conv3 = Conv2dBlock(128, 64, 3, 1, 1, norm='bn', activation=activ, pad_type=pad_type)
up3 = nn.Upsample(scale_factor=2, mode='bilinear')
conv4 = Conv2dBlock(64, 32, 3, 1, 1, norm='bn', activation=activ, pad_type=pad_type)
up4 = nn.Upsample(scale_factor=2, mode='bilinear')
conv5 = Conv2dBlock(32, 32, 3, 1, 1, norm='bn', activation=activ, pad_type=pad_type)
conv6 = Conv2dBlock(32, 3, 3, 1, 1, norm='none', activation=activ, pad_type=pad_type)

