How neural style transfer work in pytorch?
Question 1


  My question is how the second method maintained the height and widght of style tensor (300, 374)?


Applying the second piece of code does not call all layers but only the ones listed in the layers dict. In contrast to that the first method runs through all layers. Now, if you look at the network architecture:

IN: model
OUT:
Sequential(
  (0): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (1): ReLU(inplace=True)
  (2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (3): ReLU(inplace=True)
  (4): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  (5): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (6): ReLU(inplace=True)
  (7): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (8): ReLU(inplace=True)
  (9): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  (10): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (11): ReLU(inplace=True)
  (12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (13): ReLU(inplace=True)
  (14): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (15): ReLU(inplace=True)
  (16): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (17): ReLU(inplace=True)
  (18): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  (19): Conv2d(256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (20): ReLU(inplace=True)
  (21): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (22): ReLU(inplace=True)
  (23): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (24): ReLU(inplace=True)
  (25): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (26): ReLU(inplace=True)
  (27): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  (28): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (29): ReLU(inplace=True)
  (30): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (31): ReLU(inplace=True)
  (32): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (33): ReLU(inplace=True)
  (34): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  (35): ReLU(inplace=True)
  (36): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
)


You can see that in fact all max pooling layers are being skipped with the second method. Accordingly, your input does not get pooled, i.e. height and width remain unchanged. (the CNN layers in this network do not change height and width but only the channels) 

If you slightly change the second piece of code to:

x = style
x = x.unsqueeze(0)
for name,layer in model._modules.items():
    x = layer(x)
    if name in layers:
        print(x.shape)


It runs through all layers again and gives:

torch.Size([1, 64, 300, 374])
torch.Size([1, 128, 150, 187])
torch.Size([1, 256, 75, 93])
torch.Size([1, 512, 37, 46])
torch.Size([1, 512, 37, 46])
torch.Size([1, 512, 18, 23])




Question 2


  Why is the optimizer initialization in neural style transfer different from other neural network tutorials?


In image style transfer learning you do not optimize the weights (model parameters) but the target image. Since you would like the target image to resemble your content and style pictures as close as possible. All weights (model parameters) are being kept constant. Therefore, the optimizer needs to receive the target image as an input and provide updates for it (and not for your model parameters like usually).  



Question 3


  What is reason behind this optimizer = torch.optim.Adam([target],lr=0.007) >


See Question 2. The target is initialized in line 69. Typically, you use the content image as a start:

target = content.clone().requires_grad_(True).to(device)


As you can see it also activates the gradient for the target.
And this target image is then being optimized in order to minimize the loss
total_loss = content_wt*content_loss + style_wt*style_loss.



A good read on the general approach is Image Style Transfer Using Convolutional Neural Networks.