How do I pass a color image through a Pytorch convolutional layer with custom filter?
I'm trying to visualize what happens when a color image passes through a convolutional layer. For that, I'm setting custom weights with zeroes and ones. The problem I'm facing is that I'm losing the 3D channels, and get a 1D channel after passing the data through the layer. 

import requests
from io import BytesIO
from PIL import Image
import torch
import torch.nn as nn
import matplotlib.pyplot as plt
import numpy as np

link = 'https://audimediacenter-a.akamaihd.net/system/production/media/85094/images' \
       '/2a4e98976b1f9088fe6ae883f2f29e4d8f3ed473/A1912967_x500.jpg?1575885688'

r = requests.get(link, timeout=10)
im = Image.open(BytesIO(r.content))

pic = np.array(im)

horizontal_filter = torch.zeros(5, 5)
horizontal_filter[2, :] = 1

print(horizontal_filter)


This is my custom filter:

tensor([[0., 0., 0., 0., 0.],
        [0., 0., 0., 0., 0.],
        [1., 1., 1., 1., 1.],
        [0., 0., 0., 0., 0.],
        [0., 0., 0., 0., 0.]])


Now I'm using the custom filter and repeat it to fit 3 channels.

hz = nn.Conv2d(in_channels=3,
               out_channels=3,
               kernel_size=5,
               stride=1,
               bias=None)

hz.weight.data =  horizontal_filter.type('torch.FloatTensor').repeat(1, 3, 1, 1)

print(hz.weight.data.shape)


This is the shape of the filter:

torch.Size([1, 3, 5, 5])


I pass it through the convolutional filter and I'm losing the 3 channels:

zz = hz(torch.tensor(pic[None, ...]).permute(0, 3, 1, 2).type('torch.FloatTensor'))

print(np.transpose(zz.detach().numpy(), (0, 2, 3, 1)).shape)


(1, 329, 496, 1)


If I plot it, I don't have the colors anymore.

z = np.transpose(zz.detach().numpy(), (0, 2, 3, 1))[0, :, :, 0]

f, axarr = plt.subplots()
axarr.imshow(z)
plt.show()




tl;dr: How do I pass a 3D picture through a convolutional layer and return an image with 3 channels? 

The issue is that you are not repeating the channels enough. Since you have 3 input and output channels, the Conv weight matrix would be 3x3x5x5. Since you had set it to 1x3x5x5, it was able to output only 1 channel.

You need to make the following change

hz.weight.data =  horizontal_filter.type('torch.FloatTensor').repeat(3, 3, 1, 1)



Because of your filter, your output would have a max value of ~3700. So to view, divide by the max using z = z/np.max(z) and then you get 

