PyTorch 2d Convolution with sparse filters
I am trying to perform a spatial convolution (e.g. on an image) in pytorch on dense input using a sparse filter matrix.

Sparse Tensors are implemented in PyTorch. I tried to use a sparse Tensor, but it ends up with a segmentation fault.

import torch
from torch.autograd import Variable
from torch.nn import functional as F

# build sparse filter matrix
i = torch.LongTensor([[0, 1, 1],[2, 0, 2]])
v = torch.FloatTensor([3, 4, 5])
filter =  Variable(torch.sparse.FloatTensor(i, v, torch.Size([3,3])))

inputs = Variable(torch.randn(1,1,6,6))

F.conv2d(inputs, filter)


Can anyone just give me a hint how to do that?

Thanks in advance!
dymat

I know this question is outdated but I also know that there are still people looking for an answer (like myself) so here goes...

On sparse filters

If you'd like sparse convolution without the freedom to specify the sparsity pattern yourself, take a look at dilated conv (also called atrous conv). This is implemented in PyTorch and you can control the degree of sparsity by adjusting the dilation param in Conv2d.

If you'd like to specify the sparsity pattern yourself, to the best of my knowledge, this feature is not currently available in PyTorch. But you may want to check this out if you are ok with using Tensorflow. There is also a blog post providing more details on this repo.

On sparse input

A list of existing and TODO sparse tensor operations is available here.

This talks about the current state of sparse tensors in PyTorch.

This lets you propose your own sparse tensor use case to the PyTorch contributors.

But at the time of this writing, I did not see conv on sparse tensors being an implemented feature or on the TODO list. nn.Linear on sparse input, however, is supported.

And if you build a sparse tensor and apply a conv layer to it, PyTorch (1.1.0) throws an exception:

&gt;&gt;&gt; a = torch.zeros((1, 3, 2, 2), layout=torch.sparse_coo)
&gt;&gt;&gt; net = torch.nn.Conv2d(1, 1, 1)
&gt;&gt;&gt; b = net(a)
Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 338, in forward
    self.padding, self.dilation, self.groups)
RuntimeError: sparse tensors do not have is_contiguous
&gt;&gt;&gt; torch.__version__
'1.1.0'


Changing to a linear layer and it would work:

&gt;&gt;&gt; c = torch.zeros((1, 2), layout=torch.sparse_coo)
&gt;&gt;&gt; another_net = torch.nn.Linear(2, 1)
&gt;&gt;&gt; d = another_net(c)
&gt;&gt;&gt; d
tensor([[0.1944]], grad_fn=&lt;AddmmBackward&gt;)
&gt;&gt;&gt; d.backward()
&gt;&gt;&gt; another_net.weight.grad
tensor([[0., 0.]])
&gt;&gt;&gt; another_net.bias.grad
tensor([1.])

