title: Is it possible to "mask out" unwanted inputs and feature maps in PyTorch so that PyTorch simply ignores them?
body: <p>Recently, I've been playing around with PyTorch and I'm curious if it's possible to modify a Fully Convolutional Network so that it operates on a nonrectangular region of an image <strong>only</strong>.</p>
  <p>Let's take <a href="https://i.stack.imgur.com/8cIHH.png" rel="nofollow noreferrer">an arbitrary image</a>. Within this image, there is a non-rectangular region of pixels that I already know that I am interested in (the blue region of <a href="https://i.stack.imgur.com/8cIHH.png" rel="nofollow noreferrer">this image</a>, where each square is conveniently the size of the kernel). Is there a way for me to tell a network to only operate on the blue region and not bother with the rest of the image (yellow region)?</p>
  <p>I know I can just turn the unwanted part of the image to zeroes, but that seems like a waste; the network will still slide a convolutional kernel over the zeroed-out part of the image uselessly. <strong>Is it possible to provide a mask or something to tell PyTorch</strong> &quot;don't bother sliding the conv operator over this region of the image; everything downstream is automatically 0&quot;?</p>
  <p>I've read about geometric neural networks for graph inputs and such, but in my head, it seems a bit overkill for this idea.</p>