Is there a known convolutional net architecture to calculate object masks for images? I would like to train a convnet to do the following:


Input is a set of single channel (from black to tones of grey to white) pictures with a given object, let's say cars.
Target is, for every picture in the set, the same picture, however pixels are either black or white. The pixels corresponding to the car object are in white (i.e. intensity 255) and the pixels corresponding to the background are black (i.e. intensity 0).


After training I would like to feed the net with pictures of cars and I would like the prediction - the ideal prediction at least - to be a picture with pixels either black or white, where white corresponds to the object and black to the background.

I assume that the input layer is a 2D convolutional layer and the output layer is also a 2D convolutional layer, each one with as many neurons as pixels in the pictures.

Can anyone please explain what kind of network architecture would accomplish just that?

It could be either the architecture (in theory) or implemented in code.

I expect to tweak it, but it would be nice not to start from scratch.
I'm surprised nobody mention fully convolutional neural networks (FCNs) for semantic segmentation.

They are inspired by the original AlexNet style convnets that ended with one or two densely connected layers and softmax classifier. But FCNs dispense with the dense layers and stay fully convolutional all the way to the end.



Here's the basic principle. Take AlexNet, or VGG or something like that. But instead of using the parameters in the classifier to compute a scalar for each category, use them to compute a whole array (i.e. image) using a  1x1xNUM_CATEGORIES convolution. The output will be NUM_CATEGORIES feature maps, each representing a coarse-grained "heat map" for that category. A map of dogness, a map of catness. It can be sharpened by including information from earlier layers with "skip connections".

EDIT: Just one further bit of good news: the authors of that paper provide implementations of their nets in Caffe's Model Zoo. Tweak away!