How to reverse ReLU activation in deconvolution I recently came across Matt Zeiler's deconvolution (reversing convolution) paper . How is deconvolution able to reverse the rectified scalar output? From what I understand it sounds analogous to reversing the output of a linear / logistic regression model. How is it possible to go from a scalar to a vector?
It's not reversing the ReLU, but applying normal ReLU activation again after max pooling and max unpooling.  

It is used again 


  "to obtain valid feature reconstructions at each layer (which also should be positive)"