I'm having trouble understanding the difference between equivariant to translation and invariant to translationIn the book Deep Learning. MIT Press, 2016 (I. Goodfellow, A. Courville, and Y. Bengio), one can find on convolutional networks:[...] the particular form of parameter sharing causes the layer to have a property called equivariance to translation [...] pooling helps to make the representation become approximately invariant to small translations of the inputIs there any difference between them or are the terms interchangeably used?
Difference between equivariant to translation and invariant to translation
neural network;deep learning;image recognition
They are sometimes  used interchangeably, but they should not. Let us start from simple image features, and suppose that image $I$ has a unique maximum $m$ at location $(x_m,y_m)$.An interesting property of classifiers is the ability to classify in the same manner some distorted versions $I'$ of $I$, for instance translations by all vectors  $(u,v)$. The maximum $m'$ of $I'$ is invariant: $m'=m$. While its location will be at $(x'_m,y'_m)=(x_m-u,y_m-v)$, and is equivariant, meaning that is varies equally with the distortion. The precise formulations given in mathematics for equivariance may depend on the objects and transformations one considers, so I prefer here the notion that is most often used in practice (and I may get the blame from a theoretical stand-point).Here, translations can be equipped with the structure of a group $G$, $g$ being one specific translation operator. A function or feature $f$ is invariant under $G$ if for all images in a class, and for any $g$, $$f(g(I)) = f(I)\,.$$ It becomes equivariant if there exists another structure (often a group) $G'$ that reflects  thetransformations in $G$ in a meaningful way. In other words, such that for each $g$, you have one a unique $g' \in G'$ such that $$f(g(I)) = g'(f(I))\,.$$ In the above example of the group of translations, $g$ and $g'$ are the same (and $G'=G$): an integer translation of the image reflects as the exact same translation of the maximum location. Another common definition is:$$f(g(I)) = g(f(I))\,.$$ I however used potentially different $G$ and $G'$ because sometimes $f(I)$ and $g(I)$ are not in the same domain. This happens for instance in multivariate statistics (see eg Equivariance and invariance properties of multivariate quantile and related functions, and the role of standardisation).But here, the uniqueness of the mapping between $g$ and $g'$ allows to get back to the original transformation $g$. Often people use the term invariance because the equivariance concept is unknown, or everybody else uses invariance, and equivariance would seem pedantic. For the record, other related notions (esp. in maths and physics) are termed covariance, contravariance, differential invariance. In addition, translation-invariance, as least approximate, or in envelope, has been a quest for several signal and image processing tools. Notably, multi-rate (filter-banks) and multi-scale (wavelets or pyramids) transformations has been design in the past 25 years, for instance under the hood of shift-invariant, cycle-spinning, stationary, complex, dual-tree wavelet transforms (for a review on 2D wavelets, A panorama on multiscale geometric representations). The wavelets can absorb a few discrete scale variations. All theses (approximate) invariances  often come with the price of redundancy in the number of transformed coefficients.