Are neural networks invertible? I am interested in learning about the inverse of neural networks and I would like to understand about the invertibility of neural networks, as for example described in On the Invertibility of Invertible Neural Networks.
Researchers who are working on this domain, can you help me understand these two questions.

Are all neural network invertible ?
What exactly qualifies a neural network to be invertible ?
The meaning of invertible here is the standard definition of invertibility for a mathematical function $f \colon X \to Y$. Invertible simply means &quot;the function has an inverse map $f^{-1} \colon Y \to X$&quot;. Equivalently the function $f$ is bijective, which means the following two conditions hold:

$f$ is injective: for any two distinct $x_1, x_2 \in X$, $f(x_1) \ne f(x_2)$.

$f$ is surjective: for any $y \in Y$, there exists an $x \in X$ such that $f(x) = y$.


If this is unfamiliar, you should be able to find some helpful references on Google using these terms.
Most obvious neural network architectures cannot possibly be invertible. Consider for example a classifier which takes an image or some other high-dimensional input, and outputs a classification label. This network could only be invertible if there was only one possible input1 which corresponds to each label, which is not the goal of the network.

1 I mean this rather literally: if one pixel is even slightly different, this would be a different input to the network and would have to have a different output.