Understanding Terminology in Goodfellow's paper on GANS
Question 1


  1) From what I understand, the generator is a multi-layer perceptron input with a random vector sampled from a predefined latent space, (where wikipedia gives the example of a multivariate normal distribution). What does it mean to learn the generator's distribution $p_g$ over data x?


Generally, supervised learning means to learn a distribution $p_{model}(y\mid x)$ where $x$ are your examples and $y$ your labels. However, in unsupervised learning it is not about labels but instead you are interested in unconditional probabilities and train a model to learn $p_{model}(x)$ (e.g. see p.485/486 in "The Elements of Statistical Learning" by Hastie et al). When applying GANs this $model$ is the generator $g$, i.e. you are learning $p_{g}(x)$. Accordingly, in the original paper the authors state:


  The generator $G$ implicitly defines a probability distribution $p_g$ as the distribution of the samples $G(z)$ obtained when $z ∼ p_z$.




Question 2


  2) What does "we define a prior on input noise variables $p_z(z)$" mean? I understand that in Bayesian statistical inference, that a prior probability distribution, called a prior for short, gives a probability distribution on outcomes before some evidence is collected.


Prior means we are talking unconditional probabilities, i.e. $p(z)$ is not conditioned on anything related to the task and examples on hand. While for example, $p_d(y\mid x)$ (the distribution learned by the Discriminator) is conditioned on $x$. Often $z$ is sampled from the standard normal distribution $\mathcal{N}(0,1)$.

In "NIPS 2016 Tutorial: Generative Adversarial Networks" the authors provide a good summary of the overall concept of $G$:


  The generator is simply a differentiable function $G$. When
  $z$ is sampled from some simple prior distribution, $G(z)$ yields a sample of $x$
  drawn from $p_{model}$. Typically, a deep neural network is used to represent $G$.