strategy set for the generator contains arbitrary probability distributions on . However, as shown below, the optimal discriminator strategy against any is deterministic, so there is no loss of generality in restricting the discriminator's strategies to deterministic functions . In most applications, is a deep neural network function. As for the generator, while could theoretically be any computable probability distribution, in practice, it is usually implemented as a pushforward: . That is, start with a random variable , where is a probability distribution that is easy to compute (such as the uniform distribution, or the Gaussian distribution), then define a function . Then the distribution is the distribution of . Consequently, the generator's strategy is usually defined as just , leaving implicit. In this formalism, the GAN game objective is Generative reparametrization The GAN architecture has two main components. One is casting optimization into a game, of form , which is different from the usual kind of optimization, of form . The other is the decomposition of into , which can be understood as a reparametrization trick. To see its significance, one must compare GAN with previous methods for learning generative models, which were plagued with "intractable probabilistic computations that arise in maximum likelihood estimation and related strategies". At the same time, Kingma and Welling and Rezende et al. developed the same idea of reparametrization into a general stochastic backpropagation method. Among its first applications was the variational autoencoder. Move order and strategic equilibria In the original paper, as well as most subsequent papers, it is usually assumed that the generator moves first, and the discriminator moves second, thus giving the following minimax game: If both the generator's and the discriminator's strategy sets are spanned by a finite number of strategies, then by the minimax theorem,that is, the move order does not matter. However, since the strategy sets are both not finitely spanned, the minimax theorem does not apply, and the idea of an "equilibrium" becomes delicate. To wit, there are the following different concepts of equilibrium: Equilibrium when generator moves first, and discriminator moves second: Equilibrium when discriminator moves first, and generator moves second: Nash equilibrium , which is stable under simultaneous move order: For general games, these equilibria do not have to agree, or even to exist. For the original GAN game, these equilibria all exist, and are all equal. However, for more general GAN games, these do