Understanding math notation in infoGAN paper I'm reading this paper about mutual information in infoGAN infoGAN_paper_link and already have the code to run it. I pretty much found code for it which is fine and dandy except for the fact that I kinda don't understand some of the code in the cost function. So, I looked at the paper to dissect it for better understanding and came across some math notation that I don't understand (pic below). The usage of the notations I'm trying to figure out are how the &quot;~&quot; in the Expectation subscripts, &quot;'&quot;, and &quot;||&quot; symbols are used.
This is what I think the notation means.

&quot;~&quot; in the expectation subscript: the variable on the left of &quot;~&quot; can be any continuous value coming from whatever is to the right of the &quot;~&quot;
&quot;'&quot; next to the &quot;c&quot; in P(c'|x): I have no clue. I thought those were symbols for derivates but that makes no sense for this equation so it's def not that.
&quot;||&quot;: I'm not sure. I only know of these symbols being used in Norms but that's obviously not the case here.


Screenshot of a formula from the paper

The actual code I was trying to figure out was this in PyTorch. It's the variational lower bound (mutual information term) in the cost (It's not the formulas in the screenshot above). However, I ran into the formulas in the screenshot first before getting to the formula calculations for the variational lower bound formula.
mutual_information_lower_bound = lambda c_true, mean, logvar: Normal(mean,logvar.exp()).log_prob(c_true).mean()

Thanks for the help!
The notation $D_\mathrm{KL}(P \| Q)$ is pretty much standard for the Kullback-Leibler divergence. There is an interesting discussion on Mathematics SE on the reasons for the fairly unusual notation used for divergences.

In general, $x \sim G(z, c)$ means &quot;$x$ is a random variable distributed according to $G(z, c)$&quot;. In the subscript the author means to take the expectation with respect to the random variable $x$, supposing that $x$ has that distribution.

If you have a variable, such as $a$, it is not unusual to call a related variable $a'$ (usually pronounced &quot;a prime&quot;). In this case $c'$ is just a random variable distributed according to $P(c\mid x)$, with the name $c'$ probably meant to remind you that the variable is in some way connected to $c$. In cases where derivatives are obviously not the right thing, this is probably what a prime symbol means.