GANs (mainly in image synthesis)

# Survey Papers / Repos

• Are GANs Created Equal? A Large-Scale Study [1711.10337]

• Which Training Methods for GANs do actually Converge? [1801.04406]

• A Large-Scale Study on Regularization and Normalization in GANs [1807.04720]

# Others

## Metrics (my implementation: lzhbrian/metrics)

• Inception Score [1606.03498] [1801.01973]

• Assumption

• MEANINGFUL: The generated image should be clear, the output probability of a classifier network should be [0.9, 0.05, ...] (largely skewed to a class). $p(y|\mathbf{x})$ is of low entropy.

• DIVERSITY: If we have 10 classes, the generated image should be averagely distributed. So that the marginal distribution $p(y) = \frac{1}{N} \sum_{i=1}^{N} p(y|\mathbf{x}^{(i)})$ is of high entropy.

• Better models: KL Divergence of $p(y|\mathbf{x})$ and $p(y)$ should be high.

• Formulation

• $\text{IS} = \exp (\mathbb{E}_{\mathbf{x} \sim p_g} D_{KL} [p(y|\mathbf{x}) || p(y)] )$

• where

• $\mathbf{x}$ is sampled from generated data

• $p(y|\mathbf{x})​$ is the output probability of Inception v3 when input is $\mathbf{x}​$

• $p(y) = \frac{1}{N} \sum_{i=1}^{N} p(y|\mathbf{x}^{(i)})$ is the average output probability of all generated data (from InceptionV3, 1000-dim vector)

• $D_{KL} (\mathbf{p}||\mathbf{q}) = \sum_{j} p_{j} \log \frac{p_j}{q_j}$, where $j$ is the dimension of the output probability.

• Reference

• FID Score [1706.08500]

• Formulation

• $\text{FID} = ||\mu_r - \mu_g||^2 + Tr(\Sigma_{r} + \Sigma_{g} - 2(\Sigma_r \Sigma_g)^{1/2})​$

• where

• $Tr$ is trace of a matrix (wikipedia)

• $X_r \sim \mathcal{N}(\mu_r, \Sigma_r)$ and $X_g \sim \mathcal{N}(\mu_g, \Sigma_g)$ are the 2048-dim activations the Inception v3 pool3 layer

• $\mu_r$ is the mean of real photo's feature

• $\mu_g$ is the mean of generated photo's feature

• $\Sigma_r$ is the covariance matrix of real photo's feature

• $\Sigma_g$ is the covariance matrix of generated photo's feature

• Reference
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 19, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 0, "mathjax_display_tex": 0, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0, "katex": 0, "math-container": 0, "wp-katex-eq": 0, "align": 0, "equation": 0, "x-ck12": 0, "texerror": 0, "math_score": 0.8637895584106445, "perplexity": 4784.0994842220825}, "config": {"markdown_headings": true, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 10, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "plain"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "warc_path": "s3://commoncrawl/crawl-data/CC-MAIN-2020-29/segments/1593655906934.51/warc/CC-MAIN-20200710082212-20200710112212-00143.warc.gz"}