Issue title: Questions on reproducing the results
 Issue body: Hi, thanks for providing the source code! 

I reran the code with the command line `python oracle_relgan.py 0 <gpud_id>` but got the final `nll_oracle` with **9.4007** with length 20 (and 7.810 several steps before), while that in the paper was **~6.680**. All hyper-parameters are remained by default. And I noticed that $\beta_\max$ was set to 1 for length 20 and 2 for length 40, but no these options in the provided code. Setting `job_id` to 1 would choose the temperature of 2. 

Looking forward to your reply;)
 Comments: 
 Comment 0: Yes, the default settings in `oracle_relgan.py` are not used to get results in the table.  But you can change `temperature` and `seq_len` in `oracle_relgan.py` as follows: For length 20, you can set `temperature=['1’]` and `seq_len='20’`. For length 20, you can set `temperature=[‘2’]` and `seq_len=’40’`. Then, run the code with the command line `python oracle_relgan.py 0 <gpud_id>` and see if they work.
 Comment 1: Hi @weilinie , thank you so much for the prompt response. Have rerun it but got **8.3460** at the final step (minimum **7.1345**) with temperature 1 and length 20. ;(
 Comment 2: It looks like your score is getting closer to our reported one. Might need multiple runs to confirm as the error bars with `temperature=['1’]` tend to be larger. Note that due to the diversity-quality nature in the training curve, we use the best score, rather than the final one, for reporting the final results. 
 Comment 3: Thank you for the response. Would you use the same oracle model when comparing different models on synthetic data? I am trying to reproduce these results ;)
 Comment 4: Yes, the oracle generator is kept the same for different models.
 Comment 5: Hi @weilinie , I tried to reproduce your baseline results (Texygen implementation as in your paper), but got IndexError (list index out of range) and ValueError by running Texygen source code. Have you met such bugs when running your baseline? 
 Comment 6: Hi @cyk1337, as far as I can remember, I didn’t have this error while running the Texygen code. Sorry it’s hard to diagnose the issue without more details but I guess one possible reason might be that the file you tried to access does not exist.
 Comment 7: I ran the experiment on the oracle data, with command line `python main.py -g [gsgan | rankgan]`. Error messages when evaluating the DocEmbSim:
```bash
Traceback (most recent call last):
  File "/home/xxx/Texygen/main.py", line 78, in parse_cmd
    gan.train_oracle()
  File "/home/xxx/Texygen/models/rankgan/Rankgan.py", line 121, in train_oracle
    self.evaluate()
  File "/home/chaiyekun/GAN.tf/Texygen/models/rankgan/Rankgan.py", line 93, in evaluate
    scores = super().evaluate()
  File "/home/xxx/Texygen/models/Gan.py", line 57, in evaluate
    score = metric.get_score()
  File "/home/xxx/Texygen/utils/metrics/DocEmbSim.py", line 32, in get_score
    self.get_gen_sim()
  File "/home/xxx/Texygen/utils/metrics/DocEmbSim.py", line 153, in get_gen_sim
    self.gen_sim = self.get_wordvec(self.generator_file)
  File "/home/xxx/Texygen/utils/metrics/DocEmbSim.py", line 142, in get_wordvec
    batch_size, num_skips, skip_window, data[index])
  File "/home/xxx/Texygen/utils/metrics/DocEmbSim.py", line 72, in generate_batch
    buffer.append(data[self.data_index])
IndexError: list index out of range
```

*Runtime version and other system information (Python version, TensorFlow version, OS)*:

- python 3.7
- TensorFlow 1.14 / 1.9
- OS Ubuntu 16.04 LST

Tried:
- commented out `DocEmbSim` # but threw another error

Besides, the results of seggan/leakgan/rankgan are worse than those reported in the reference paper (7.736/7/038/8.247 respectively).

![image](https://user-images.githubusercontent.com/13767887/91378091-2ee5dd00-e852-11ea-98de-3decd78545ad.png)

I'm glad for any pointers on how to fix it.
 Comment 8: Another question: is it fair to pretrain RelGAN for 80 epochs (rather than 150 in the paper) and compare with other models in Texygen?  Baseline models report the nll-oracle per 5 epoch while RelGAN applies 3000 steps rather than epochs.

 Comment 9: 1. The `data` variable in this line `buffer.append(data[self.data_index])` comes from [here](https://github.com/geek-ai/Texygen/blob/master/utils/metrics/DocEmbSim.py#L133), so I guess it could be the filename here has not been set correctly. 2. For the worse results of these models, I guess you may need hyperparameter tuning as their default setting may not be optimized for the oracle setting.
3. The end of the pretraining stage better be set to where the curve stops getting improved. In such sense, I used 150 epochs.
 Comment 10: Thank you for the response. The filename path may not be incorrect. The IndexError occurred during the training, not at the beginning. All settings are set as default. I changed another environment to run it but still met the same error! Just gave up for reproduce the results in Texygen.

I tried to run RelGAN using architecture `rmc_att` and `rmc_vdcnn`, by setting the architecture names. Got the error for both settings:
````bash
Traceback (most recent call last):
  File "run.py", line 109, in <module>
    main()
  File "run.py", line 84, in main
    oracle_train(generator, discriminator, oracle_model, oracle_loader, gen_loader, config)
  File "/home/xxx/GAN.tf/RelGAN/oracle/oracle_gan/oracle_train.py", line 69, in oracle_train
    log_pg, temperature, global_step)
  File "/home/xxx/GAN.tf/RelGAN/oracle/oracle_gan/oracle_train.py", line 443, in get_train_ops
    temp_train_op = temp_optimizer.apply_gradients(zip(temp_grads, [temperature]))
  File "/home/xxx/anaconda3/envs/tf14/lib/python3.7/site-packages/tensorflow/python/training/optimizer.py", line 595, in apply_gradients
    ([str(v) for _, v, _ in converted_grads_and_vars],))
ValueError: No gradients provided for any variable: ["<tf.Variable 'temperature:0' shape=() dtype=float32_ref>"].

```
Any solutions to it? 
 Comment 11: Thanks for pointing out this issue: its main reason is that the generator method in `rmc_att` and `rmc_vdcnn` has not been updated accordingly. I have updated the repo to resolve it (with some other minor code optimizations).
 Comment 12: Thank you for the update. Also found that there is only one discriminator in `rmc_vanilla` setting rather than averaged multiple-representations as mentioned in the paper. Anything I missed?
 Comment 13: Yes, there is one discriminator but multiple representations, so you can see in the [code](https://github.com/weilinie/RelGAN/blob/master/models/rmc_vanilla.py#L98) we used the variable `num_rep` to control the number of representations.
 Comment 14: Yes, and Fig.2 in the paper shows an average operation for multiple discriminators, whereas the code concatenates different representations before a unified highway layer. Is it correct?
 Comment 15: Yes and note that multiple discriminators share the same weights.
 Comment 16: Yes, but it seemed no average operation in the code. Concatenation instead. Is it right?
 Comment 17: The averaging is performed when computing the loss functions, since the D output is a vector of length `batch_size*num_rep`.
 Comment 18: Closing it now. Feel free the reopen it if you have further questions.
{'source': 'GithubIssues', 'repo_name_with_owner': 'weilinie/RelGAN', 'issue_id': 680128603, 'repo_stars': 125.0}