How to report results of RL with high variance? I run Q-learning and SARSA algortihms on the same problem but the results fluctuate heavily and when I draw them, there is no smooth graph. How should I repost the results? I run algorithms for 500 runs and 40000 episodes; however, it seems that 500 runs are not sufficient, and because of timing, it is difficult to increase the number of runs.
This graph shows reward_q which is a vector of 40000 and each element number shows the summation of rewards in each episods for 500 runs, and I draw the graph based on number of episodes vs reward_q/n_runs.
Note: The title of the x-axis is not correct.
One way is to calculate a moving average and moving standard deviation.
My preferable method is smoothing (like tensorboard's smoothing function). Here is a very nice script in python. Just tune the weight parameter to get the results you want. For example: