10.16 Continuous State Spaces |The state of reinforcement learning -[Machine Learning-Andrew Ng]
 Reinforcement learning is an exciting set of technologies. In fact, when I was working on my PhD thesis, reinforcement learning was the subject of my thesis. So I was and still am excited about these ideas. Despite all the research momentum and excitement behind reinforcement learning though, I think there is a bit or maybe sometimes a lot of hype around it. So what I hope to do is share with you a practical sense of where reinforcement learning is today in terms of its utility for applications. One of the reasons for some of the hype about reinforcement learning is it turns out many of the research publications have been on simulated environments. And having worked in both simulations and on real robots myself, I can tell you that it's much easier to get a reinforcement learning algorithm to work in a simulation or in a video game than in a real robot. So a lot of developers have commented that even after they got it to work in simulation, it turned out to be surprisingly challenging to get something to work in the real world or in a real robot. And so if you apply these algorithms to a real application, this is one limitation that I hope you pay attention to, to make sure what you do does work on the real application. Second, despite all the media coverage about reinforcement learning, today there are far fewer applications of reinforcement learning than supervised and unsupervised learning. So if you are building a practical application, the odds that you will find supervised learning or unsupervised learning useful or the right tool for the job is much higher than the odds that you would end up using reinforcement learning. I have used reinforcement learning a few times myself, especially on robotic control applications, but in my day-to-day applied work, I end up using supervised and unsupervised learning much more. There is a lot of exciting research in reinforcement learning right now, and I think the potential of reinforcement learning for future applications is very large. And reinforcement learning still remains one of the major pillars of machine learning. And so having it as a framework as you develop your own machine learning algorithms, I hope will make you more effective at building working machine learning systems as well. So I hope you've enjoyed this week's materials on reinforcement learning, and specifically I hope you have fun getting the lunar lander to land for yourself. I hope it will be a satisfying experience when you implement an algorithm and see that lunar lander land safely on the moon because of code that you wrote. That brings us to the end of this specialization. Let's go on to the last video where we'll wrap up.
[{"start": 0.0, "end": 6.0, "text": " Reinforcement learning is an exciting set of technologies."}, {"start": 6.0, "end": 9.0, "text": " In fact, when I was working on my PhD thesis,"}, {"start": 9.0, "end": 12.0, "text": " reinforcement learning was the subject of my thesis."}, {"start": 12.0, "end": 16.0, "text": " So I was and still am excited about these ideas."}, {"start": 16.0, "end": 21.0, "text": " Despite all the research momentum and excitement behind reinforcement learning though,"}, {"start": 21.0, "end": 26.0, "text": " I think there is a bit or maybe sometimes a lot of hype around it."}, {"start": 26.0, "end": 32.0, "text": " So what I hope to do is share with you a practical sense of where reinforcement learning is today"}, {"start": 32.0, "end": 36.0, "text": " in terms of its utility for applications."}, {"start": 36.0, "end": 40.0, "text": " One of the reasons for some of the hype about reinforcement learning is"}, {"start": 40.0, "end": 46.0, "text": " it turns out many of the research publications have been on simulated environments."}, {"start": 46.0, "end": 50.0, "text": " And having worked in both simulations and on real robots myself,"}, {"start": 50.0, "end": 55.0, "text": " I can tell you that it's much easier to get a reinforcement learning algorithm to work"}, {"start": 55.0, "end": 60.0, "text": " in a simulation or in a video game than in a real robot."}, {"start": 60.0, "end": 67.0, "text": " So a lot of developers have commented that even after they got it to work in simulation,"}, {"start": 67.0, "end": 73.0, "text": " it turned out to be surprisingly challenging to get something to work in the real world or in a real robot."}, {"start": 73.0, "end": 77.0, "text": " And so if you apply these algorithms to a real application,"}, {"start": 77.0, "end": 81.0, "text": " this is one limitation that I hope you pay attention to,"}, {"start": 81.0, "end": 84.0, "text": " to make sure what you do does work on the real application."}, {"start": 84.0, "end": 88.0, "text": " Second, despite all the media coverage about reinforcement learning,"}, {"start": 88.0, "end": 94.0, "text": " today there are far fewer applications of reinforcement learning than supervised and unsupervised learning."}, {"start": 94.0, "end": 97.0, "text": " So if you are building a practical application,"}, {"start": 97.0, "end": 102.0, "text": " the odds that you will find supervised learning or unsupervised learning useful"}, {"start": 102.0, "end": 108.0, "text": " or the right tool for the job is much higher than the odds that you would end up using reinforcement learning."}, {"start": 108.0, "end": 111.0, "text": " I have used reinforcement learning a few times myself,"}, {"start": 111.0, "end": 114.0, "text": " especially on robotic control applications,"}, {"start": 114.0, "end": 120.0, "text": " but in my day-to-day applied work, I end up using supervised and unsupervised learning much more."}, {"start": 120.0, "end": 124.0, "text": " There is a lot of exciting research in reinforcement learning right now,"}, {"start": 124.0, "end": 130.0, "text": " and I think the potential of reinforcement learning for future applications is very large."}, {"start": 130.0, "end": 136.0, "text": " And reinforcement learning still remains one of the major pillars of machine learning."}, {"start": 136.0, "end": 142.0, "text": " And so having it as a framework as you develop your own machine learning algorithms,"}, {"start": 142.0, "end": 148.0, "text": " I hope will make you more effective at building working machine learning systems as well."}, {"start": 148.0, "end": 152.0, "text": " So I hope you've enjoyed this week's materials on reinforcement learning,"}, {"start": 152.0, "end": 158.0, "text": " and specifically I hope you have fun getting the lunar lander to land for yourself."}, {"start": 158.0, "end": 162.0, "text": " I hope it will be a satisfying experience when you implement an algorithm"}, {"start": 162.0, "end": 167.0, "text": " and see that lunar lander land safely on the moon because of code that you wrote."}, {"start": 167.0, "end": 171.0, "text": " That brings us to the end of this specialization."}, {"start": 171.0, "end": 199.0, "text": " Let's go on to the last video where we'll wrap up."}]