text generation with lstm in this sectionwe'll explore how recurrent neural networks can be used to generate sequence data we'll use text generation as an examplebut the exact same techniques can be generalized to any kind of sequence datayou could apply it to sequences of musical notes in order to generate new musicto timeseries of brushstroke data (for examplerecorded while an artist paints on an ipadto generate paintings stroke by strokeand so on sequence data generation is in no way limited to artistic content generation it has been successfully applied to speech synthesis and to dialogue generation for chatbots the smart reply feature that google released in capable of automatically generating selection of quick replies to emails or text messagesis powered by similar techniques brief history of generative recurrent networks in late few people had ever seen the initials lstmeven in the machine-learning community successful applications of sequence data generation with recurrent networks only began to appear in the mainstream in but these techniques have fairly long historystarting with the development of the lstm algorithm in this new algorithm was used early on to generate text character by character in douglas eckthen at schmidhuber' lab in switzerlandapplied lstm to music generation for the first timewith promising results eck is now researcher at google brainand in he started new research group therecalled magentafocused on applying modern deep-learning techniques to produce engaging music sometimesgood ideas take years to get started in the late and early salex graves did important pioneering work on using recurrent networks for sequence data generation in particularhis work on applying recurrent mixture density networks to generate human-like handwriting using timeseries of pen positions is seen by some as turning point this specific application of neural networks at that specific moment in time captured for me the notion of machines that dream and was significant inspiration around the time started developing keras graves left similar commented-out remark hidden in latex file uploaded to the preprint server arxiv"generating sequential data is the closest computers get to dreaming several years laterwe take lot of these developments for grantedbut at the timeit was difficult to watch graves' demonstrations and not walk away awe-inspired by the possibilities since thenrecurrent neural networks have been successfully used for music generationdialogue generationimage generationspeech synthesisand molecule design they were even used to produce movie script that was then cast with live actors sepp hochreiter and jurgen schmidhuber"long short-term memory,neural computation no ( alex graves"generating sequences with recurrent neural networks,arxiv ( )abs/ licensed to