
# 🎸 🥁 Rockbot 🎤 🎧 
A [GPT-2](https://openai.com/blog/better-language-models/) based lyrics generator fine-tuned on the writing styles of 16000 songs by 270 artists across MANY genres (not just rock).

**Instructions:** Type in a fake song title, pick an artist, click "Generate".

Most language models are imprecise and Rockbot is no exception. You may see NSFW lyrics unexpectedly. I have made no attempts to censor. Generated lyrics may be repetitive and/or incoherent at times, but hopefully you'll encounter something interesting or memorable.

Oh, and generation is resource intense and can be slow. I set governors on song length to keep generation time somewhat reasonable. You may adjust song length and other parameters on the left or check out [Github](https://github.com/bigjoedata/rockbot) to spin up your own Rockbot.

Just have fun.

[Demo](https://share.streamlit.io/bigjoedata/rockbot/main/src/main.py) Adjust settings to increase speed

[Github](https://github.com/bigjoedata/rockbot)

[GPT-2 124M version Model page on Hugging Face](https://huggingface.co/bigjoedata/rockbot)

[DistilGPT2 version Model page on Hugging Face](https://huggingface.co/bigjoedata/rockbot-distilgpt2/) This is leaner with the tradeoff being that the lyrics are more simplistic.

🎹 🪘 🎷 🎺 🪗  🪕 🎻
## Background
With the shutdown of [Google Play Music](https://en.wikipedia.org/wiki/Google_Play_Music) I used Google's takeout function to gather the metadata from artists I've listened to over the past several years. I wanted to take advantage of this bounty to build something fun. I scraped the top 50 lyrics for artists I'd listened to at least once from [Genius](https://genius.com/), then fine tuned [GPT-2's](https://openai.com/blog/better-language-models/) 124M token model using the [AITextGen](https://github.com/minimaxir/aitextgen) framework after considerable post-processing. For more on generation, see [here.](https://huggingface.co/blog/how-to-generate)

### Full Tech Stack
[Google Play Music](https://en.wikipedia.org/wiki/Google_Play_Music)  (R.I.P.). 
[Python](https://www.python.org/). 
[Streamlit](https://www.streamlit.io/). 
[GPT-2](https://openai.com/blog/better-language-models/). 
[AITextGen](https://github.com/minimaxir/aitextgen). 
[Pandas](https://pandas.pydata.org/). 
[LyricsGenius](https://lyricsgenius.readthedocs.io/en/master/). 
[Google Colab](https://colab.research.google.com/) (GPU based Training). 
[Knime](https://www.knime.com/) (data cleaning). 


## How to Use The Model
Please refer to [AITextGen](https://github.com/minimaxir/aitextgen) for much better documentation.

### Training Parameters Used

    ai.train("lyrics.txt",
             line_by_line=False,
             from_cache=False,
             num_steps=10000,
             generate_every=2000,
             save_every=2000,
             save_gdrive=False,
             learning_rate=1e-3,
             batch_size=3,
             eos_token="<|endoftext|>",
             #fp16=True
             )
###  To Use


    Generate With Prompt (Use Title Case):
    Song Name
    BY
    Artist Name
 

{}
transformers pytorch jax gpt2 text-generation autotrain_compatible endpoints_compatible text-generation-inference region:us
TAGS
#transformers #pytorch #jax #gpt2 #text-generation #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 


#   Rockbot   
A GPT-2 based lyrics generator fine-tuned on the writing styles of 16000 songs by 270 artists across MANY genres (not just rock).

Instructions: Type in a fake song title, pick an artist, click "Generate".

Most language models are imprecise and Rockbot is no exception. You may see NSFW lyrics unexpectedly. I have made no attempts to censor. Generated lyrics may be repetitive and/or incoherent at times, but hopefully you'll encounter something interesting or memorable.

Oh, and generation is resource intense and can be slow. I set governors on song length to keep generation time somewhat reasonable. You may adjust song length and other parameters on the left or check out Github to spin up your own Rockbot.

Just have fun.

Demo Adjust settings to increase speed

Github

GPT-2 124M version Model page on Hugging Face

DistilGPT2 version Model page on Hugging Face This is leaner with the tradeoff being that the lyrics are more simplistic.

       
## Background
With the shutdown of Google Play Music I used Google's takeout function to gather the metadata from artists I've listened to over the past several years. I wanted to take advantage of this bounty to build something fun. I scraped the top 50 lyrics for artists I'd listened to at least once from Genius, then fine tuned GPT-2's 124M token model using the AITextGen framework after considerable post-processing. For more on generation, see here.

### Full Tech Stack
Google Play Music  (R.I.P.). 
Python. 
Streamlit. 
GPT-2. 
AITextGen. 
Pandas. 
LyricsGenius. 
Google Colab (GPU based Training). 
Knime (data cleaning). 


## How to Use The Model
Please refer to AITextGen for much better documentation.

### Training Parameters Used

    URL("URL",
             line_by_line=False,
             from_cache=False,
             num_steps=10000,
             generate_every=2000,
             save_every=2000,
             save_gdrive=False,
             learning_rate=1e-3,
             batch_size=3,
             eos_token="<|endoftext|>",
             #fp16=True
             )
###  To Use


    Generate With Prompt (Use Title Case):
    Song Name
    BY
    Artist Name
 

#   Rockbot   
A GPT-2 based lyrics generator fine-tuned on the writing styles of 16000 songs by 270 artists across MANY genres (not just rock).

Instructions: Type in a fake song title, pick an artist, click "Generate".

Most language models are imprecise and Rockbot is no exception. You may see NSFW lyrics unexpectedly. I have made no attempts to censor. Generated lyrics may be repetitive and/or incoherent at times, but hopefully you'll encounter something interesting or memorable.

Oh, and generation is resource intense and can be slow. I set governors on song length to keep generation time somewhat reasonable. You may adjust song length and other parameters on the left or check out Github to spin up your own Rockbot.

Just have fun.

Demo Adjust settings to increase speed

Github

GPT-2 124M version Model page on Hugging Face

DistilGPT2 version Model page on Hugging Face This is leaner with the tradeoff being that the lyrics are more simplistic. ## Background
With the shutdown of Google Play Music I used Google's takeout function to gather the metadata from artists I've listened to over the past several years. I wanted to take advantage of this bounty to build something fun. I scraped the top 50 lyrics for artists I'd listened to at least once from Genius, then fine tuned GPT-2's 124M token model using the AITextGen framework after considerable post-processing. For more on generation, see here. ### Full Tech Stack
Google Play Music  (R.I.P.). 
Python. 
Streamlit. 
GPT-2. 
AITextGen. 
Pandas. 
LyricsGenius. 
Google Colab (GPU based Training). 
Knime (data cleaning). ## How to Use The Model
Please refer to AITextGen for much better documentation. ### Training Parameters Used

    URL("URL",
             line_by_line=False,
             from_cache=False,
             num_steps=10000,
             generate_every=2000,
             save_every=2000,
             save_gdrive=False,
             learning_rate=1e-3,
             batch_size=3,
             eos_token="<|endoftext|>",
             #fp16=True
             ) ###  To Use


    Generate With Prompt (Use Title Case):
    Song Name
    BY
    Artist Name
TAGS
#transformers #pytorch #jax #gpt2 #text-generation #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
 #   Rockbot   
A GPT-2 based lyrics generator fine-tuned on the writing styles of 16000 songs by 270 artists across MANY genres (not just rock).

Instructions: Type in a fake song title, pick an artist, click "Generate".

Most language models are imprecise and Rockbot is no exception. You may see NSFW lyrics unexpectedly. I have made no attempts to censor. Generated lyrics may be repetitive and/or incoherent at times, but hopefully you'll encounter something interesting or memorable.

Oh, and generation is resource intense and can be slow. I set governors on song length to keep generation time somewhat reasonable. You may adjust song length and other parameters on the left or check out Github to spin up your own Rockbot.

Just have fun.

Demo Adjust settings to increase speed

Github

GPT-2 124M version Model page on Hugging Face

DistilGPT2 version Model page on Hugging Face This is leaner with the tradeoff being that the lyrics are more simplistic. ## Background
With the shutdown of Google Play Music I used Google's takeout function to gather the metadata from artists I've listened to over the past several years. I wanted to take advantage of this bounty to build something fun. I scraped the top 50 lyrics for artists I'd listened to at least once from Genius, then fine tuned GPT-2's 124M token model using the AITextGen framework after considerable post-processing. For more on generation, see here. ### Full Tech Stack
Google Play Music  (R.I.P.). 
Python. 
Streamlit. 
GPT-2. 
AITextGen. 
Pandas. 
LyricsGenius. 
Google Colab (GPU based Training). 
Knime (data cleaning). ## How to Use The Model
Please refer to AITextGen for much better documentation. ### Training Parameters Used

    URL("URL",
             line_by_line=False,
             from_cache=False,
             num_steps=10000,
             generate_every=2000,
             save_every=2000,
             save_gdrive=False,
             learning_rate=1e-3,
             batch_size=3,
             eos_token="<|endoftext|>",
             #fp16=True
             ) ###  To Use


    Generate With Prompt (Use Title Case):
    Song Name
    BY
    Artist Name
passage: TAGS
#transformers #pytorch #jax #gpt2 #text-generation #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
#   Rockbot   
A GPT-2 based lyrics generator fine-tuned on the writing styles of 16000 songs by 270 artists across MANY genres (not just rock).

Instructions: Type in a fake song title, pick an artist, click "Generate".

Most language models are imprecise and Rockbot is no exception. You may see NSFW lyrics unexpectedly. I have made no attempts to censor. Generated lyrics may be repetitive and/or incoherent at times, but hopefully you'll encounter something interesting or memorable.

Oh, and generation is resource intense and can be slow. I set governors on song length to keep generation time somewhat reasonable. You may adjust song length and other parameters on the left or check out Github to spin up your own Rockbot.

Just have fun.

Demo Adjust settings to increase speed

Github

GPT-2 124M version Model page on Hugging Face

DistilGPT2 version Model page on Hugging Face This is leaner with the tradeoff being that the lyrics are more simplistic.## Background
With the shutdown of Google Play Music I used Google's takeout function to gather the metadata from artists I've listened to over the past several years. I wanted to take advantage of this bounty to build something fun. I scraped the top 50 lyrics for artists I'd listened to at least once from Genius, then fine tuned GPT-2's 124M token model using the AITextGen framework after considerable post-processing. For more on generation, see here.### Full Tech Stack
Google Play Music  (R.I.P.). 
Python. 
Streamlit. 
GPT-2. 
AITextGen. 
Pandas. 
LyricsGenius. 
Google Colab (GPU based Training). 
Knime (data cleaning).## How to Use The Model
Please refer to AITextGen for much better documentation.