"""
# **What are differences of job contents among different jobs? **


No matter if you already have worked with data or plan to step into this field,  I think you all may be curious about what are the differences in job contents among different job positions! Knowing this can help you better blueprint your future and find the position that fits your capability & interest & strength the most. Especially for people who haven't learned about this, they might get confused with so many titles, such like analysts, scientists, engineers etc, therefore a deep insight of how these titles differ will be quite helpful (instead of looking at the salary ^^). We might have general ideas of what people with these jobs do, but let's dive into the Kaggle's survey data to see if our intuitions are the same as the fact (I assume all respondents are honest and serious about this survey) !!

## Help prepare for your next move!

My idea is to use the survey data, and give the insights of what it looks like in different jobs. For those who haven't but prepare to step into this field, they could learn what they could prepare and which skills are most needed so as to expand their skill set. For those who have worked in this field, but prepare to either stay or switch to other positions, they could also have a self-check and see if they really like the job content of that job title in bigger picture, and plan their career path!

## The questions I would like to explore:
1. What percentage of time do people spend on coding? (Reason: especially for whose who are amateurs and lack of coding background, they can better evaluate themselves and improve in the correct direction towards different occupations in data.)
2. Distribution of how long people have used coding skill to analyze data for different jobs? (Reason: Well, after knowing the coding, we can take a closer look to see the distribution of how many years people have in analyzing data with code in different occupations )
3. Do people have machine learning experience? (Reason: Once we have learned about general coding requirements in different occupation, we defnitely want to know if machine learning is a must!)
4. How many years of machine learning experience do people have in different jobs? (Reason: to get a more comprehensive picture of the machine learning experience distribution.)
5. What types of data do people work with in different jobs? (Reason: SImple and direct, we wanna see what type of data people usually work with.)
6. Find out the activities that make up an important part of role at work? (Reason: Is data work all about coding?! NO! In our real work, we have a lot of other things to do, so i wanna check out the important activities invovled at work, this should differ from job to job. It's good to improve your soft skills in the meantime!!)
7. During a typical data project at work, approximately where and how much of your time is devoted to? (Reason: We already know the main acitivities at work, but what about that in a data project?? and then i also wanna know what makes up the time in a data project!!)
8. Percentage of time people spend on exploring model insights? (Reason: You get your model! Don't you wanna explore the insights of it to be able to better interpret it to non-technical audience? How much time you need on this part?)
9. What's the primary tool people use at work? (Reason: We have talked about all the experience, the work, the activities, the time allocations, and it makes you excited! And then you wanna learn and prepare for getting that job, and it comes to a simple question: what tools you need to master to be an eligible candidate?!)
10. Through which category people have their training on machine learning? (Reason: We all need to carry on learning and growing, and how and where people learn after they start their career?)
11. How to evaluate if your current education background is enough for the occupation? (Reason: If you would like to have a long-term plan to see if you need further education for career goal, check it out!)
12. Are you young enough? Interested in knowing the age distribution among different jobs? (Reason: People may say they are still young, so they are not worried about things, or some group of people complain they are too old to become something, wanna see how old your peers are?)
"""
import os
print(os.listdir("../input"))

import numpy as np # linear algebra
import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)
import matplotlib.pyplot as plt
import seaborn as sns
from wordcloud import WordCloud, STOPWORDS, ImageColorGenerator

# Any results you write to the current directory are saved as output.
"""
## Data Loading and scope definition

Due to the fact that in this analysis, I would like to focus on difference of job contents amaong different job titles, therefore, I will only consider the respondents who have a clear job title. For those who don't selected "Others" or "Not employed" or "Students", it's quite hard and unfair to be included in this analysis.
"""
mcr = pd.read_csv("../input/multipleChoiceResponses.csv", low_memory=False).iloc[1:, :]
CONSIDERED_JOB_TITLE = ['Business Analyst', 'Chief Officer', 'Consultant',
       'DBA/Database Engineer', 'Data Analyst', 'Data Engineer',
       'Data Journalist', 'Data Scientist', 'Developer Advocate',
       'Manager', 'Marketing Analyst','Principal Investigator', 'Product/Project Manager',
       'Research Assistant', 'Research Scientist', 'Salesperson',
       'Software Engineer', 'Statistician']
mcr_jobs = mcr.loc[mcr["Q6"].isin(CONSIDERED_JOB_TITLE)]
"""
## Gerneral look at distribution of job titles among all qualified respondents

As we expect, due to the nature of Kaggle, we can imagine that data scientist are the most.
"""
plt.figure(figsize=(18, 10))
mcr_jobs["Q6"].value_counts().plot(kind='bar')
_ = plt.style.use('ggplot')
_ = plt.title("Number of respondents in each job title")
_ = plt.xlabel("Job title")
_ = plt.ylabel("Counts")
"""
## How much time do they code?!

Since I am a data scientist, and I know in my daily work, except for writing codes, I also need to have meetings with different stakeholders. I think it's the same for other data scientist. But if you are not a meeting guy, and you want to work in data, you definitely want to know which occupation requires less meeting time. Let's take a look at how much time people in different occupations spend in coding! 

### Results are not very surprising:
1. As the result presents, for occupations that are quite technical, like data scientist, data engineer, software engineers, most of them spend at least half of their time in coding! 
2. For managers, chiefs and salesperson, they spend least of time in coding, which is quite common sense. Their focus is not coding but business, strategy etc.
3. And analysts are looking like positions in between, on the one hand, they will analyze the data and find insights, and on the other hand, they work closely with business and engineers. 
"""
CODING_TIME = ['0% of my time', '1% to 25% of my time',
       '25% to 49% of my time','50% to 74% of my time', 
       '75% to 99% of my time', '100% of my time']

mcr_jobs[["Q6", "Q23"]]\
        .dropna().groupby(["Q6", "Q23"]).size()\
        .groupby("Q6").apply(lambda x: 100 * x / float(x.sum()))\
        .unstack(1)[CODING_TIME]\
        .plot(kind="bar", stacked=True, figsize=(18, 10), 
              fontsize=15, colormap=plt.get_cmap("tab20")
             )

plt.style.use("ggplot")
plt.title("Percentage of time on coding for different jobs", fontsize=20)
plt.ylabel("Percentage of Time on Coding", fontsize=15)
plt.xlabel("Job Titles", fontsize=15)
plt.legend(loc="center left", bbox_to_anchor=(1.0, 0.5), fontsize=14)
plt.show()
"""
## How long have people been writing code to analyze data?

After knowing much time they need to spend on coding, i would like to know the experience they have to analyze data with code. And I would like to see if some positions require loads of coding experience.

### Wanna get to principal/chief/higher position? it will take time! (From the first question, they seem have no time for coding now, but probably most of them have been through it):
1. As we can see, for some positions, especially high level positions, like chief/principal, althgough right now, it looks like they are not coding, based on the figure below, most of them have lots of years of coding experience. It's quite easy to understand, a good leader should be very familiar to the work, and thus be able to lead and plan the project.
2. Don't be afraid! Most of people have similar experience as you! We could see that most of people in these technical positions have less than 5 years of using code to analyze data.
"""
CODING_YEAR = ['I have never written code and I do not want to learn',
        'I have never written code but I want to learn', '< 1 year', '1-2 years', 
        '3-5 years', '5-10 years', '10-20 years', '20-30 years', 
        '30-40 years', '40+ years']
mcr_code_year = mcr_jobs[["Q6", "Q24"]]\
        .groupby(["Q6", "Q24"])\
        .size()\
        .to_frame("counts")\
        .unstack(1)\
        .fillna(0)
mcr_code_year = mcr_code_year["counts"][CODING_YEAR]
mcr_code_year = mcr_code_year.apply(lambda x: ((x / mcr_code_year.sum(axis=1)) * 100).round(0))
plt.figure(figsize=(20, 10))
sns.heatmap(mcr_code_year, annot=True)
_ = plt.title("Percentage of how many years of experience people have to use coding skill to analyze data for different jobs")
_ = plt.xlabel("Years of experience")
_ = plt.ylabel("Job titles")
"""
## Is machine learning experience important to your work?

We all talk about machine learning every single day, but is having machine learning a must?!

### Wanna be data scientist, you'd better know it:
1. For occupations like data scientist, statistician and research scientist, there is no doubt that they have experience since they constantly use them and develop new models and algorithms.
2. For positions like chief or principal, similarly to coding experience, they also have learned machine learning for quite long time. It's not hard to understand, since they need to fully understand how models/algorithms work so as to better interpret them to non-technical groups and lead a project (especially a huge one).
3. If you are only into being analysts, you can spend less time on learning machine learning!

"""
ML_YEARS = ['I have never studied machine learning and I do not plan to',
        'I have never studied machine learning but plan to learn in the future',
        '< 1 year', '1-2 years', '2-3 years', '3-4 years', '4-5 years', '5-10 years', 
        '10-15 years', '20+ years']
mcr_ml_year = mcr_jobs[["Q6", "Q25"]]\
        .groupby(["Q6", "Q25"])\
        .size()\
        .to_frame("counts")\
        .unstack(1)\
        .fillna(0)
mcr_ml_year = mcr_ml_year["counts"][ML_YEARS]
# create features for ml and non-ml

no_ml = mcr_ml_year.iloc[:, :2].sum(axis=1).values
with_ml = mcr_ml_year.iloc[:, 2:].sum(axis=1).values
mcr_ml_year_bi = pd.DataFrame(index=CONSIDERED_JOB_TITLE)
mcr_ml_year_bi["No machine learning"] = no_ml
mcr_ml_year_bi["With machine learning"] = with_ml

mcr_ml_year_bi = mcr_ml_year_bi.apply(lambda x: ((x / mcr_ml_year_bi.sum(axis=1)) * 100).round(0))\
                            .stack()\
                            .reset_index()\
                            .rename(columns={"level_0": "Q6", "level_1": "Q25", 0: "percentage"})

sns.catplot(x="Q6", y='percentage', hue='Q25', height=12, data=mcr_ml_year_bi, kind='bar', palette="muted")
_ = plt.xticks(rotation=90)
_ = plt.title("Percentage of positions having machine learning experience")
mcr_ml_year_multi = mcr_ml_year.apply(lambda x: ((x / mcr_ml_year.sum(axis=1)) * 100).round(0))
plt.figure(figsize=(20, 10))
sns.heatmap(mcr_ml_year_multi, annot=True)
_ = plt.title("Percentage of people's ML experience in different job positions")
_ = plt.xlabel("Years of experience")
_ = plt.ylabel("Job titles")
"""
## What type of data do people work with?

Here we take a glimpse at the different data type that people in different occupations work with! Although this can be significantly impacted by the industry you are in, we stilll can get some insights out of it!

### We all use numeric data:
1. Apparently, most of people regardless of occupation are working with numeric data. 
2. Categorical, text, tabular and time series are also common!
3. Tecnical positions have more chance to explore more type of data.
"""
DATA_TYPE_LIST = ['Audio Data', 'Categorical Data', 'Genetic Data',
       'Geospatial Data', 'Image Data', 'Numerical Data', 'Sensor Data',
       'Tabular Data', 'Text Data', 'Time Series Data', 'Video Data',
       'Other Data']
mcr_data_type = mcr_jobs.filter(regex="^(?!.*TEXT)(Q31)").fillna(0)
ori_list = mcr_data_type.columns.tolist()

for col in mcr_data_type:
    mcr_data_type[col] = mcr_data_type[col].apply(lambda x: 1 if x!=0 else 0)
    
mcr_data_type = mcr_data_type.rename(columns=dict(zip(ori_list, DATA_TYPE_LIST)))
mcr_data_type = mcr_data_type.merge(mcr_jobs[["Q6"]], left_index=True, right_index=True)
sum_datatype = mcr_data_type.groupby("Q6")[DATA_TYPE_LIST].sum().T
fig, ax = plt.subplots(nrows=6, ncols=3, sharex=True, sharey=True, figsize=(20, 30))
n = 1
for job in sum_datatype:
    plt.subplot(6, 3, n)
    sum_datatype[job].plot(kind="bar", title=job, fontsize=8)
    n +=1
plt.subplots_adjust(hspace = 1)
plt.show()
"""
## Which activity makes up an important part of role at work?

There are many types of activities at work, and we would like to take a deeper look at which activities are important to which occupations. Once we know this, we can have a more complete picture of what job content they have.

### You can have lots of ideas, but understand your data first!!!
1. We can have a lot of ideas, but before buiding something from scratch, you have to spend time collecting, cleaning, analyzing and understanding your data!
2. Engineers live with building infrastructure & service! Scientists and analysts can create hundreds of models, but they all need to place to run (especially if your company wants to productionize it), data engineers/database engineer build the foundation for you!
3. Build prototypes or experiment machine learning models are the first-go! Before putting things in production, they need prototypes to prove the feasibility.
"""
ACTIVITIES = ["Understand data", "Build service", 
              "Build infrastructure", "Build prototype", 
              "Advanced research", "Not related", "Other"]

mcr_activity = mcr_jobs.filter(regex="^(?!.*TEXT)(Q11)").fillna(0)
activity_list = mcr_activity.columns.tolist()
mcr_activity = mcr_activity.rename(columns=dict(zip(activity_list, ACTIVITIES)))
for col in mcr_activity:
    mcr_activity[col] = mcr_activity[col].apply(lambda x: 1 if x!=0 else 0)
mcr_activity = mcr_activity.merge(mcr_jobs[["Q6"]], left_index=True, right_index=True)
transposed_activities = mcr_activity.groupby("Q6")[ACTIVITIES].sum().T
full_activity = ["Analyze and understand data to influence product or business decisions",
                "Build and/or run a machine learning service that operationally improves my product or workflows",
                "Build and/or run the data infrastructure that my business uses for storing, analyzing, and operationalizing data",
                "Build prototypes to explore applying machine learning to new areas",
                "Do research that advances the state of the art of machine learning",
                "None of these activities are an important part of my role at work",
                "Others"]
colors = ["Red", "Blue", "Purple", "Grey", "Yellow", "Green", "Pink"]
for col, act in zip(colors, full_activity):
    print( col + "  ->  " + act)
fig, ax = plt.subplots(nrows=6, ncols=3, sharex=True, sharey=True, figsize=(20, 20))
n = 1
for job in transposed_activities:
    plt.subplot(6, 3, n)
    transposed_activities[job].plot(kind="bar", title=job, fontsize=6)
    n +=1
plt.subplots_adjust(hspace = 1)
plt.show()

"""
## How the time is distributed in data project?

In a general data project, where do you spend your most time on?

### From the last question, we know collecting and cleaning are very important part in the role. But they are always not EASY to do! 
1. Collecting and cleaning data are time consuming.
2. Seeing 'putting model into production' just takes very little time (around 5% on average), this can explain the fact that most of our experiements/exploration/models will not impact the real world since the project will die before this stage. And putting the model into production really means something!
"""
WORK_DETAILS = ['Gathering data', 'Cleaning data', 
            'Visualizing data', 'Model building/model selection', 
            'Putting the model into production', 
            'Finding insights in the data and communicating with stakeholders']
mcr_time = mcr_jobs.filter(regex=("^(?!.*TEXT)(Q6|Q34)"), axis=1)\
                .dropna()
mcr_time.columns = ["Q6"] + WORK_DETAILS
mcr_time[WORK_DETAILS] = mcr_time[WORK_DETAILS].astype(float)
mcr_time.groupby("Q6").agg(np.mean)\
    .plot.barh(stacked=True, figsize=(20, 10), fontsize=15)

plt.title("Average time percentage devoted on data projects for different titles", fontsize=20)
plt.ylabel("Job Titles", fontsize=15)
plt.xlabel("Time percentage", fontsize=15)
plt.legend(loc="center left", bbox_to_anchor=(1.0, 0.5), fontsize=14)
plt.show()
"""
## How much time you have on exploring model insights?

This is usually the part coming after obtaining the result! A deep and clear insight of the model can help improve the communication between business and technique very much!

### Wanna tell a better story? Spend more time on exploring insights:
1. Chief officers & pricipals & managers need to spend more time on it to tell a better story.
2. Data scientists and statisticians need to gain more insights to help themselves evaluate, analyze and improve the result!
"""
mcr_insight = mcr_jobs[["Q6", "Q46"]]\
                        .dropna()\
                        .groupby(["Q6", "Q46"])\
                        .size()\
                        .unstack(1)\
                        .fillna(0)

mcr_insight = mcr_insight.apply(lambda x: (x / mcr_insight.sum(axis=1) * 100).round(0))

plt.figure(figsize=(20, 10))
sns.heatmap(mcr_insight, cmap="YlGnBu")

_ = plt.title("Distribution of percentage of time people have on exploring model insights.")
_ = plt.xlabel("Slots")
_ = plt.ylabel("Job titles")
"""
## Here it is, what tools are they primarily using at work?

You know all their work from a macro perspective, now it comes down to the ground, what tools do they use at work? If you wanna be in this occupation but haven't learned the tools that they all use, better start learning it! FYI, this counts are quite rough and i have to say not very accurate since it's extracted from free form responses, but the result at least tells something.

### Jupyter dominates!!
1. Jupyter are very widely used nowadays.
2. Python, excel and Rstudio are coming after! We can aware that for the occupations that need storytelling or interpretation, Rstudio are very popular! It's not hard to understand since R is a very nice tool for visualization. 
3. Excel are still popular. And they are even more popular among data scientist!! hope it is because my mistake in transforming data :)
"""
job_index = mcr_jobs.index
mcr_ffr = pd.read_csv("../input/freeFormResponses.csv", low_memory=False).filter(regex="Q12").iloc[job_index, :]
drop_cols = mcr_ffr.columns.tolist()
TOOLS = ['jupyter', 'python', 'pycharm', 'anaconda', 'matlab', 'rstudio', 'visual studio', 'excel', 'docker', 'sql'
    , 'bash', 'spark', 'scala', 'emacs', 'tensor', 'torch', 'scikit', 'xgb', 'hadoop', 'jupyterlab', 'cloudera'
, 'c++', 'tableau', 'pandas', 'keras', 'ide', 'orange', 'colab', 'aws', 'mxnet', 'databricks', 'spyder', 'java',
'h2o', 'slack', 'zeppelin', 'sas', 'spss', 'nltk', 'powerbi']

mcr_ffr["summary"] = mcr_ffr.apply(lambda x: set([i for i in x if i is not np.nan]), axis=1)
mcr_ffr["summary"] = mcr_ffr["summary"].apply(lambda x: [i.lower() for i in x])
mcr_ffr = mcr_ffr[mcr_ffr["summary"].apply(lambda x: len(x)) == 1]
for t in TOOLS:
    mcr_ffr[t] = 0
    
for idx, r in mcr_ffr.iterrows():
    for t in TOOLS:
        if t in r['summary'][0]:
            mcr_ffr.set_value(idx, t, 1)
            
drop_cols = drop_cols + ["summary"]
mcr_tool = mcr_ffr.merge(mcr_jobs[['Q6']], right_index=True, left_index=True)\
        .drop(drop_cols, axis=1)
fig, ax = plt.subplots(nrows=6, ncols=3, sharex=True, sharey=True, figsize=(20, 20))
n = 1
for job in CONSIDERED_JOB_TITLE:
    sub_job = mcr_tool[mcr_tool["Q6"] == job]
    txt = ",".join(sub_job[TOOLS].sum(axis=0).sort_values(ascending=False).index)
    plt.subplot(6, 3, n)
    wordcloud = WordCloud(colormap="Reds", width=900, height=480,
                      normalize_plurals=False).generate(txt)
    plt.imshow(wordcloud, interpolation='bilinear')
    plt.title(job)
    plt.axis("off")
    n +=1
plt.subplots_adjust(hspace = 0.2)
plt.show()
"""
### How many language tools people use primarily?

Looks like most of people have only one main tool to use, quite disappointed at the result, but probably after some more corrections and better data wrangling, should get better and more accurate result!
"""
mcr_tool["num_tools"] = mcr_tool.iloc[:, :-1].apply(lambda x: sum(x), axis=1)
mcr_num_tool = mcr_tool[["num_tools", "Q6"]]
pal = sns.cubehelix_palette(10, rot=-.25, light=1)
g = sns.FacetGrid(mcr_num_tool, row="Q6", hue="Q6", aspect=15, height=1, palette=pal)

g.map(sns.kdeplot, "num_tools", clip_on=False, shade=True, alpha=1, lw=1.5, bw=.2)
g.map(sns.kdeplot, "num_tools", clip_on=False, color="w", lw=2, bw=.2)
g.map(plt.axhline, y=0, lw=2, clip_on=False)

def label(x, color, label):
    ax = plt.gca()
    ax.text(0, .2, label, fontweight="bold", color=color,
            ha="left", va="center", transform=ax.transAxes)

g.map(label, "num_tools")
g.fig.subplots_adjust(hspace=.25)

g.set_titles("")
g.set(yticks=[])
g.despine(bottom=True, left=True)
"""
## Where do they train on machine learning?

You have known the work acitivities, the coding requirement, and the tool they use at work. But learning never stops, and we are curious where they learn and train their machine leanring skills?

### You should believe in and rely on yourselves!!!
1. Self-taught and online courses are most commonly ways people train their machine learning skill. Nowadays, we can easily get free online resources and learn everything online and by yourself, as long as you have strong learning ability, you can do a good job as well!
2. Data scientists are working while learning!! I can also feel the same that through doing a project, I learn a lot along with the process.
"""
TRAIN_CAT = ["Self-taught", "Online courses", "Work", "University", "Kaggle competitions", "Other"]
mcr_train = mcr_jobs.filter(regex="^(?!.*TEXT)(Q35|Q6)").dropna(axis=0)
mcr_train.columns = ["Q6"] + TRAIN_CAT
mcr_train[TRAIN_CAT] = mcr_train[TRAIN_CAT].astype(float)

mcr_train.groupby("Q6").mean()\
        .plot(kind="bar", stacked=True, 
              figsize=(18, 10), fontsize=15, 
              colormap=plt.get_cmap("tab20"))

plt.title("Percentage of training on ML fell on different categories", fontsize=20)
plt.ylabel("Percentage", fontsize=15)
plt.xlabel("Job Titles", fontsize=15)
plt.legend(loc="center left", bbox_to_anchor=(1.0, 0.5), fontsize=14)
plt.show()
"""
## What kind of education background do they have?
If you want to evaluate yourself and think long-term of yourself in the career development at a certain stage, look at the education background most people have in that occupation.

### Good! PhD is not a MUST-HAVE for data scientist!?
1. Most of people have Master's degree. Should be sufficient for most of jobs.
2. Would like to end up high? PhD might help!
3. Researches really need high education background!
4. Data Engineers/DBA engineers prefer practical over academical.
"""
EDU_BACKGROUND = ["Bachelor’s degree", "Master’s degree", "Doctoral degree", 
                  "Professional degree", "Some college/university study without earning a bachelor’s degree",
                  "No formal education past high school"]

mcr_job_edu = mcr_jobs[["Q6", "Q4"]]\
        .groupby(["Q6", "Q4"])\
        .size()\
        .to_frame("counts")\
        .unstack(1)\
        .fillna(0)
mcr_job_edu = mcr_job_edu['counts'][EDU_BACKGROUND]
mcr_job_edu = mcr_job_edu.apply(lambda x: ((x / mcr_job_edu.sum(axis=1)) * 100).round(0))
plt.figure(figsize=(20, 10))
sns.heatmap(mcr_job_edu, cmap="Oranges", annot=True)

_ = plt.title("Distribution of percentage of education background people have in different occupations.")
_ = plt.xlabel("Educations")
_ = plt.ylabel("Job titles")
"""
## Age distribution in different job occupations

In order better visualize this, I randomly sampled their age based on the range they select using np.random.randint. 

### Interesting facts:
1. Most of research assistants are quite young, however, research scientists' ages are more sparsed. Probably, assistants will become scientists later, so, keey working!!
2. It needs time to become principals, probably because it requires years of experience. However, Chief officers look more achievable at younger age, probably due to the fact that there are a lot of talented people in start-ups etc. 
3. 25-40 looks like the main forces!!
"""
mcr_age = mcr_jobs[["Q6", "Q2"]]
random_sampled_age = mcr_age["Q2"].apply(lambda x: np.random.randint(int(x.split("-")[0]), int(x.split("-")[1])) 
                                                  if (len(x.split("-")) == 2) else 80)
mcr_age["age"] = random_sampled_age
plt.figure(figsize=(30, 15))
sns.swarmplot(x="Q6", y="age", hue="Q2",
              palette="RdBu", data=mcr_age)
_ = plt.xticks(rotation=90)
_ = plt.title("Age distribution among different job occupations")
_ = plt.xlabel("Job titles")
_ = plt.ylabel("Age")
"""
# Future work and to be done:
1. Get more insights through digging deeper, for example, check the impact of industry/country/age/etc on the job content for different occupations
2. Suggest a path or doing what can lead you to be a good ** for different occupations
3. Trying to explore columns and responds to make up a final story to submit.
4. Structure the functions and clean the codes to make it more efficient and reusable.
"""