"""
As any other Kaggle newbie, I baptized myself into Kagglism by completing 'Housing Prices Competition for Kaggle Learn Users'. My best submission so far got me into top 7% (I know it's nothing impressive) where I played around with XGBoost. However, I was curious how would Google Cloud Platform's AutoML would do with Housing Price data. So, I gave it a try and would like to share my experience.
The first thing you should keep in mind before using AutoML is that it is not free (apparently not cheap too) and has a lot of limitations. The data cannot be modified after you have uploaded it, therefore you should do all your data cleaning and feature engineering beforehand. Here’s the data processing I went through before uploading my training data:

"""
import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)

X_full = pd.read_csv('train.csv', index_col='Id')
X_test = pd.read_csv('test.csv', index_col='Id')

# Remove rows with missing target
X_full.dropna(axis=0, subset=['SalePrice'], inplace=True)

# I got the following idea from someone else's Notebook which made a lot of sence to me. The logic is that for example months are represented as numerical but in fact they are categorical
# Convert some numeric columns to categorical
X_full['MSSubClass'] = X_full['MSSubClass'].apply(str)
X_full['MoSold'] = X_full['MoSold'].apply(str)

X_test['MSSubClass'] = X_full['MSSubClass'].apply(str)
X_test['MoSold'] = X_test['MoSold'].apply(str)


# The next spet we merge columns that depend on each other into one
# Merge 'Exterior1st' and 'Exterior2nd' to 'Exterior' since 'Exterior2nd' depends on 'Exterior1st'
X_full['Exterior'] = X_full.apply(lambda x:x['Exterior1st'] if (pd.isnull(x['Exterior2nd'])) 
                                                            else str(x['Exterior1st'])+'-'+str(x['Exterior2nd']), axis=1)

# Merge 'Condition1', 'Condition2' to 'Condition'
X_full['Condition'] = X_full.apply(lambda x: x['Condition1'] if (pd.isnull(x['Condition2']))
                                                             else str(x['Condition1'])+'-'+str(x['Condition2']), axis=1)
X_test['Exterior'] = X_test.apply(lambda x:x['Exterior1st'] if (pd.isnull(x['Exterior2nd'])) 
                                                            else str(x['Exterior1st'])+'-'+str(x['Exterior2nd']), axis=1)

# Merge 'Condition1', 'Condition2' to 'Condition'
X_test['Condition'] = X_test.apply(lambda x: x['Condition1'] if (pd.isnull(x['Condition2']))
                                                             else str(x['Condition1'])+'-'+str(x['Condition2']), axis=1)
# Drop the merged columns since we do not need them anymore
X_full.drop(['Exterior1st', 'Exterior2nd'], axis=1, inplace=True)
X_full.drop(['Condition1', 'Condition2'], axis=1, inplace=True)

X_test.drop(['Exterior1st', 'Exterior2nd'], axis=1, inplace=True)
X_test.drop(['Condition1', 'Condition2'], axis=1, inplace=True)

# Select categorical columns with relatively low cardinality
categorical_cols = [cname for cname in X_full.columns if X_full[cname].nunique()<15 and X_full[cname].dtype == 'object']
# Select numerical columns
numerical_cols = [cname for cname in X_full.columns if X_full[cname].dtype in ['int64', 'float64']]

# We keep only selected colums
my_cols = categorical_cols + numerical_cols

X_train = X_full[my_cols].copy()
X_test_final = X_test[my_cols].copy()

# One-hot encode the data (to shorten the code, we use pandas)
X_train = pd.get_dummies(X_train)
X_test_final = pd.get_dummies(X_test_final)
X_train, X_test_final = X_train.align(X_test_final, join='left', axis=1)

X_train.to_csv('processed_train.csv', index=False)
X_test_final.to_csv('processed_test.csv', index=False)

"""
Standard procedure to get your data One-hot encoded read.

"""
"""
- In Google Cloud Platform you have to create a project and enable AutoML API for it. 
- From navigation menu you can find Tables. 
- On Table's Dataset you should create a new dataset. Remember the region of your dataset becsuse when uploading your data your storage bucket should be located in the same region. 
- Once you create dataset you can import data into it
- You have three options for importing your data: importing form BigQuery, selecting CSV from Cloud Storage or uploading file form your computer. Second and third options are essentially the same (in both cases you upload data to Cloud Storage and select it from there)
- If you are importing CSV file there are some requirements in terms of size and number of rows. But since Housing Data is not large the only requirement you should keep in mind is column names. The name of the column can contain alphanumeric characters or underscore (*_*). If you do not match those requirements the file import process will give an error
- Before you start training your data you can see the summary of your data as below 

![Summary](https://i.imgur.com/iNDVTuu.png)

- You can select target column, which in our case is 'SalePrice'
- You can have few other options such as training budget, selecting features (I went with all features from the code) and optimization objective (I selected MAE)
"""
"""
The training took around three hours and here's the result:
Tables AutoML picked regression model based on input data characteristics.
MAE which I chose as my optimization objective is 15,651.237


![Models Summary](https://i.imgur.com/P2DhHmu.png)
"""
"""
And finnaly AutoML provides a downlaodable feature importance which you could find helpful in building your own model.

![Feature importance](https://i.imgur.com/DovCleJ.png)
"""
"""
There are three options to test and use your model. The first one is, Batch Prediction where you upload you test dataset and AutoML will use your model with your test data and saves the result in your storage bucket (as you can expect this requires some computing power so it is not free). Second you can manually input data and you will get the predicted result (useful for 1 or 2 test cases). Finally, you can export your model as a TensorFlow package and run it locally on your Docker container.

I used test data from the above code and prepared submission file for Housing Data Prediction Competition. My submission scored 15162.77580 which is less than my previous submission using XGBoost.
"""
"""
To sum it up, GCP's Tables AutoML is expensive, easy to use and gives decent result. Since for starters it gives you around $200 free credit that expires in three days, I did not spend any money. I would like to hear your suggestions and input on how I could improve the above processing step to get better result.   
"""