# ML Workbench magics (%%ml) are under google.datalab.contrib namespace. It is not enabled by default and you need to import it before use.
import google.datalab.contrib.mlworkbench.commands

"""
Explanation: ML Workbench Sample --- Image Classification
<br><br>
Introduction of ML Workbench
ML Workbench provides an easy command line interface for machine learning life cycle, which involves four stages:
* analyze: gather stats and metadata of the training data, such as numeric stats, vocabularies, etc. Analysis results are used in transforming raw data into numeric features, which can be consumed by training directly.
* transform: explicitly transform raw data into numeric features which can be used for training.
* train: training model using transformed data.
* predict/batch_predict: given a few instances of prediction data, make predictions instantly / with large number of instances of prediction data, make predictions in a batched fassion.
There are "local" and "cloud" run mode for each stage. "cloud" run mode is recommended if your data is big.
<br><br>
ML Workbench supports numeric, categorical, text, image training data. For each type, there are a set of "transforms" to choose from. The "transforms" indicate how to convert the data into numeric features. For images, it is converted to fixed size vectors representing high level features.
<br><br>
Transfer learning using Inception Package - Cloud Run Experience With Large Data
ML Workbench supports image transforms (image to vec) with transfer learning.
This notebook continues the codifies the capabilities discussed in this blog post. In a nutshell, it uses the pre-trained inception model as a starting point and then uses transfer learning to train it further on additional, customer-specific images. For explanation, simple flower images are used. Compared to training from scratch, the time and costs are drastically reduced.
This notebook does preprocessing, training and prediction by calling CloudML API instead of running them "locally" in the Datalab container. It uses full data.
End of explanation
"""


# Create a temp GCS bucket. If the bucket already exists and you don't have permissions, rename it.
!gsutil mb gs://flower-datalab-demo-bucket-large-data

"""
Explanation: Setup
End of explanation
"""


%%ml dataset create
name: flower_data_full
format: csv
train: gs://cloud-datalab/sampledata/flower/train3000.csv
eval: gs://cloud-datalab/sampledata/flower/eval670.csv
schema:
    - name: image_url
      type: STRING
    - name: label
      type: STRING

"""
Explanation: Next cell, we will create a dataset representing our training data.
End of explanation
"""


%%ml analyze --cloud
output: gs://flower-datalab-demo-bucket-large-data/analysis
data: flower_data_full
features:
    image_url:
        transform: image_to_vec
    label:
        transform: target

# Check analysis results
!gsutil list gs://flower-datalab-demo-bucket-large-data/analysis

"""
Explanation: Analyze
Analysis step includes computing numeric stats (i.e. min/max), categorical classes, text vocabulary and frequency, etc. Run "%%ml analyze --help" for usage. The analysis results will be used for transforming raw data into numeric features that the model can deal with. For example, to convert categorical value to a one-hot vector ("Monday" becomes [1, 0, 0, 0, 0, 0, 0]). The data may be very large, so sometimes a cloud run is needed by adding --cloud flag. Cloud run will start BigQuery jobs, which may incur some costs.
In this case, analysis step only collects unique labels.
Note that we run analysis only on training data, but not evaluation data.
End of explanation
"""


# Remove previous results
!gsutil -m rm gs://flower-datalab-demo-bucket-large-data/transform

%%ml transform --cloud
analysis: gs://flower-datalab-demo-bucket-large-data/analysis
output: gs://flower-datalab-demo-bucket-large-data/transform
data: flower_data_full

"""
Explanation: Transform
With analysis results we can transform raw data into numeric features. This needs to be done for both training and eval data. The data may be very large, so sometimes a cloud pipeline is needed by adding --cloud. Cloud run is implemented by DataFlow jobs, so it may incur some costs.
In this case, transform is required. It downloads image, resizes it, and generate embeddings from each image by running a pretrained TensorFlow graph. Note that it creates two jobs --- one for training data and one for eval data.
End of explanation
"""


%%ml dataset create
name: flower_data_full_transformed
format: transformed
train: gs://flower-datalab-demo-bucket-large-data/transform/train-*
eval: gs://flower-datalab-demo-bucket-large-data/transform/eval-*

"""
Explanation: After transformation is done, create a new dataset referencing the training data.
End of explanation
"""


# Remove previous training results.
!gsutil -m rm -r gs://flower-datalab-demo-bucket-large-data/train

%%ml train --cloud
output: gs://flower-datalab-demo-bucket-large-data/train
analysis: gs://flower-datalab-demo-bucket-large-data/analysis
data: flower_data_full_transformed
model_args:
    model: dnn_classification
    hidden-layer-size1: 100
    top-n: 0
cloud_config:
    region: us-central1
    scale_tier: BASIC

"""
Explanation: Train
Training starts from transformed data. If training work is too much to do on the local VM, --cloud is recommended so training happens in cloud, in a distributed way. Run %%ml train --help for details.
Training in cloud is implemented with Cloud ML Engine. It may incur some costs.
End of explanation
"""


# List the model files
!gsutil list gs://flower-datalab-demo-bucket-large-data/train/model 

"""
Explanation: After training is complete, you should see model files like the following.
End of explanation
"""


%%ml batch_predict --cloud
model: gs://flower-datalab-demo-bucket-large-data/train/evaluation_model
output: gs://flower-datalab-demo-bucket-large-data/evaluation
cloud_config:
    region: us-central1
data:
    csv: gs://cloud-datalab/sampledata/flower/eval670.csv

# after prediction is done, check the output
!gsutil list -l -h gs://flower-datalab-demo-bucket-large-data/evaluation

# Take a look at the file.
!gsutil cat -r -500 gs://flower-datalab-demo-bucket-large-data/evaluation/prediction.results-00000-of-00006

"""
Explanation: Batch Prediction
Batch prediction performs prediction in a batched fashion. The data can be large, and is specified by files.  
Note that, we use the "evaluation_model" which sits in "evaluation_model_dir". There are two models created in training. One is a regular model under "model" dir, the other is "evaluation_model". The difference is the regular one takes prediction data without target and the evaluation model takes data with target and output the target as is. So evaluation model is good for evaluating the quality of the model because the targets and predicted values are included in output.
End of explanation
"""


import google.datalab.bigquery as bq

schema = [
  {'name': 'predicted', 'type': 'STRING'},
  {'name': 'target', 'type': 'STRING'},
  {'name': 'daisy', 'type': 'FLOAT'},  
  {'name': 'dandelion', 'type': 'FLOAT'},  
  {'name': 'roses', 'type': 'FLOAT'},
  {'name': 'sunflowers', 'type': 'FLOAT'},
  {'name': 'tulips', 'type': 'FLOAT'},
]

bq.Dataset('image_classification_results').create()
t = bq.Table('image_classification_results.flower').create(schema = schema, overwrite = True)
t.load('gs://flower-datalab-demo-bucket-large-data/evaluation/prediction.results-*', mode='overwrite', source_format='json')

"""
Explanation: Prediction results are in JSON format. We can load the results into BigQuery table and performa analysis.
End of explanation
"""


%%bq query
SELECT * FROM image_classification_results.flower WHERE predicted != target

%%ml evaluate confusion_matrix --plot
bigquery: image_classification_results.flower

%%ml evaluate accuracy
bigquery: image_classification_results.flower

"""
Explanation: Check wrong predictions.
End of explanation
"""


!gsutil -m rm -rf gs://flower-datalab-demo-bucket-large-data

"""
Explanation: Online Prediction and Build Your Own Prediction Client
Please see "Flower Classification (small dataset experience)" notebook for how to deploy the trained model and build your own prediction client.
Cleanup
End of explanation
"""


