ENDPOINT = '<YOUR_ENDPOINT>'
PROJECT_ID = !(gcloud config get-value core/project)
PROJECT_ID = PROJECT_ID[0]

"""
Explanation: CI/CD for a KFP pipeline
Learning Objectives:
1. Learn how to create a custom Cloud Build builder to pilote CAIP Pipelines
1. Learn how to write a Cloud Build config file to build and push all the artifacts for a KFP
1. Learn how to setup a Cloud Build Github trigger to rebuild the KFP
In this lab you will walk through authoring of a Cloud Build CI/CD workflow that automatically builds and deploys a KFP pipeline. You will also integrate your workflow with GitHub by setting up a trigger that starts the  workflow when a new tag is applied to the GitHub repo hosting the pipeline's code.
Configuring environment settings
Update  the ENDPOINT constat  with the settings reflecting your lab environment. 
Then endpoint to the AI Platform Pipelines instance can be found on the AI Platform Pipelines page in the Google Cloud Console.

Open the SETTINGS for your instance
Use the value of the host variable in the Connect to this Kubeflow Pipelines instance from a Python client via Kubeflow Pipelines SKD section of the SETTINGS window.
End of explanation
"""


%%writefile kfp-cli/Dockerfile

# TODO

"""
Explanation: Creating the KFP CLI builder
Exercise
In the cell below, write a docker file that
* Uses gcr.io/deeplearning-platform-release/base-cpu as base image
* Install the python package kfp with version 0.2.5
* Starts /bin/bash as entrypoint
End of explanation
"""


IMAGE_NAME='kfp-cli'
TAG='latest'
IMAGE_URI='gcr.io/{}/{}:{}'.format(PROJECT_ID, IMAGE_NAME, TAG)

"""
Explanation: Build the image and push it to your project's Container Registry.
End of explanation
"""


!gcloud builds # COMPLETE THE COMMAND

"""
Explanation: Exercise
In the cell below, use gcloud builds to build the kfp-cli Docker image and push it to the project gcr.io registry.
End of explanation
"""


%%writefile cloudbuild.yaml

steps:
# Build the trainer image
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'gcr.io/$PROJECT_ID/$_TRAINER_IMAGE_NAME:$TAG_NAME', '.']
  dir: $_PIPELINE_FOLDER/trainer_image
  
# TODO: Build the base image for lightweight components
- name: # TODO
  args: # TODO
  dir: # TODO

# Compile the pipeline
# TODO: Set the environment variables below for the $_PIPELINE_DSL script
# HINT: https://cloud.google.com/cloud-build/docs/configuring-builds/substitute-variable-values
- name: 'gcr.io/$PROJECT_ID/kfp-cli'
  args:
  - '-c'
  - |
    dsl-compile --py $_PIPELINE_DSL --output $_PIPELINE_PACKAGE
  env:
  - 'BASE_IMAGE= # TODO
  - 'TRAINER_IMAGE= # TODO
  - 'RUNTIME_VERSION= # TODO
  - 'PYTHON_VERSION= # TODO
  - 'COMPONENT_URL_SEARCH_PREFIX= # TODO
  - 'USE_KFP_SA=$_USE_KFP_SA'
  dir: $_PIPELINE_FOLDER/pipeline
  
# Upload the pipeline
# TODO: Use the kfp-cli Cloud Builder and write the command to upload the ktf pipeline 
- name: # TODO
  args:
  - '-c'
  - |
    # TODO
  dir: $_PIPELINE_FOLDER/pipeline


# Push the images to Container Registry
# TODO: List the images to be pushed to the project Docker registry
images: # TODO


"""
Explanation: Understanding the Cloud Build workflow.
Exercise
In the cell below, you'll complete the cloudbuild.yaml file describing the CI/CD workflow and prescribing how environment specific settings are abstracted using Cloud Build variables.
The CI/CD workflow automates the steps you walked through manually during lab-02-kfp-pipeline:
1. Builds the trainer image
1. Builds the base image for custom components
1. Compiles the pipeline
1. Uploads the pipeline to the KFP environment
1. Pushes the trainer and base images to your project's Container Registry
Although the KFP backend supports pipeline versioning, this feature has not been yet enable through the KFP CLI. As a temporary workaround, in the Cloud Build configuration a value of the TAG_NAME variable is appended to the name of the pipeline. 
The Cloud Build workflow configuration uses both standard and custom Cloud Build builders. The custom builder encapsulates KFP CLI.
End of explanation
"""


SUBSTITUTIONS="""
_ENDPOINT={},\
_TRAINER_IMAGE_NAME=trainer_image,\
_BASE_IMAGE_NAME=base_image,\
TAG_NAME=test,\
_PIPELINE_FOLDER=.,\
_PIPELINE_DSL=covertype_training_pipeline.py,\
_PIPELINE_PACKAGE=covertype_training_pipeline.yaml,\
_PIPELINE_NAME=covertype_continuous_training,\
_RUNTIME_VERSION=1.15,\
_PYTHON_VERSION=3.7,\
_USE_KFP_SA=True,\
_COMPONENT_URL_SEARCH_PREFIX=https://raw.githubusercontent.com/kubeflow/pipelines/0.2.5/components/gcp/
""".format(ENDPOINT).strip()

!gcloud builds submit . --config cloudbuild.yaml --substitutions {SUBSTITUTIONS}

"""
Explanation: Manually triggering CI/CD runs
You can manually trigger Cloud Build runs using the gcloud builds submit command.
End of explanation
"""


