import os

# The Vertex AI Workbench Notebook product has specific requirements
IS_WORKBENCH_NOTEBOOK = os.getenv("DL_ANACONDA_HOME")
IS_USER_MANAGED_WORKBENCH_NOTEBOOK = os.path.exists(
    "/opt/deeplearning/metadata/env_version"
)

# Vertex AI Notebook requires dependencies to be installed with '--user'
USER_FLAG = ""
if IS_WORKBENCH_NOTEBOOK:
    USER_FLAG = "--user"

! pip3 install -U tensorflow $USER_FLAG -q
! pip3 install --upgrade google-cloud-aiplatform[tensorboard] $USER_FLAG -q
! pip3 install --upgrade google-cloud-pipeline-components $USER_FLAG -q
! pip3 install --upgrade google-cloud-bigquery $USER_FLAG -q
! pip3 install --upgrade kfp $USER_FLAG -q

"""
Explanation: E2E ML on GCP: MLOps stage 3 : formalization: get started with BigQuery ML pipeline components
<table align="left">
  <td>
    <a href="https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/ml_ops/stage3/get_started_with_bqml_pipeline_components.ipynb">
      <img src="https://cloud.google.com/ml-engine/images/colab-logo-32px.png" alt="Colab logo"> Run in Colab
    </a>
  </td>  
  <td>
    <a href="https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/ml_ops/stage3/get_started_with_bqml_pipeline_components.ipynb">
      <img src="https://cloud.google.com/ml-engine/images/github-logo-32px.png" alt="GitHub logo">
      View on GitHub
    </a>
  </td>
  <td>
    <a href="https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/community/ml_ops/stage3/get_started_with_bqml_pipeline_components.ipynb">
      <img src="https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32" alt="Vertex AI logo">
      Open in Vertex AI Workbench
    </a>
  </td>
</table>
<br/><br/><br/>
Overview
This tutorial demonstrates how to use Vertex AI for E2E MLOps on Google Cloud in production. This tutorial covers stage 3 : formalization: get started with BigQuery ML pipeline components.
Dataset
The dataset used for this tutorial is the Penguins dataset from BigQuery public datasets. The version of the dataset predicts the species.
Objective
In this tutorial, you learn how to use prebuilt Google Cloud Pipeline Components for BigQuery ML.
This tutorial uses the following Google Cloud ML services:

BigQuery ML
Google Cloud Pipeline Components
Vertex AI Dataset, Model and Endpoint resources
Vertex AI Prediction

The steps performed include:

Construct a pipeline for:
Training BigQuery ML model.
Evaluating the BigQuery ML model.
Exporting the BigQuery ML model.
Importing the BigQuery ML model to a Vertex AI model.
Deploy the Vertex AI model.


Execute a Vertex AI pipeline.
Make a prediction with the deployed Vertex AI model.

Costs
This tutorial uses billable components of Google Cloud:

Vertex AI
Cloud Storage

Learn about Vertex AI
pricing and Cloud Storage
pricing, and use the Pricing
Calculator
to generate a cost estimate based on your projected usage.
Installations
Install the packages required for executing the notebook.
End of explanation
"""


import os

if not os.getenv("IS_TESTING"):
    # Automatically restart kernel after installs
    import IPython

    app = IPython.Application.instance()
    app.kernel.do_shutdown(True)

"""
Explanation: Restart the kernel
Once you've installed the additional packages, you need to restart the notebook kernel so it can find the packages.
End of explanation
"""


import os

PROJECT_ID = "[your-project-id]"  # @param {type:"string"}

if PROJECT_ID == "" or PROJECT_ID is None or PROJECT_ID == "[your-project-id]":
    # Get your GCP project id from gcloud
    shell_output = ! gcloud config list --format 'value(core.project)' 2>/dev/null
    PROJECT_ID = shell_output[0]
    print("Project ID:", PROJECT_ID)

! gcloud config set project $PROJECT_ID

"""
Explanation: Set up your Google Cloud project
The following steps are required, regardless of your notebook environment.


Select or create a Google Cloud project. When you first create an account, you get a $300 free credit towards your compute/storage costs.


Make sure that billing is enabled for your project.


Enable the Vertex AI API.


If you are running this notebook locally, you will need to install the Cloud SDK.


Enter your project ID in the cell below. Then run the cell to make sure the
Cloud SDK uses the right project for all the commands in this notebook.


Note: Jupyter runs lines prefixed with ! as shell commands, and it interpolates Python variables prefixed with $ into these commands.
Set your project ID
If you don't know your project ID, you may be able to get your project ID using gcloud.
End of explanation
"""


REGION = "[your-region]"  # @param {type:"string"}
if REGION == "[your-region]":
    REGION = "us-central1"

"""
Explanation: Region
You can also change the REGION variable, which is used for operations
throughout the rest of this notebook.  Below are regions supported for Vertex AI. We recommend that you choose the region closest to you.

Americas: us-central1
Europe: europe-west4
Asia Pacific: asia-east1

You may not use a multi-regional bucket for training with Vertex AI. Not all regions provide support for all Vertex AI services.
Learn more about Vertex AI regions.
End of explanation
"""


from datetime import datetime

TIMESTAMP = datetime.now().strftime("%Y%m%d%H%M%S")

"""
Explanation: Timestamp
If you are in a live tutorial session, you might be using a shared test account or project. To avoid name collisions between users on resources created, you create a timestamp for each instance session, and append the timestamp onto the name of resources you create in this tutorial.
End of explanation
"""


# If you are running this notebook in Colab, run this cell and follow the
# instructions to authenticate your GCP account. This provides access to your
# Cloud Storage bucket and lets you submit training jobs and prediction
# requests.

import os
import sys

# If on Vertex AI Workbench, then don't execute this code
IS_COLAB = False
if not os.path.exists("/opt/deeplearning/metadata/env_version") and not os.getenv(
    "DL_ANACONDA_HOME"
):
    if "google.colab" in sys.modules:
        IS_COLAB = True
        from google.colab import auth as google_auth

        google_auth.authenticate_user()

    # If you are running this notebook locally, replace the string below with the
    # path to your service account key and run this cell to authenticate your GCP
    # account.
    elif not os.getenv("IS_TESTING"):
        %env GOOGLE_APPLICATION_CREDENTIALS ''

"""
Explanation: Authenticate your Google Cloud account
If you are using Vertex AI Workbench Notebooks, your environment is already
authenticated. Skip this step.
If you are using Colab, run the cell below and follow the instructions when prompted to authenticate your account via oAuth.
Otherwise, follow these steps:


In the Cloud Console, go to the Create service account key
   page.


Click Create service account.


In the Service account name field, enter a name, and
   click Create.


In the Grant this service account access to project section, click the Role drop-down list. Type "Vertex AI"
into the filter box, and select
   Vertex AI Administrator. Type "Storage Object Admin" into the filter box, and select Storage Object Admin.


Click Create. A JSON file that contains your key downloads to your
local environment.


Enter the path to your service account key as the
GOOGLE_APPLICATION_CREDENTIALS variable in the cell below and run the cell.
End of explanation
"""


BUCKET_NAME = "[your-bucket-name]"  # @param {type:"string"}
BUCKET_URI = f"gs://{BUCKET_NAME}"

if BUCKET_URI == "" or BUCKET_URI is None or BUCKET_URI == "gs://[your-bucket-name]":
    BUCKET_URI = "gs://" + PROJECT_ID + "aip-" + TIMESTAMP

"""
Explanation: Create a Cloud Storage bucket
The following steps are required, regardless of your notebook environment.
When you initialize the Vertex SDK for Python, you specify a Cloud Storage staging bucket. The staging bucket is where all the data associated with your dataset and model resources are retained across sessions.
Set the name of your Cloud Storage bucket below. Bucket names must be globally unique across all Google Cloud projects, including those outside of your organization.
End of explanation
"""


! gsutil mb -l $REGION $BUCKET_URI

"""
Explanation: Only if your bucket doesn't already exist: Run the following cell to create your Cloud Storage bucket.
End of explanation
"""


! gsutil ls -al $BUCKET_URI

"""
Explanation: Finally, validate access to your Cloud Storage bucket by examining its contents:
End of explanation
"""


SERVICE_ACCOUNT = "[your-service-account]"  # @param {type:"string"}

if (
    SERVICE_ACCOUNT == ""
    or SERVICE_ACCOUNT is None
    or SERVICE_ACCOUNT == "[your-service-account]"
):
    # Get your service account from gcloud
    if not IS_COLAB:
        shell_output = !gcloud auth list 2>/dev/null
        SERVICE_ACCOUNT = shell_output[2].replace("*", "").strip()

    if IS_COLAB:
        shell_output = ! gcloud projects describe  $PROJECT_ID
        # print("shell_output=", shell_output)
        project_number = shell_output[-1].split(":")[1].strip().replace("'", "")
        SERVICE_ACCOUNT = f"{project_number}-compute@developer.gserviceaccount.com"

    print("Service Account:", SERVICE_ACCOUNT)

"""
Explanation: Service Account
If you don't know your service account, try to get your service account using gcloud command by executing the second cell below.
Note: The code for automatically finding your service account works on a user-managed Workbench AI noteboook. If you are using a fully-managed notebook, you will need to manually enter your service account.
End of explanation
"""


! gsutil iam ch serviceAccount:{SERVICE_ACCOUNT}:roles/storage.objectCreator $BUCKET_URI

! gsutil iam ch serviceAccount:{SERVICE_ACCOUNT}:roles/storage.objectViewer $BUCKET_URI

"""
Explanation: Set service account access for Vertex AI Pipelines
Run the following commands to grant your service account access to read and write pipeline artifacts in the bucket that you created in the previous step -- you only need to run these once per service account.
End of explanation
"""


import google.cloud.aiplatform as aip

import json

from kfp import dsl
from kfp.v2 import compiler

"""
Explanation: Set up variables
Next, set up some variables used throughout the tutorial.
Import libraries and define constants
End of explanation
"""


from google.cloud import bigquery

"""
Explanation: Import BigQuery
Import the BigQuery package into your Python environment.
End of explanation
"""


import tensorflow as tf

"""
Explanation: Import TensorFlow
Import the TensorFlow package into your Python environment.
End of explanation
"""


aip.init(project=PROJECT_ID, location=REGION, staging_bucket=BUCKET_URI)

"""
Explanation: Initialize Vertex AI SDK for Python
Initialize the Vertex AI SDK for Python for your project and corresponding bucket.
End of explanation
"""


bqclient = bigquery.Client()

"""
Explanation: Create BigQuery client
Create the BigQuery client.
End of explanation
"""


if os.getenv("IS_TESTING_DEPLOY_GPU"):
    DEPLOY_GPU, DEPLOY_NGPU = (
        aip.gapic.AcceleratorType.NVIDIA_TESLA_K80,
        int(os.getenv("IS_TESTING_DEPLOY_GPU")),
    )
else:
    DEPLOY_GPU, DEPLOY_NGPU = (aip.gapic.AcceleratorType.NVIDIA_TESLA_K80, 1)

"""
Explanation: Set hardware accelerators
You can set hardware accelerators for prediction.
Set the variable DEPLOY_GPU/DEPLOY_NGPU to use a container image supporting a GPU and the number of GPUs allocated to the virtual machine (VM) instance. For example, to use a GPU container image with 4 Nvidia Telsa K80 GPUs allocated to each VM, you would specify:
(aip.AcceleratorType.NVIDIA_TESLA_K80, 4)

Otherwise specify (None, None) to use a container image to run on a CPU.
Learn more here hardware accelerator support for your region
End of explanation
"""


if os.getenv("IS_TESTING_TF"):
    TF = os.getenv("IS_TESTING_TF")
else:
    TF = "2.5".replace(".", "-")

if TF[0] == "2":
    if DEPLOY_GPU:
        DEPLOY_VERSION = "tf2-gpu.{}".format(TF)
    else:
        DEPLOY_VERSION = "tf2-cpu.{}".format(TF)
else:
    if DEPLOY_GPU:
        DEPLOY_VERSION = "tf-gpu.{}".format(TF)
    else:
        DEPLOY_VERSION = "tf-cpu.{}".format(TF)

DEPLOY_IMAGE = "{}-docker.pkg.dev/vertex-ai/prediction/{}:latest".format(
    REGION.split("-")[0], DEPLOY_VERSION
)

print("Deployment:", DEPLOY_IMAGE, DEPLOY_GPU)

"""
Explanation: Set pre-built containers
Set the pre-built Docker container image for prediction.

Set the variable TF to the TensorFlow version of the container image. For example, 2-1 would be version 2.1, and 1-15 would be version 1.15. The following list shows some of the pre-built images available:

For the latest list, see Pre-built containers for prediction.
End of explanation
"""


IMPORT_FILE = "bq://bigquery-public-data.ml_datasets.penguins"
BQ_TABLE = "bigquery-public-data.ml_datasets.penguins"

"""
Explanation: Location of BigQuery training data.
Now set the variable IMPORT_FILE to the location of the data table in BigQuery.
End of explanation
"""


PIPELINE_ROOT = f"{BUCKET_URI}/bq_query"


@dsl.pipeline(name="bq-hello-world", pipeline_root=PIPELINE_ROOT)
def pipeline(
    bq_table: str,
    label: str,
    dataset: str,
    model: str,
    artifact_uri: str,
    num_trials: int,
    deploy_image: str,
    machine_type: str,
    min_replica_count: int,
    max_replica_count: int,
    display_name: str,
    accelerator_type: str = "",
    accelerator_count: int = 0,
    project: str = PROJECT_ID,
    location: str = "US",
    region: str = "us-central1",
):
    from google_cloud_pipeline_components.types import artifact_types
    from google_cloud_pipeline_components.v1.bigquery import (
        BigqueryCreateModelJobOp, BigqueryEvaluateModelJobOp,
        BigqueryExportModelJobOp, BigqueryPredictModelJobOp,
        BigqueryQueryJobOp)
    from google_cloud_pipeline_components.v1.endpoint import (EndpointCreateOp,
                                                              ModelDeployOp)
    from google_cloud_pipeline_components.v1.model import ModelUploadOp
    from kfp.v2.components import importer_node

    bq_dataset = BigqueryQueryJobOp(
        project=project, location="US", query=f"CREATE SCHEMA {dataset}"
    )

    bq_model = BigqueryCreateModelJobOp(
        project=project,
        location=location,
        query=f"CREATE OR REPLACE MODEL {dataset}.{model} OPTIONS (model_type='dnn_classifier', labels=['{label}'], num_trials={num_trials}) AS SELECT * FROM `{bq_table}` WHERE body_mass_g IS NOT NULL AND sex IS NOT NULL",
    ).after(bq_dataset)

    _ = BigqueryEvaluateModelJobOp(
        project=PROJECT_ID, location="US", model=bq_model.outputs["model"]
    ).after(bq_model)

    _ = BigqueryPredictModelJobOp(
        project=project,
        location=location,
        model=bq_model.outputs["model"],
        table_name=f"`{bq_table}`",
        # query_statement=f"SELECT * EXCEPT ({label}) FROM {bq_table} WHERE body_mass_g IS NOT NULL AND sex IS NOT NULL"
        job_configuration_query={
            "destinationTable": {
                "projectId": PROJECT_ID,
                "datasetId": "bqml_tutorial",
                "tableId": "results_1",
            }
        },
    ).after(bq_model)

    bq_export = BigqueryExportModelJobOp(
        project=project,
        location=location,
        model=bq_model.outputs["model"],
        model_destination_path=artifact_uri,
    ).after(bq_model)

    import_unmanaged_model_task = importer_node.importer(
        artifact_uri=artifact_uri,
        artifact_class=artifact_types.UnmanagedContainerModel,
        metadata={
            "containerSpec": {
                "imageUri": DEPLOY_IMAGE,
            },
        },
    ).after(bq_export)

    model_upload = ModelUploadOp(
        project=project,
        display_name=display_name,
        unmanaged_container_model=import_unmanaged_model_task.outputs["artifact"],
    ).after(import_unmanaged_model_task)

    endpoint = EndpointCreateOp(
        project=project,
        location=region,
        display_name=display_name,
    ).after(model_upload)

    _ = ModelDeployOp(
        model=model_upload.outputs["model"],
        endpoint=endpoint.outputs["endpoint"],
        dedicated_resources_min_replica_count=min_replica_count,
        dedicated_resources_max_replica_count=max_replica_count,
        dedicated_resources_machine_type=machine_type,
        dedicated_resources_accelerator_type=accelerator_type,
        dedicated_resources_accelerator_count=accelerator_count,
        traffic_split={"0": 100},
    )

"""
Explanation: Construct pipeline for BigQuery ML training, exporting and deployment
Next, construct the pipeline with the following tasks:

Create a BigQuery schema for the dataset.
Train a BigQuery ML model.
Evaluate the BigQuery ML model.
Make a batch prediction with the BigQuery ML model. Note: The same data (table) is used for both training and prediction.
Export the BigQuery ML model.
Import the exported model artifacts to a Vertex AI Model resource.
Deploy the Vertex AI Model resource to a Vertex AI Endpoint resource.
End of explanation
"""


# If DEPLOY_GPU is None, keeping gpu as no accelerator and accelerator_count as 0
accelerator_count = 0
if DEPLOY_GPU:
    gpu = DEPLOY_GPU.name
    accelerator_count = 1
else:
    gpu = "ACCELERATOR_TYPE_UNSPECIFIED"  # Unspecified accelerator type, which means no accelerator.
    accelerator_count = 0

print("gpu=", gpu)
print("accelerator_count=", accelerator_count)

MODEL_DIR = BUCKET_URI + "/bqmodel"

compiler.Compiler().compile(pipeline_func=pipeline, package_path="bqml.json")

pipeline = aip.PipelineJob(
    display_name="bqml",
    template_path="bqml.json",
    pipeline_root=PIPELINE_ROOT,
    parameter_values={
        "bq_table": BQ_TABLE,
        "label": "species",
        "dataset": "bqml_tutorial",
        "model": "penguins_model",
        "artifact_uri": MODEL_DIR,
        "num_trials": 2,
        "deploy_image": DEPLOY_IMAGE,
        "display_name": "penguins",
        "machine_type": "n1-standard-4",
        "min_replica_count": 1,
        "max_replica_count": 1,
        "accelerator_type": gpu,
        "accelerator_count": accelerator_count,
        "project": PROJECT_ID,
        "location": "US",
    },
    enable_caching=False,
)

pipeline.run()

! rm -rf bqml.json

"""
Explanation: Compile and execute the BigQuery ML train, export and deploy pipeline
Next, you compile the pipeline and then execute it. The pipeline takes the following parameters, which are passed as the dictionary parameter_values:

bq_table: The BigQuery dataset to train on.
label: The corresponding label for the BigQuery dataset.
dataset: The BigQuery dataset component name.
model: The BigQuery model component name.
artifact_uri: The Cloud Storage location to export the BigQuery model artifacts.
num_trials: If greater than one, will perform hyperparameter tuning for the specified number of trials using the Vertex AI Vizier service.
deploy_image: The container image for serving predictions.
machine_type: The VM for serving predictions.
min_replica_count/max_replica_count: The number of virtual machines for auto-scaling predictions.
accelerator_type: The type of HW accelerators -- if any.
accelerator_count: The number of HW accelerators -- if any.
display_name: Display name for Vertex AI Model and Endpoint resources.
project: The project ID
location: The location
End of explanation
"""


PROJECT_NUMBER = pipeline.gca_resource.name.split("/")[1]
print(PROJECT_NUMBER)


def print_pipeline_output(job, output_task_name):
    JOB_ID = job.name
    print(JOB_ID)
    for _ in range(len(job.gca_resource.job_detail.task_details)):
        TASK_ID = job.gca_resource.job_detail.task_details[_].task_id
        EXECUTE_OUTPUT = (
            PIPELINE_ROOT
            + "/"
            + PROJECT_NUMBER
            + "/"
            + JOB_ID
            + "/"
            + output_task_name
            + "_"
            + str(TASK_ID)
            + "/executor_output.json"
        )
        GCP_RESOURCES = (
            PIPELINE_ROOT
            + "/"
            + PROJECT_NUMBER
            + "/"
            + JOB_ID
            + "/"
            + output_task_name
            + "_"
            + str(TASK_ID)
            + "/gcp_resources"
        )
        EVAL_METRICS = (
            PIPELINE_ROOT
            + "/"
            + PROJECT_NUMBER
            + "/"
            + JOB_ID
            + "/"
            + output_task_name
            + "_"
            + str(TASK_ID)
            + "/evaluation_metrics"
        )
        if tf.io.gfile.exists(EXECUTE_OUTPUT):
            ! gsutil cat $EXECUTE_OUTPUT
            return EXECUTE_OUTPUT
        elif tf.io.gfile.exists(GCP_RESOURCES):
            ! gsutil cat $GCP_RESOURCES
            return GCP_RESOURCES
        elif tf.io.gfile.exists(EVAL_METRICS):
            ! gsutil cat $EVAL_METRICS
            return EVAL_METRICS

    return None


print("bigquery-query-job")
artifacts = print_pipeline_output(pipeline, "bigquery-query-job")
print("\n\n")
print("bigquery-create-model-job")
artifacts = print_pipeline_output(pipeline, "bigquery-create-model-job")
print("\n\n")
print("bigquery-evaluate-model-job")
artifacts = print_pipeline_output(pipeline, "bigquery-evaluate-model-job")
print("\n\n")
print("bigquery-predict-model-job")
artifacts = print_pipeline_output(pipeline, "bigquery-predict-model-job")
print("\n\n")
print("bigquery-export-model-job")
artifacts = print_pipeline_output(pipeline, "bigquery-export-model-job")
print("\n\n")
print("model-upload")
artifacts = print_pipeline_output(pipeline, "model-upload")
output = !gsutil cat $artifacts
output = json.loads(output[0])
model_id = output["artifacts"]["model"]["artifacts"][0]["metadata"]["resourceName"]
print("\n\n")
print("endpoint-create")
artifacts = print_pipeline_output(pipeline, "endpoint-create")
print("\n\n")
output = !gsutil cat $artifacts
output = json.loads(output[0])
endpoint_id = output["artifacts"]["endpoint"]["artifacts"][0]["metadata"][
    "resourceName"
]
print("\n")
print(endpoint_id)
print("\n")
print("model-deploy")
artifacts = print_pipeline_output(pipeline, "model-deploy")
print("\n\n")

"""
Explanation: View the BigQuery ML pipeline results
End of explanation
"""


pipeline.delete()

"""
Explanation: Delete a pipeline job
After a pipeline job is completed, you can delete the pipeline job with the method delete().  Prior to completion, a pipeline job can be canceled with the method cancel().
End of explanation
"""


endpoint = aip.Endpoint(endpoint_id)

"""
Explanation: Load an endpoint
The 'Endpoint' initializer will load an endpoint from an endpoint identifier.
End of explanation
"""


import json

from google.api import httpbody_pb2
from google.cloud import aiplatform_v1

DATA = {
    "signature_name": "predict",
    "instances": [
        {
            "island": "DREAM",
            "culmen_length_mm": 36.6,
            "culmen_depth_mm": 18.4,
            "flipper_length_mm": 184.0,
            "body_mass_g": 3475.0,
            "sex": "FEMALE",
        }
    ],
}

http_body = httpbody_pb2.HttpBody(
    data=json.dumps(DATA).encode("utf-8"),
    content_type="application/json",
)

req = aiplatform_v1.RawPredictRequest(http_body=http_body, endpoint=endpoint_id)

"""
Explanation: Make prediction instances
Next, you prepare a prediction request using a synthetic example. The format for making a prediction request to an export BigQuery ML model is dependent on the exported model format. In the case where model_type=DNN_CLASSIFIER, the exported model format is a TensorFlow estimator format. For this format, you use the raw_predict(), with the following request format:
http_body -&gt; {
    'signature_name' : serving_signature,
    'instances': [ {instance_1}, {instance_2}, ... ]
}

instance -&gt; { 'feature_1': value_1, 'feature_2': value_2, ... }

serving_signature -&gt; "predict"

Below is a partial list of mapping BigQuery ML model types to their corresponding exported model format:
'LINEAR_REG'<br/>
'LOGISTIC_REG' -> TensorFlow SavedFormat
'AUTOML_CLASSIFIER'<br/>
'AUTOML_REGRESSOR' -> TensorFlow SavedFormat
'BOOSTED_TREE_CLASSIFIER'<br/>
'BOOSTED_TREE_REGRESSOR' -> XGBoost format
'DNN_CLASSIFIER'<br/>
'DNN_REGRESSOR'<br/>
'DNN_LINEAR_COMBINED_CLASSIFIER'<br/>
'DNN_LINEAR_COMBINED_REGRESSOR' -> TensorFlow Estimator
End of explanation
"""


API_ENDPOINT = "{}-aiplatform.googleapis.com".format(REGION)
client_options = {"api_endpoint": API_ENDPOINT}

pred_client = aip.gapic.PredictionServiceClient(client_options=client_options)

response = pred_client.raw_predict(req)
print(response)

"""
Explanation: Make a prediction
Finally, you make the prediction request using Vertex AI Prediction service.
End of explanation
"""


try:
    job = bqclient.delete_model(f"{PROJECT_ID}.bqml_tutorial.penguins_model")
except:
    pass
job = bqclient.delete_dataset(f"{PROJECT_ID}.bqml_tutorial", delete_contents=True)

"""
Explanation: Delete the BigQuery model and dataset
Next, delete the BigQuery model and dataset.
End of explanation
"""


endpoint.undeploy_all()
endpoint.delete()

model = aip.Model(model_id)
model.delete()

"""
Explanation: Delete the Vertex AI Model and Endpoint resources
Next, undelpoy and delete the Vertex AI Model and Endpoint resources.
End of explanation
"""


# Set this to true only if you'd like to delete your bucket
delete_bucket = False

if delete_bucket or os.getenv("IS_TESTING"):
    ! gsutil rm -r $BUCKET_URI

"""
Explanation: Cleaning up
To clean up all Google Cloud resources used in this project, you can delete the Google Cloud
project you used for the tutorial.
Otherwise, you can delete the individual resources you created in this tutorial:
End of explanation
"""


