import os

# The Vertex AI Workbench Notebook product has specific requirements
IS_WORKBENCH_NOTEBOOK = os.getenv("DL_ANACONDA_HOME")
IS_USER_MANAGED_WORKBENCH_NOTEBOOK = os.path.exists(
    "/opt/deeplearning/metadata/env_version"
)

# Vertex AI Notebook requires dependencies to be installed with '--user'
USER_FLAG = ""
if IS_WORKBENCH_NOTEBOOK:
    USER_FLAG = "--user"

# Install the packages

! pip3 install --upgrade google-cloud-aiplatform $USER_FLAG -q
! pip3 install --upgrade google-cloud-storage $USER_FLAG -q

"""
Explanation: E2E ML on GCP: MLOps stage 4 : evaluation: get started with AutoML Training and ML Metadata
<table align="left">
  <td>
    <a href="https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/ml_ops/stage4/get_started_with_vertex_ml_metadata_and_automl.ipynb">
      <img src="https://cloud.google.com/ml-engine/images/github-logo-32px.png" alt="GitHub logo">
      View on GitHub
    </a>
  </td>
  <td>
        <a href="https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/ml_ops/stage4/get_started_with_vertex_ml_metadata_and_automl.ipynb">
        <img src="https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt="Colab logo"> Run in Colab
        </a>
  </td>
  <td>
    <a href="https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/community/ml_ops/stage4/get_started_with_vertex_ml_metadata_and_automl.ipynb">
      <img src="https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32" alt="Vertex AI logo">
      Open in Vertex AI Workbench
    </a>
  </td>
</table>
<br/><br/><br/>
Overview
This tutorial demonstrates how to use Vertex AI for E2E MLOps on Google Cloud in production. This tutorial covers stage 4 : evaluation: get started with Vertex ML Metadata and AutoML.
Datasets
The  dataset used for this tutorial is the Flowers dataset from TensorFlow Datasets. The version of the dataset you will use in this tutorial is stored in a public Cloud Storage bucket. The trained model predicts the type of flower in a given image from a class of five flowers: daisy, dandelion, rose, sunflower, or tulip.
Objective
In this tutorial, you learn how to use AutoML for training and assemble the corresponding artifact linkage for Vertex ML Metadata.
This tutorial uses the following Google Cloud ML services:

AutoML Training
Vertex ML Metadata
Vertex AI Model resource
Vertex AI Endpoint resource

The steps performed include:

Create a Dataset resource.
Create a corresponding google.VertexDataset artifact.
Train a model using AutoML.
Create a corresponding google.VertexModel artifact.
Create an Endpoint resource.
Create a corresponding google.Endpoint artifact.
Deploy the train model to the Endpoint.
Create an execution and context for the AutoML training job and deployment.
Add the corresponding artifacts and context to the execution.
Add artifact links (event) to the execution.
Display the execution graph.

Costs
This tutorial uses billable components of Google Cloud:

Vertex AI
Cloud Storage

Learn about Vertex AI pricing and Cloud Storage pricing and use the Pricing Calculator to generate a cost estimate based on your projected usage.
Installations
Install the packages required for executing this notebook.
End of explanation
"""


import os

if not os.getenv("IS_TESTING"):
    # Automatically restart kernel after installs
    import IPython

    app = IPython.Application.instance()
    app.kernel.do_shutdown(True)

"""
Explanation: Restart the kernel
Once you've installed the additional packages, you need to restart the notebook kernel so it can find the packages.
End of explanation
"""


PROJECT_ID = "[your-project-id]"  # @param {type:"string"}

if PROJECT_ID == "" or PROJECT_ID is None or PROJECT_ID == "[your-project-id]":
    # Get your GCP project id from gcloud
    shell_output = ! gcloud config list --format 'value(core.project)' 2>/dev/null
    PROJECT_ID = shell_output[0]
    print("Project ID:", PROJECT_ID)

! gcloud config set project $PROJECT_ID

"""
Explanation: Set up your Google Cloud project
The following steps are required, regardless of your notebook environment.


Select or create a Google Cloud project. When you first create an account, you get a $300 free credit towards your compute/storage costs.


Make sure that billing is enabled for your project.


Enable the Vertex AI, Compute Engine and Cloud Storage APIs.


If you are running this notebook locally, you need to install the Cloud SDK.


Enter your project ID in the cell below. Then run the cell to make sure the
Cloud SDK uses the right project for all the commands in this notebook.


Note: Jupyter runs lines prefixed with ! as shell commands, and it interpolates Python variables prefixed with $ into these commands.
Set your project ID
If you don't know your project ID, you may be able to get your project ID using gcloud.
End of explanation
"""


REGION = "[your-region]"  # @param {type: "string"}

if REGION == "[your-region]":
    REGION = "us-central1"

"""
Explanation: Region
You can also change the REGION variable, which is used for operations
throughout the rest of this notebook.  Below are regions supported for Vertex AI. We recommend that you choose the region closest to you.

Americas: us-central1
Europe: europe-west4
Asia Pacific: asia-east1

You may not use a multi-regional bucket for training with Vertex AI. Not all regions provide support for all Vertex AI services.
Learn more about Vertex AI regions.
End of explanation
"""


from datetime import datetime

TIMESTAMP = datetime.now().strftime("%Y%m%d%H%M%S")

"""
Explanation: Timestamp
If you are in a live tutorial session, you might be using a shared test account or project. To avoid name collisions between users on resources created, you create a timestamp for each instance session, and append the timestamp onto the name of resources you create in this tutorial.
End of explanation
"""


# If you are running this notebook in Colab, run this cell and follow the
# instructions to authenticate your GCP account. This provides access to your
# Cloud Storage bucket and lets you submit training jobs and prediction
# requests.

import os
import sys

# If on Vertex AI Workbench, then don't execute this code
IS_COLAB = "google.colab" in sys.modules
if not os.path.exists("/opt/deeplearning/metadata/env_version") and not os.getenv(
    "DL_ANACONDA_HOME"
):
    if "google.colab" in sys.modules:
        from google.colab import auth as google_auth

        google_auth.authenticate_user()

    # If you are running this notebook locally, replace the string below with the
    # path to your service account key and run this cell to authenticate your GCP
    # account.
    elif not os.getenv("IS_TESTING"):
        %env GOOGLE_APPLICATION_CREDENTIALS ''

"""
Explanation: Authenticate your Google Cloud account
If you are using Vertex AI Workbench Notebooks, your environment is already authenticated. Skip this step.
If you are using Colab, run the cell below and follow the instructions when prompted to authenticate your account via oAuth.
Otherwise, follow these steps:
In the Cloud Console, go to the Create service account key page.


Click Create service account.


In the Service account name field, enter a name, and click Create.


In the Grant this service account access to project section, click the Role drop-down list. Type "Vertex AI" into the filter box, and select Vertex AI Administrator. Type "Storage Object Admin" into the filter box, and select Storage Object Admin.


Click Create. A JSON file that contains your key downloads to your local environment.


Enter the path to your service account key as the GOOGLE_APPLICATION_CREDENTIALS variable in the cell below and run the cell.
End of explanation
"""


BUCKET_NAME = "[your-bucket-name]"  # @param {type:"string"}
BUCKET_URI = f"gs://{BUCKET_NAME}"

if BUCKET_URI == "" or BUCKET_URI is None or BUCKET_URI == "gs://[your-bucket-name]":
    BUCKET_NAME = PROJECT_ID + "aip-" + TIMESTAMP
    BUCKET_URI = "gs://" + BUCKET_NAME

"""
Explanation: Create a Cloud Storage bucket
The following steps are required, regardless of your notebook environment.
When you initialize the Vertex SDK for Python, you specify a Cloud Storage staging bucket. The staging bucket is where all the data associated with your dataset and model resources are retained across sessions.
Set the name of your Cloud Storage bucket below. Bucket names must be globally unique across all Google Cloud projects, including those outside of your organization.
End of explanation
"""


! gsutil mb -l $REGION $BUCKET_URI

"""
Explanation: Only if your bucket doesn't already exist: Run the following cell to create your Cloud Storage bucket.
End of explanation
"""


! gsutil ls -al $BUCKET_URI

"""
Explanation: Finally, validate access to your Cloud Storage bucket by examining its contents:
End of explanation
"""


import google.cloud.aiplatform as aiplatform
import google.cloud.aiplatform_v1beta1 as aip_beta

"""
Explanation: Set up variables
Next, set up some variables used throughout the tutorial.
Import libraries and define constants
End of explanation
"""


aiplatform.init(project=PROJECT_ID, staging_bucket=BUCKET_URI)

"""
Explanation: Initialize Vertex AI SDK for Python
Initialize the Vertex AI SDK for Python for your project and corresponding bucket.
End of explanation
"""


# API service endpoint
API_ENDPOINT = "{}-aiplatform.googleapis.com".format(REGION)

# Vertex location root path for your dataset, model and endpoint resources
PARENT = "projects/" + PROJECT_ID + "/locations/" + REGION

"""
Explanation: Vertex AI constants
Setup up the following constants for Vertex AI:

API_ENDPOINT: The Vertex AI API service endpoint for ML Metadata services.
End of explanation
"""


# client options same for all services
client_options = {"api_endpoint": API_ENDPOINT}


def create_metadata_client():
    client = aip_beta.MetadataServiceClient(client_options=client_options)
    return client


clients = {}
clients["metadata"] = create_metadata_client()

for client in clients.items():
    print(client)

"""
Explanation: Set up clients
The Vertex  works as a client/server model. On your side (the Python script) you will create a client that sends requests and receives responses from the Vertex AI server.
You will use different clients in this tutorial for different steps in the workflow. So set them all up upfront.

Metadata Service for creating recording, searching and analyzing artifacts and metadata.
End of explanation
"""


IMPORT_FILE = (
    "gs://cloud-samples-data/vision/automl_classification/flowers/all_data_v2.csv"
)

"""
Explanation: AutoML training job
AutoML can be used to automatically train a wide variety of image model types. AutoML automates the following:

Dataset preprocessing
Feature Engineering
Data feeding
Model Architecture selection
Hyperparameter tuning
Training the model

Learn more about Vertex AI for AutoML users
AutoML image models
AutoML can train the following types of image models:

classification
objection detection
segmentation

A model can be trained for either deployment to the cloud or exported to the edge.
Learn more about AutoML Model Types
Location of Cloud Storage training data.
Now set the variable IMPORT_FILE to the location of the CSV index file in Cloud Storage.
End of explanation
"""


FILE = IMPORT_FILE

count = ! gsutil cat $FILE | wc -l
print("Number of Examples", int(count[0]))

print("First 10 rows")
! gsutil cat $FILE | head

"""
Explanation: Quick peek at your data
This tutorial uses a version of the Happy Moments dataset that is stored in a public Cloud Storage bucket, using a CSV index file.
Start by doing a quick peek at the data. You count the number of examples by counting the number of rows in the CSV index file  (wc -l) and then peek at the first few rows.
End of explanation
"""


dataset = aiplatform.ImageDataset.create(
    display_name="flowers_" + TIMESTAMP,
    gcs_source=[IMPORT_FILE],
    import_schema_uri=aiplatform.schema.dataset.ioformat.image.single_label_classification,
)

print(dataset.resource_name)

"""
Explanation: Create the Dataset
Next, create the Dataset resource using the create method for the ImageDataset class, which takes the following parameters:

display_name: The human readable name for the Dataset resource.
gcs_source: A list of one or more dataset index files to import the data items into the Dataset resource.
import_schema_uri: The data labeling schema for the data items:
single_label: Binary and multi-class classification
multi_label: Multi-label multi-class classification
bounding_box: Object detection
image_segmentation: Segmentation

Learn more about ImageDataset.
End of explanation
"""


from google.cloud.aiplatform_v1beta1.types import Artifact

artifact_item = Artifact(
    display_name="flowers_" + TIMESTAMP,
    uri="https://us-central1-aiplatform.googleapis.com/v1/" + dataset.resource_name,
    name=dataset.resource_name,
    schema_title="google.VertexDataset",
    metadata={"data_type": "image", "annotation_type": "image classification"},
    labels={"my_label": "value"},
)

artifact_dataset = clients["metadata"].create_artifact(
    parent="projects/759209241365/locations/us-central1/metadataStores/default",
    artifact=artifact_item,
    artifact_id=dataset.resource_name.split("/")[-1],
)

print(artifact_dataset)

"""
Explanation: Create an Artifact resource for the Vertex AI Dataset
Next, you create an Artifact resource using the create_artifact() method, with the following parameters:

parent: The fully qualified resource identifier to the Metadatastore resource -- in this example, you use the default Metadatastore.
artifact: The definition of the Artifact resource
display_name: The human readable name for the Artifact resource.
uri: The uniform resource identifier of the artifact file. In this example, the Dataset resource identifier, prepended with https://{REGION}-aiplatform.googleapis.com/v1/
name: The resource name of the artifact file. In this example, the Dataset resource identifier.
labels: User defined labels to assign to the Artifact resource.
schema_title: The title of the schema that describes the metadata. In this example: google.VertexDataset.
metadata: The metadata key/value pairs to associate with the Artifact resource.


artifact_id: (optional) A user defined short ID for the Artifact resource.
End of explanation
"""


dag = aiplatform.AutoMLImageTrainingJob(
    display_name="flowers_" + TIMESTAMP,
    prediction_type="classification",
    multi_label=False,
    model_type="MOBILE_TF_LOW_LATENCY_1",
    base_model=None,
)

print(dag)

"""
Explanation: Create and run training pipeline
To train an AutoML model, you perform two steps: 1) create a training pipeline, and 2) run the pipeline.
Create training pipeline
An AutoML training pipeline is created with the AutoMLImageTrainingJob class, with the following parameters:

display_name: The human readable name for the TrainingJob resource.
prediction_type: The type task to train the model for.
classification: An image classification model.
object_detection: An image object detection model.
multi_label: If a classification task, whether single (False) or multi-labeled (True).
model_type: The type of model for deployment.
CLOUD: Deployment on Google Cloud
CLOUD_HIGH_ACCURACY_1: Optimized for accuracy over latency for deployment on Google Cloud.
CLOUD_LOW_LATENCY_: Optimized for latency over accuracy for deployment on Google Cloud.
MOBILE_TF_VERSATILE_1: Deployment on an edge device.
MOBILE_TF_HIGH_ACCURACY_1:Optimized for accuracy over latency for deployment on an edge device.
MOBILE_TF_LOW_LATENCY_1: Optimized for latency over accuracy for deployment on an edge device.
base_model: (optional) Transfer learning from existing Model resource -- supported for image classification only.

The instantiated object is the DAG (directed acyclic graph) for the training job.
End of explanation
"""


model = dag.run(
    dataset=dataset,
    model_display_name="flowers_" + TIMESTAMP,
    training_fraction_split=0.8,
    validation_fraction_split=0.1,
    test_fraction_split=0.1,
    budget_milli_node_hours=8000,
    disable_early_stopping=False,
)

"""
Explanation: Run the training pipeline
Next, you run the created DAG to start the training job by invoking the method run, with the following parameters:

dataset: The Dataset resource to train the model.
model_display_name: The human readable name for the trained model.
training_fraction_split: The percentage of the dataset to use for training.
test_fraction_split: The percentage of the dataset to use for test (holdout data).
validation_fraction_split: The percentage of the dataset to use for validation.
budget_milli_node_hours: (optional) Maximum training time specified in unit of milli node-hours (1000 = node-hour).
disable_early_stopping: If True, training maybe completed before using the entire budget if the service believes it cannot further improve on the model objective measurements.

The run method when completed returns the Model resource.
The execution of the training pipeline will take upto > 30 minutes.
End of explanation
"""


artifact_item = Artifact(
    display_name="flowers_" + TIMESTAMP,
    uri="https://us-central1-aiplatform.googleapis.com/v1/" + model.resource_name,
    name=model.resource_name,
    schema_title="google.VertexModel",
    metadata={"model_type": "image classification"},
    labels={"my_label": "value"},
)

artifact_model = clients["metadata"].create_artifact(
    parent="projects/759209241365/locations/us-central1/metadataStores/default",
    artifact=artifact_item,
    artifact_id=model.resource_name.split("/")[-1],
)

print(artifact_model)

"""
Explanation: Create an Artifact resource for the Vertex AI Model
Next, you create an Artifact resource using the create_artifact() method, with the following parameters:

parent: The fully qualified resource identifier to the Metadatastore resource -- in this example, you use the default Metadatastore.
artifact: The definition of the Artifact resource
display_name: The human readable name for the Artifact resource.
uri: The uniform resource identifier of the artifact file. In this example, the Model resource identifier, prepended with https://{REGION}-aiplatform.googleapis.com/v1/
name: The resource name of the artifact file. In this example, the Model resource identifier.
labels: User defined labels to assign to the Artifact resource.
schema_title: The title of the schema that describes the metadata. In this example: google.VertexModel.
metadata: The metadata key/value pairs to associate with the Artifact resource.


artifact_id: (optional) A user defined short ID for the Artifact resource.
End of explanation
"""


model_evaluations = model.list_model_evaluations()

for model_evaluation in model_evaluations:
    print(model_evaluation.to_dict())

"""
Explanation: Review model evaluation scores
After your model training has finished, you can review the evaluation scores for it using the list_model_evaluations() method. This method will return an iterator for each evaluation slice.
End of explanation
"""


artifact_item = Artifact(
    display_name="flowers_" + TIMESTAMP,
    uri="https://us-central1-aiplatform.googleapis.com/v1/"
    + model_evaluations[0].resource_name,
    name=model_evaluations[0].resource_name,
    schema_title="system.SlicedClassificationMetrics",
    metadata={"param": "value"},
    labels={"my_label": "value"},
)

artifact_metrics = clients["metadata"].create_artifact(
    parent="projects/759209241365/locations/us-central1/metadataStores/default",
    artifact=artifact_item,
    artifact_id=model_evaluations[0].resource_name.split("/")[-1],
)

print(artifact_metrics)

"""
Explanation: Create an Artifact resource for the Vertex AI EvaluationMetrics
Next, you create an Artifact resource using the create_artifact() method, with the following parameters:

parent: The fully qualified resource identifier to the Metadatastore resource -- in this example, you use the default Metadatastore.
artifact: The definition of the Artifact resource
display_name: The human readable name for the Artifact resource.
uri: The uniform resource identifier of the artifact file. In this example, the EvaluationMetrics resource identifier, prepended with https://{REGION}-aiplatform.googleapis.com/v1/
name: The resource name of the artifact file. In this example, the EvaluationMetrics resource identifier.
labels: User defined labels to assign to the Artifact resource.
schema_title: The title of the schema that describes the metadata. In this example: system.SlicedClassificationMetrics.
metadata: The metadata key/value pairs to associate with the Artifact resource.


artifact_id: (optional) A user defined short ID for the Artifact resource.
End of explanation
"""


endpoint = model.deploy()

"""
Explanation: Deploy the model
Next, deploy your model for online prediction. To deploy the model, you invoke the deploy method.
End of explanation
"""


artifact_item = Artifact(
    display_name="flowers_" + TIMESTAMP,
    uri="https://us-central1-aiplatform.googleapis.com/v1/" + endpoint.resource_name,
    name=endpoint.resource_name,
    schema_title="google.VertexEndpoint",
    metadata={"param": "value"},
    labels={"my_label": "value"},
)

artifact_endpoint = clients["metadata"].create_artifact(
    parent="projects/759209241365/locations/us-central1/metadataStores/default",
    artifact=artifact_item,
    artifact_id=endpoint.resource_name.split("/")[-1],
)

print(artifact_endpoint)

"""
Explanation: Create an Artifact resource for the Vertex AI Endpoint
Next, you create an Artifact resource using the create_artifact() method, with the following parameters:

parent: The fully qualified resource identifier to the Metadatastore resource -- in this example, you use the default Metadatastore.
artifact: The definition of the Artifact resource
display_name: The human readable name for the Artifact resource.
uri: The uniform resource identifier of the artifact file. In this example, the Endpoint resource identifier, prepended with https://{REGION}-aiplatform.googleapis.com/v1/
name: The resource name of the artifact file. In this example, the Endpoint resource identifier.
labels: User defined labels to assign to the Artifact resource.
schema_title: The title of the schema that describes the metadata. In this example: google.VertexEndpoint.
metadata: The metadata key/value pairs to associate with the Artifact resource.


artifact_id: (optional) A user defined short ID for the Artifact resource.
End of explanation
"""


from google.cloud.aiplatform_v1beta1.types import Execution

execution = clients["metadata"].create_execution(
    parent="projects/759209241365/locations/us-central1/metadataStores/default",
    execution=Execution(
        display_name="AutoML training and deployment",
        schema_title="system.ContainerExecution",
        metadata={"model_type": "image classification"},
    ),
    execution_id=dag.resource_name.split("/")[-1],
)

print(execution)

"""
Explanation: Create an Execution resource
A sequence of steps, with inputs and outputs, is referred to as an execution.
Next, you create an Execution resource for the AutoML training and model deployment, using the create_execution() method, with the following parameters:

parent: (optional) The fully qualified resource identifier to the Metadatastore resource. 
execution:
display_name: A human readable name for the Execution resource.
schema_title: The title of the schema that describes the metadata. In this example: system.ContainerExecution.
metadata: The metadata key/value pairs to associate with the Execution resource.


execution_id: (optional) A user defined short ID for the Execution resource.
End of explanation
"""


from google.cloud.aiplatform_v1beta1.types import Event

clients["metadata"].add_execution_events(
    execution=execution.name,
    events=[
        Event(
            artifact=artifact_dataset.name,
            type_=Event.Type.INPUT,
            labels={"my_label": "my_value"},
        ),
        Event(
            artifact=artifact_model.name,
            type_=Event.Type.OUTPUT,
            labels={"my_label": "my_value"},
        ),
        Event(
            artifact=artifact_metrics.name,
            type_=Event.Type.OUTPUT,
            labels={"my_label": "my_value"},
        ),
        Event(
            artifact=artifact_endpoint.name,
            type_=Event.Type.OUTPUT,
            labels={"my_label": "my_value"},
        ),
    ],
)

"""
Explanation: Add events to Execution resource
An Execution resource consists of a sequence of events that occurred during the execution. Each event consists of an artifact that is either an input or an output of the Execution resource.
You add execution events to an Execution resource using the add_execution_events() method, with the following parameters:

execution: The fully qualified resource identifier for the Execution resource. In this example, the execution steps for AutoML training and deployment.
events: The sequence of events constituting the execution. In this example the artifacts: VertexDataset, VertexModel, SlicedClassificationMetrics, and VertexEndpoint.
artifact_dataset: The input to the execution.
artifact_model:
artifact_metrics:
artifact_endpoint: The outputs from the execution.
End of explanation
"""


from google.cloud.aiplatform_v1beta1.types import Context

context = clients["metadata"].create_context(
    parent="projects/759209241365/locations/us-central1/metadataStores/default",
    context=Context(
        display_name="flowers_" + TIMESTAMP,
        schema_title="system.Pipeline",
        metadata={"param": "value"},
        labels=[{"my_label", "my_value"}],
    ),
    context_id=dag.resource_name.split("/")[-1],
)

print(context)

"""
Explanation: Create a Context resource
A Context resource represents the context consisting of one or more executions. You create an Context resource using the create_context() method, with the following parameters:

parent: The fully qualified resource identifier to the Metadatastore resource.
context:
display_name: A human readable name for the Execution resource.
schema_title: The title of the schema that describes the metadata. In this example: system.Pipeline.
labels: User defined labels to assign to the Context resource.
metadata: The metadata key/value pairs to associate with the Execution resource.


context_id: (optional) A user defined short ID for the Context resource.
End of explanation
"""


clients["metadata"].add_context_artifacts_and_executions(
    context=context.name,
    artifacts=[
        artifact_dataset.name,
        artifact_model.name,
        artifact_metrics.name,
        artifact_endpoint.name,
    ],
    executions=[execution.name],
)

"""
Explanation: Combine Artifacts and Executions into a Context
A Context is used to group Artifact resources and Execution resources together under a single, queryable, and typed category. Contexts can be used to represent sets of metadata.
You combine a set of Artifact and Execution resources into a Context resource using the add_context_artifacts_and_executions() method, with the following parameters:

context: The fully qualified resource identifier of the Context resource.
artifacts: A list of fully qualified resource identifiers of the Artifact resources.
executions: A list of fully qualified resource identifiers of the Execution resources.
End of explanation
"""


subgraph = clients["metadata"].query_context_lineage_subgraph(context=context.name)

print(subgraph)

"""
Explanation: Query a context
You can query the subgraph of a Context resource using the method query_context_lineage_subgraph() method, with the following parameters:

context: The fully qualified resource identifier of the Context resource.
End of explanation
"""


clients["metadata"].delete_execution(name=execution.name)

"""
Explanation: Viewing the graph in the UI
Finally, go to the UI for Vertex AI ML Metadata, and select any of your created artifacts. It will appear similar to:
<img src='automl_mlmd.png'/>
Delete an Execution resource
You can delete an Execution resource using the delete_execution() method, with the following parameters:

name: The fully qualified resource identifier for the Execution resource.
End of explanation
"""


clients["metadata"].delete_context(name=context.name)

"""
Explanation: Delete a Context resource
You can delete an Context resource using the delete_context() method, with the following parameters:

name: The fully qualified resource identifier for the Context resource.
End of explanation
"""


clients["metadata"].delete_artifact(name=artifact_dataset.name)
clients["metadata"].delete_artifact(name=artifact_model.name)
clients["metadata"].delete_artifact(name=artifact_metrics.name)
clients["metadata"].delete_artifact(name=artifact_endpoint.name)

"""
Explanation: Delete an Artifact resource
You can delete an Artifact resource using the delete_artifact() method, with the following parameters:

name: The fully qualified resource identifier for the Artifact resource.
End of explanation
"""


endpoint.undeploy_all()

"""
Explanation: Undeploy the model
When you are done doing predictions, you undeploy the model from the Endpoint resouce. This deprovisions all compute resources and ends billing for the deployed model.
End of explanation
"""


model.delete()

"""
Explanation: Delete the model
The method 'delete()' will delete the model.
End of explanation
"""


dataset.delete()

"""
Explanation: Delete the dataset
The method 'delete()' will delete the dataset.
End of explanation
"""


endpoint.delete()

"""
Explanation: Delete the endpoint
The method 'delete()' will delete the endpoint.
End of explanation
"""


# Set this to true only if you'd like to delete your bucket
delete_bucket = False

if delete_bucket or os.getenv("IS_TESTING"):
    ! gsutil rm -r $BUCKET_URI

"""
Explanation: Cleaning up
To clean up all Google Cloud resources used in this project, you can delete the Google Cloud
project you used for the tutorial.
Otherwise, you can delete the individual resources you created in this tutorial.
End of explanation
"""


