Error while deploying on SageMaker

#34
by tararelan - opened

I'm trying to deploy jina-clip-v2 on SageMaker using HuggingFace (not JumpStart) to embed some text and images. Below is my code:

import sagemaker
import json
from sagemaker.huggingface import HuggingFaceModel

Initialize SageMaker session and role

sess = sagemaker.Session()
role = sagemaker.get_execution_role()

model_kwargs = {"device": "cpu", "trust_remote_code":"True"}
encode_kwargs = {"normalize_embeddings": True}

Define model configuration

model_id = "jinaai/jina-clip-v2" # Model ID for Jina CLIP v2 from Hugging Face
instance_type = "ml.g5.xlarge" # Choose an appropriate instance type

Define environment variables

config = {
'HF_MODEL_ID': model_id,
'HF_TASK': 'feature-extraction', # Define the task
'HF_MODEL_TRUST_REMOTE_CODE': json.dumps(True) # Allow remote code execution
}

Create HuggingFaceModel

clip_model = HuggingFaceModel(
role=role,
env=config,
transformers_version='4.37.0',
pytorch_version='2.1.0',
py_version='py310'
)

Deploy the model to an endpoint

clip_endpoint = clip_model.deploy(
initial_instance_count=1,
instance_type=instance_type,
)

Prepare data for prediction (example)

data = {
"model": "jina-clip-v2",
"dimensions": 1024,
"normalized": True,
"embedding_type": "float",
"input": [
{"text": "A beautiful sunset over the beach"},
{"text": "Un beau coucher de soleil sur la plage"},
{"text": "海滩上美丽的日落"},
{"text": "浜辺に沈む美しい夕日"},
{"image": "https://i.ibb.co/nQNGqL0/beach1.jpg"},
{"image": "https://i.ibb.co/r5w8hG8/beach2.jpg"},
{
"image": "R0lGODlhEAAQAMQAAORHHOVSKudfOulrSOp3WOyDZu6QdvCchPGolfO0o/XBs/fNwfjZ0frl3/zy7////wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACH5BAkAABAALAAAAAAQABAAAAVVICSOZGlCQAosJ6mu7fiyZeKqNKToQGDsM8hBADgUXoGAiqhSvp5QAnQKGIgUhwFUYLCVDFCrKUE1lBavAViFIDlTImbKC5Gm2hB0SlBCBMQiB0UjIQA7"
}
]
}

Make a prediction

res = clip_endpoint.predict(data=data)

Print results

print(res)

Clean up the endpoint after use

sagemaker.Session().delete_endpoint(clip_endpoint.endpoint_name)

However when I run this, I get the following error:
"Loading /.sagemaker/mms/models/jinaai__jina-clip-v2 requires you to execute the configuration file in that repo on your local machine. Make sure you have read the code there to avoid malicious use, then set the option trust_remote_code\u003dTrue to remove this error."

I'd appreciate any help with this.

Hi @tararelan , thanks for reporting this.

We recommend subscribing and deploying Jina CLIP v2 Sagemaker model directly from our AWS page, because we do not maintain the HuggingFace Sagemaker package.

You will find a notebook describing how to start using your Sagemaker endpoint here.

Does that help?

Sign up or log in to comment