Error while deploying on SageMaker
I'm trying to deploy jina-clip-v2 on SageMaker using HuggingFace (not JumpStart) to embed some text and images. Below is my code:
import sagemaker
import json
from sagemaker.huggingface import HuggingFaceModel
Initialize SageMaker session and role
sess = sagemaker.Session()
role = sagemaker.get_execution_role()
model_kwargs = {"device": "cpu", "trust_remote_code":"True"}
encode_kwargs = {"normalize_embeddings": True}
Define model configuration
model_id = "jinaai/jina-clip-v2" # Model ID for Jina CLIP v2 from Hugging Face
instance_type = "ml.g5.xlarge" # Choose an appropriate instance type
Define environment variables
config = {
'HF_MODEL_ID': model_id,
'HF_TASK': 'feature-extraction', # Define the task
'HF_MODEL_TRUST_REMOTE_CODE': json.dumps(True) # Allow remote code execution
}
Create HuggingFaceModel
clip_model = HuggingFaceModel(
role=role,
env=config,
transformers_version='4.37.0',
pytorch_version='2.1.0',
py_version='py310'
)
Deploy the model to an endpoint
clip_endpoint = clip_model.deploy(
initial_instance_count=1,
instance_type=instance_type,
)
Prepare data for prediction (example)
data = {
"model": "jina-clip-v2",
"dimensions": 1024,
"normalized": True,
"embedding_type": "float",
"input": [
{"text": "A beautiful sunset over the beach"},
{"text": "Un beau coucher de soleil sur la plage"},
{"text": "海滩上美丽的日落"},
{"text": "浜辺に沈む美しい夕日"},
{"image": "https://i.ibb.co/nQNGqL0/beach1.jpg"},
{"image": "https://i.ibb.co/r5w8hG8/beach2.jpg"},
{
"image": "R0lGODlhEAAQAMQAAORHHOVSKudfOulrSOp3WOyDZu6QdvCchPGolfO0o/XBs/fNwfjZ0frl3/zy7////wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACH5BAkAABAALAAAAAAQABAAAAVVICSOZGlCQAosJ6mu7fiyZeKqNKToQGDsM8hBADgUXoGAiqhSvp5QAnQKGIgUhwFUYLCVDFCrKUE1lBavAViFIDlTImbKC5Gm2hB0SlBCBMQiB0UjIQA7"
}
]
}
Make a prediction
res = clip_endpoint.predict(data=data)
Print results
print(res)
Clean up the endpoint after use
sagemaker.Session().delete_endpoint(clip_endpoint.endpoint_name)
However when I run this, I get the following error:
"Loading /.sagemaker/mms/models/jinaai__jina-clip-v2 requires you to execute the configuration file in that repo on your local machine. Make sure you have read the code there to avoid malicious use, then set the option trust_remote_code\u003dTrue
to remove this error."
I'd appreciate any help with this.
Hi @tararelan , thanks for reporting this.
We recommend subscribing and deploying Jina CLIP v2 Sagemaker model directly from our AWS page, because we do not maintain the HuggingFace Sagemaker package.
You will find a notebook describing how to start using your Sagemaker endpoint here.
Does that help?