!function(){function t(t){document.documentElement.setAttribute("data-theme",t)}var e=function(){var t=null;try{t=new URLSearchParams(window.location.search).get("docusaurus-theme")}catch(t){}return t}()||function(){var t=null;try{t=localStorage.getItem("theme")}catch(t){}return t}();t(null!==e?e:"light")}(),document.documentElement.setAttribute("data-announcement-bar-initially-dismissed",function(){try{return"true"===localStorage.getItem("docusaurus.announcement.dismiss")}catch(t){}return!1}())

[Skip to main content](#__docusaurus_skipToContent_fallback)

LangChain v0.2 is coming soon! Preview the new docs [here](/v0.2/docs/introduction/).

[

![🦜️🔗 Langchain](/v0.1/img/brand/wordmark.png)![🦜️🔗 Langchain](/v0.1/img/brand/wordmark-dark.png)

](/v0.1/)[Docs](/v0.1/docs/get_started/introduction/)[Use cases](/v0.1/docs/use_cases/)[Integrations](/v0.1/docs/integrations/platforms/)[API Reference](https://api.js.langchain.com)

[More](#)

*   [People](/v0.1/docs/people/)
*   [Community](/v0.1/docs/community/)
*   [Tutorials](/v0.1/docs/additional_resources/tutorials/)
*   [Contributing](/v0.1/docs/contributing/)

[v0.1](#)

*   [v0.2](https://js.langchain.com/v0.2/docs/introduction)
*   [v0.1](/v0.1/docs/get_started/introduction/)

[🦜🔗](#)

*   [LangSmith](https://smith.langchain.com)
*   [LangSmith Docs](https://docs.smith.langchain.com)
*   [LangChain Hub](https://smith.langchain.com/hub)
*   [LangServe](https://github.com/langchain-ai/langserve)
*   [Python Docs](https://python.langchain.com/)

[Chat](https://chatjs.langchain.com)[](https://github.com/langchain-ai/langchainjs)

Search

*   [Get started](/v0.1/docs/get_started/)
    
    *   [Introduction](/v0.1/docs/get_started/introduction/)
    *   [Installation](/v0.1/docs/get_started/installation/)
    *   [Quickstart](/v0.1/docs/get_started/quickstart/)
*   [LangChain Expression Language](/v0.1/docs/expression_language/)
    
    *   [Get started](/v0.1/docs/expression_language/get_started/)
    *   [Why use LCEL?](/v0.1/docs/expression_language/why/)
    *   [Interface](/v0.1/docs/expression_language/interface/)
    *   [Streaming](/v0.1/docs/expression_language/streaming/)
    *   [How to](/v0.1/docs/expression_language/how_to/routing/)
        
    *   [Cookbook](/v0.1/docs/expression_language/cookbook/)
        
    *   [LangChain Expression Language (LCEL)](/v0.1/docs/expression_language/)
*   [Modules](/v0.1/docs/modules/)
    
    *   [Model I/O](/v0.1/docs/modules/model_io/)
        
    *   [Retrieval](/v0.1/docs/modules/data_connection/)
        
        *   [Document loaders](/v0.1/docs/modules/data_connection/document_loaders/)
            
        *   [Text Splitters](/v0.1/docs/modules/data_connection/document_transformers/)
            
        *   [Retrievers](/v0.1/docs/modules/data_connection/retrievers/)
            
        *   [Retrieval](/v0.1/docs/modules/data_connection/)
        *   [Text embedding models](/v0.1/docs/modules/data_connection/text_embedding/)
            
        *   [Vector stores](/v0.1/docs/modules/data_connection/vectorstores/)
            
        *   [Indexing](/v0.1/docs/modules/data_connection/indexing/)
        *   [Experimental](/v0.1/docs/modules/data_connection/experimental/multimodal_embeddings/google_vertex_ai/)
            
            *   [Multimodal embedding models](/v0.1/docs/modules/data_connection/experimental/multimodal_embeddings/google_vertex_ai/)
                
                *   [Google Vertex AI](/v0.1/docs/modules/data_connection/experimental/multimodal_embeddings/google_vertex_ai/)
            *   [Graph databases](/v0.1/docs/modules/data_connection/experimental/graph_databases/memgraph/)
                
    *   [Chains](/v0.1/docs/modules/chains/)
    *   [Agents](/v0.1/docs/modules/agents/)
        
    *   [More](/v0.1/docs/modules/memory/)
        
*   [Security](/v0.1/docs/security/)
*   [Guides](/v0.1/docs/guides/)
    
*   [Ecosystem](/v0.1/docs/ecosystem/)
    
*   [LangGraph](/v0.1/docs/langgraph/)
*   * * *
    

*   [](/v0.1/)
*   [Modules](/v0.1/docs/modules/)
*   [Retrieval](/v0.1/docs/modules/data_connection/)
*   Experimental
*   Multimodal embedding models
*   Google Vertex AI

On this page

Google Vertex AI
================

Experimental

This API is new and may change in future LangChain.js versions.

The `GoogleVertexAIMultimodalEmbeddings` class provides additional methods that are parallels to the `embedDocuments()` and `embedQuery()` methods:

*   `embedImage()` and `embedImageQuery()` take node `Buffer` objects that are expected to contain an image.
*   `embedMedia()` and `embedMediaQuery()` take an object that contain a `text` string field, an `image` Buffer field, or both and returns a similarly constructed object containing the respective vectors.

**Note:** The Google Vertex AI embeddings models have different vector sizes than OpenAI's standard model, so some vector stores may not handle them correctly.

*   The `textembedding-gecko` model in `GoogleVertexAIEmbeddings` provides 768 dimensions.
*   The `multimodalembedding@001` model in `GoogleVertexAIMultimodalEmbeddings` provides 1408 dimensions.

Setup[​](#setup "Direct link to Setup")
---------------------------------------

The Vertex AI implementation is meant to be used in Node.js and not directly in a browser, since it requires a service account to use.

Before running this code, you should make sure the Vertex AI API is enabled for the relevant project in your Google Cloud dashboard and that you've authenticated to Google Cloud using one of these methods:

*   You are logged into an account (using `gcloud auth application-default login`) permitted to that project.
*   You are running on a machine using a service account that is permitted to the project.
*   You have downloaded the credentials for a service account that is permitted to the project and set the `GOOGLE_APPLICATION_CREDENTIALS` environment variable to the path of this file.

tip

See [this section for general instructions on installing integration packages](/v0.1/docs/get_started/installation/#installing-integration-packages).

*   npm
*   Yarn
*   pnpm

    npm install google-auth-library @langchain/community

    yarn add google-auth-library @langchain/community

    pnpm add google-auth-library @langchain/community

Usage[​](#usage "Direct link to Usage")
---------------------------------------

Here's a basic example that shows how to embed image queries:

    import fs from "fs";import { GoogleVertexAIMultimodalEmbeddings } from "langchain/experimental/multimodal_embeddings/googlevertexai";const model = new GoogleVertexAIMultimodalEmbeddings();// Load the image into a buffer to get the embedding of itconst img = fs.readFileSync("/path/to/file.jpg");const imgEmbedding = await model.embedImageQuery(img);console.log({ imgEmbedding });// You can also get text embeddingsconst textEmbedding = await model.embedQuery(  "What would be a good company name for a company that makes colorful socks?");console.log({ textEmbedding });

#### API Reference:

*   [GoogleVertexAIMultimodalEmbeddings](https://api.js.langchain.com/classes/langchain_experimental_multimodal_embeddings_googlevertexai.GoogleVertexAIMultimodalEmbeddings.html) from `langchain/experimental/multimodal_embeddings/googlevertexai`

Advanced usage[​](#advanced-usage "Direct link to Advanced usage")
------------------------------------------------------------------

Here's a more advanced example that shows how to integrate these new embeddings with a LangChain vector store.

    import fs from "fs";import { GoogleVertexAIMultimodalEmbeddings } from "langchain/experimental/multimodal_embeddings/googlevertexai";import { FaissStore } from "@langchain/community/vectorstores/faiss";import { Document } from "@langchain/core/documents";const embeddings = new GoogleVertexAIMultimodalEmbeddings();const vectorStore = await FaissStore.fromTexts(  ["dog", "cat", "horse", "seagull"],  [{ id: 2 }, { id: 1 }, { id: 3 }, { id: 4 }],  embeddings);const img = fs.readFileSync("parrot.jpeg");const vectors: number[] = await embeddings.embedImageQuery(img);const document = new Document({  pageContent: img.toString("base64"),  // Metadata is optional but helps track what kind of document is being retrieved  metadata: {    id: 5,    mediaType: "image",  },});// Add the image embedding vectors to the vector store directlyawait vectorStore.addVectors([vectors], [document]);// Use a similar image to the one just addedconst img2 = fs.readFileSync("parrot-icon.png");const vectors2: number[] = await embeddings.embedImageQuery(img2);// Use the lower level, direct APIconst resultTwo = await vectorStore.similaritySearchVectorWithScore(  vectors2,  2);console.log(JSON.stringify(resultTwo, null, 2));/*  [    [      Document {        pageContent: '<BASE64 ENCODED IMAGE DATA>'        metadata: {          id: 5,          mediaType: "image"        }      },      0.8931522965431213    ],    [      Document {        pageContent: 'seagull',        metadata: {          id: 4        }      },      1.9188631772994995    ]  ]*/

#### API Reference:

*   [GoogleVertexAIMultimodalEmbeddings](https://api.js.langchain.com/classes/langchain_experimental_multimodal_embeddings_googlevertexai.GoogleVertexAIMultimodalEmbeddings.html) from `langchain/experimental/multimodal_embeddings/googlevertexai`
*   [FaissStore](https://api.js.langchain.com/classes/langchain_community_vectorstores_faiss.FaissStore.html) from `@langchain/community/vectorstores/faiss`
*   [Document](https://api.js.langchain.com/classes/langchain_core_documents.Document.html) from `@langchain/core/documents`

* * *

#### Help us out by providing feedback on this documentation page:

[

Previous

Indexing

](/v0.1/docs/modules/data_connection/indexing/)[

Next

Memgraph

](/v0.1/docs/modules/data_connection/experimental/graph_databases/memgraph/)

*   [Setup](#setup)
*   [Usage](#usage)
*   [Advanced usage](#advanced-usage)

Community

*   [Discord](https://discord.gg/cU2adEyC7w)
*   [Twitter](https://twitter.com/LangChainAI)

GitHub

*   [Python](https://github.com/langchain-ai/langchain)
*   [JS/TS](https://github.com/langchain-ai/langchainjs)

More

*   [Homepage](https://langchain.com)
*   [Blog](https://blog.langchain.dev)

Copyright © 2024 LangChain, Inc.