# embedding_metadata

key: chroma:document
string_value:
- name: Notion-Version
          in: header
          required: true
          schema:
            type: string
          example: 2022-06-28
          constant: 2022-06-28
      requestBody:
        required: true
        content:
          application/json:
            schema:
              type: object
              properties:
                query:
                  type: string
                filter:
                  type: object
                  properties:
                    value:
                      type: string
                    property:
                      type: string
                sort:
                  type: object
                  properties:
                    direction:
                      type: string
                    timestamp:
                      type: string
      responses:
        '200':
          description: Successful response
          content:
            application/json:
              schema:
                type: object
                properties:
                  object:
                    type: string
                  results:
                    type: array
                    items:
                      type: object


key: _node_content
string_value:
{"id_": "b9ea94ee-5082-4271-96b0-5fa7e214c1b7", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_action_notion.ipynb", "title": "GPT Action Library: Notion", "tokens": 2577, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "aaff4c3f-363a-4c5a-af5e-0b389ce78a0c", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_action_notion.ipynb", "title": "GPT Action Library: Notion", "tokens": 2577, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "ae60ddb2fce497ae81bb82ff3e96665499472ec115611458fb760a9186304e58", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "13f92fc3-288f-48cf-b950-acae7e32fb5b", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_action_notion.ipynb", "title": "GPT Action Library: Notion", "tokens": 2577, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "d33430e7d788c83614ee01cae47a6b6c828055b905afb1328a357b48f6e3b571", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "17f8c7ce-1c1c-40d4-88a5-88a17351c61a", "node_type": "1", "metadata": {}, "hash": "5afdf0547b1a660027b54010b5405481ee798e3b9539f8bcc6df3dab6422237a", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 10862, "end_char_idx": 11918, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
type: string
                sort:
                  type: object
                  properties:
                    direction:
                      type: string
                    timestamp:
                      type: string
      responses:
        '200':
          description: Successful response
          content:
            application/json:
              schema:
                type: object
                properties:
                  object:
                    type: string
                  results:
                    type: array
                    items:
                      type: object
                      properties:
                        id:
                          type: string
                        title:
                          type: array
                          items:
                            type: object
                            properties:
                              type:
                                type: string
                              text:
                                type: object


key: _node_content
string_value:
{"id_": "17f8c7ce-1c1c-40d4-88a5-88a17351c61a", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_action_notion.ipynb", "title": "GPT Action Library: Notion", "tokens": 2577, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "aaff4c3f-363a-4c5a-af5e-0b389ce78a0c", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_action_notion.ipynb", "title": "GPT Action Library: Notion", "tokens": 2577, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "ae60ddb2fce497ae81bb82ff3e96665499472ec115611458fb760a9186304e58", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "b9ea94ee-5082-4271-96b0-5fa7e214c1b7", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_action_notion.ipynb", "title": "GPT Action Library: Notion", "tokens": 2577, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "4880a446ce373c23a70d552664365648b336c4144d8e3137778868da151de66b", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "922ff86a-8fa1-429e-af6b-d3c289378b0c", "node_type": "1", "metadata": {}, "hash": "d67333315593f6ea3adf590eda97ab75757d15c4e93f4b653e4c04f01f809836", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 11460, "end_char_idx": 13312, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
type: object
                      properties:
                        id:
                          type: string
                        title:
                          type: array
                          items:
                            type: object
                            properties:
                              type:
                                type: string
                              text:
                                type: object
                                properties:
                                  content:
                                    type: string
```

## Authentication Instructions

Below are instructions on setting up authentication with this 3rd party application.

### Pre-Action Steps

Before you set up authentication in ChatGPT, please take the following steps in the Notion.

1. Go to the Notion Settings Page for your workspace
2. Navigate to My Connections > Develop or Manage Integrations
3. Create new Integration marked as Internal
4. Locate your integration and find the API Key labeled: Internal Integration Secret.  This is the bearer token for this integration.

**NOTE!** You need to share specific pages, databases, or wikis with the integration in order to access them in ChatGPT.  Do this by selecting the ... button on the upper right of a page and select the appropriate connection.

**NOTE!** Notion allows integrations to leverage OAuth if they are marked as "Public."  Review [Notion's Auth Documentation](https://developers.notion.com/docs/authorization) to determine what integration path is best for your needs.

![notion_connections.png](../../../images/creating_notion_integration.png)

![sharing_notion_pages.png](../../../images/sharing_notion_with_GPT.png)

### In ChatGPT

In ChatGPT, click on "Authentication" and choose **"API Key"**. Enter in the information below.


key: _node_content
string_value:
{"id_": "922ff86a-8fa1-429e-af6b-d3c289378b0c", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_action_notion.ipynb", "title": "GPT Action Library: Notion", "tokens": 2577, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "aaff4c3f-363a-4c5a-af5e-0b389ce78a0c", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_action_notion.ipynb", "title": "GPT Action Library: Notion", "tokens": 2577, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "ae60ddb2fce497ae81bb82ff3e96665499472ec115611458fb760a9186304e58", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "17f8c7ce-1c1c-40d4-88a5-88a17351c61a", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_action_notion.ipynb", "title": "GPT Action Library: Notion", "tokens": 2577, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "75d9d075763c5ee683031adba2e3d64c8ff9e8039e477bedc1add7b4f2160ab9", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 11906, "end_char_idx": 13751, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
type: object
                                properties:
                                  content:
                                    type: string
```

## Authentication Instructions

Below are instructions on setting up authentication with this 3rd party application.

### Pre-Action Steps

Before you set up authentication in ChatGPT, please take the following steps in the Notion.

1. Go to the Notion Settings Page for your workspace
2. Navigate to My Connections > Develop or Manage Integrations
3. Create new Integration marked as Internal
4. Locate your integration and find the API Key labeled: Internal Integration Secret.  This is the bearer token for this integration.

**NOTE!** You need to share specific pages, databases, or wikis with the integration in order to access them in ChatGPT.  Do this by selecting the ... button on the upper right of a page and select the appropriate connection.

**NOTE!** Notion allows integrations to leverage OAuth if they are marked as "Public."  Review [Notion's Auth Documentation](https://developers.notion.com/docs/authorization) to determine what integration path is best for your needs.

![notion_connections.png](../../../images/creating_notion_integration.png)

![sharing_notion_pages.png](../../../images/sharing_notion_with_GPT.png)

### In ChatGPT

In ChatGPT, click on "Authentication" and choose **"API Key"**. Enter in the information below. 

- **API Key**: Use Internal Integration Secret from steps above
- **Auth Type**: Bearer

### FAQ & Troubleshooting

- *Search returns nothing* If you don't see any pages returned when running a search, double check that you've shared relevant pages with the application from Notion

*Are there integrations that you’d like us to prioritize? Are there errors in our integrations? File a PR or issue in our github, and we’ll take a look.*


key: url
string_value:
https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_aws_function.ipynb


key: _node_content
string_value:
{"id_": "30365037-ce64-497e-9eff-be057619cdb1", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_aws_function.ipynb", "title": "GPT Action Library (Middleware): AWS Lambda", "tokens": 2682, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "48583bac-d554-4e75-88c7-1c063eaeb208", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_aws_function.ipynb", "title": "GPT Action Library (Middleware): AWS Lambda", "tokens": 2682, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "2799badb793ed877e71587d780469e74aa2b306afd6569a043865d031e793118", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "fd8273b2-baca-4711-bb22-ba8cbb7dcf45", "node_type": "1", "metadata": {}, "hash": "4133382288d57b8db8c8c3f45f84ad071e1584c7af01a09edf1df8e7dc2e166d", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 0, "end_char_idx": 3275, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
# GPT Action Library (Middleware): AWS Lambda

## Introduction

This particular GPT Action provides an overview of how to build an **AWS Lambda** function. This documentation helps a user set up an OAuth-protected AWS Function to connect to a GPT Action, and to a sample application. This example uses AWS SAM (Serverless Application Model) in this example to set-up the AWS stack.

### Value + Example Business Use Cases

**Value**: Users can now leverage ChatGPT's capabilities to connect to an AWS Function. This enables you to connect to any services in AWS and run code/applications on this. This can in a few ways:

- Access 3rd party services such as AWS Redshift, AWS DynamoDB, AWS S3 and even more!
- Allows pre-processing text responses from an API (overcoming context limits, adding context or metadata as examples).
- Enables to return files instead of retrieving text from 3rd party APIs. This can be useful to surface CSV files for Data Analysis, or bring back an PDF file and ChatGPT will treat it like an upload. 


**Example Use Cases**: 
- A user needs to look up data in Redshift, but needs a middleware app between ChatGPT and Redshift to return files (data analysis data exactitude as well as large number of data)
- A user has built several steps in an AWS function, and needs to be able to kick off that process using ChatGPT.

## Application information & prerequisites

We will leverage AWS Lambda services to create a middleware function. You can get familiar with this stack by visiting the following links: 

- Lambda Website: https://aws.amazon.com/lambda/
- Lambda Documentation: https://docs.aws.amazon.com/lambda/
- AWS SAM docs: https://docs.aws.amazon.com/serverless-application-model/

### Prerequisites

Before you get started, make sure you have an AWS Console with access to create: Lambda Function, S3 Buckets, Application Stack, Cognito User Pool, Cognito User Pool App Clients, API Gateway, Lambda roles, CloudFormation stacks (this feels like a lot but creating those services is automated!).

## Create AWS Lambda Function

To create an AWS Function you can use AWS SAM. An example of a SAM Template can be found [here](https://github.com/pap-openai/redshift-middleware/blob/main/template.yaml) [0].

This template includes:
- A User Pool & User Pool Client, used for OAuth
- A Cognito Authorizer that ensure the function can only be called by authenticated users
- Mapping the Lambda function to an existing VPC (useful to connect to other AWS services)
- Has parameters that can be set-up dynamically (e.g: credentials/variables)
- An API Gateway that maps HTTP routes to the functions

This code is purely informational to help you get started and doesn't require pre-existing AWS resources. We recommend to map existing user pools if you have any instead of creating new ones, as well as setting up your Lambda in a VPC that has access to other AWS Resources (if you need to leverage those). You can see an example of a set-up like this in the [RedShift cookbook](https://cookbook.openai.com/examples/chatgpt/gpt_actions_library/gpt_middleware_aws_function).

The Cognito Authorizer is key to make sure your function can only be called/accessed by authenticated users so make sure to set this up correctly with your environment.


key: _node_content
string_value:
{"id_": "fd8273b2-baca-4711-bb22-ba8cbb7dcf45", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_aws_function.ipynb", "title": "GPT Action Library (Middleware): AWS Lambda", "tokens": 2682, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "48583bac-d554-4e75-88c7-1c063eaeb208", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_aws_function.ipynb", "title": "GPT Action Library (Middleware): AWS Lambda", "tokens": 2682, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "2799badb793ed877e71587d780469e74aa2b306afd6569a043865d031e793118", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "30365037-ce64-497e-9eff-be057619cdb1", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_aws_function.ipynb", "title": "GPT Action Library (Middleware): AWS Lambda", "tokens": 2682, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "f51e688f3f956a0daa7b52faee231e17efd5657af9e75c981b9b85bf511ee8fa", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "a87bee54-7f65-42ff-a417-339f3d9268d6", "node_type": "1", "metadata": {}, "hash": "8a2136959021e9e0f8078fe22ebc2a6e061a3f1315c166caef6a1639eaa521ff", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 1465, "end_char_idx": 5990, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
You can get familiar with this stack by visiting the following links: 

- Lambda Website: https://aws.amazon.com/lambda/
- Lambda Documentation: https://docs.aws.amazon.com/lambda/
- AWS SAM docs: https://docs.aws.amazon.com/serverless-application-model/

### Prerequisites

Before you get started, make sure you have an AWS Console with access to create: Lambda Function, S3 Buckets, Application Stack, Cognito User Pool, Cognito User Pool App Clients, API Gateway, Lambda roles, CloudFormation stacks (this feels like a lot but creating those services is automated!).

## Create AWS Lambda Function

To create an AWS Function you can use AWS SAM. An example of a SAM Template can be found [here](https://github.com/pap-openai/redshift-middleware/blob/main/template.yaml) [0].

This template includes:
- A User Pool & User Pool Client, used for OAuth
- A Cognito Authorizer that ensure the function can only be called by authenticated users
- Mapping the Lambda function to an existing VPC (useful to connect to other AWS services)
- Has parameters that can be set-up dynamically (e.g: credentials/variables)
- An API Gateway that maps HTTP routes to the functions

This code is purely informational to help you get started and doesn't require pre-existing AWS resources. We recommend to map existing user pools if you have any instead of creating new ones, as well as setting up your Lambda in a VPC that has access to other AWS Resources (if you need to leverage those). You can see an example of a set-up like this in the [RedShift cookbook](https://cookbook.openai.com/examples/chatgpt/gpt_actions_library/gpt_middleware_aws_function).

The Cognito Authorizer is key to make sure your function can only be called/accessed by authenticated users so make sure to set this up correctly with your environment.

[0]
```
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
  aws-middleware

  AWS middleware function

Parameters:
  CognitoUserPoolName:
    Type: String
    Default: MyCognitoUserPool
  CognitoUserPoolClientName:
    Type: String
    Default: MyCognitoUserPoolClient

Resources:
  MyCognitoUserPool:
    Type: AWS::Cognito::UserPool
    Properties:
      UserPoolName: !Ref CognitoUserPoolName
      Policies:
        PasswordPolicy:
          MinimumLength: 8
      UsernameAttributes:
        - email
      Schema:
        - AttributeDataType: String
          Name: email
          Required: false

  MyCognitoUserPoolClient:
    Type: AWS::Cognito::UserPoolClient
    Properties:
      UserPoolId: !Ref MyCognitoUserPool
      ClientName: !Ref CognitoUserPoolClientName
      GenerateSecret: true

  MiddlewareApi:
    Type: AWS::Serverless::Api
    Properties:
      StageName: Prod
      Cors: "'*'"
      Auth:
        DefaultAuthorizer: MyCognitoAuthorizer
        Authorizers:
          MyCognitoAuthorizer:
            AuthorizationScopes:
              - openid
              - email
              - profile
            UserPoolArn: !GetAtt MyCognitoUserPool.Arn
        
  MiddlewareFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: aws-middleware/
      Handler: app.lambda_handler
      Runtime: python3.11
      Timeout: 45
      Architectures:
        - x86_64
      Events:
        SqlStatement:
          Type: Api
          Properties:
            Path: /my_route
            Method: post
            RestApiId: !Ref MiddlewareApi

Outputs:
  MiddlewareApi:
    Description: "API Gateway endpoint URL for Prod stage for SQL Statement function"
    Value: !Sub "https://${MiddlewareApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/my_route"
  MiddlewareFunction:
    Description: "SQL Statement Lambda Function ARN"
    Value: !GetAtt MiddlewareFunction.Arn
  MiddlewareFunctionIamRole:
    Description: "Implicit IAM Role created for SQL Statement function"
    Value: !GetAtt MiddlewareFunctionRole.Arn
  CognitoUserPoolArn:
    Description: "ARN of the Cognito User Pool"
    Value: !GetAtt MyCognitoUserPool.Arn
```

You can clone the openai-cookbook repository & take the sample python code & SAM template from the `lambda-middleware` directory:

```
git clone https://github.com/pap-openai/lambda-middleware
cd lambda-middleware
```

To build & deploy your function, run the following commands from this directory

```
sam build
sam deploy --template-file template.yaml --stack-name aws-middleware --capabilities CAPABILITY_IAM
```

Once you have this deployed, you can go check out the application on AWS Lambda:

!


key: _node_content
string_value:
{"id_": "a87bee54-7f65-42ff-a417-339f3d9268d6", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_aws_function.ipynb", "title": "GPT Action Library (Middleware): AWS Lambda", "tokens": 2682, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "48583bac-d554-4e75-88c7-1c063eaeb208", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_aws_function.ipynb", "title": "GPT Action Library (Middleware): AWS Lambda", "tokens": 2682, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "2799badb793ed877e71587d780469e74aa2b306afd6569a043865d031e793118", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "fd8273b2-baca-4711-bb22-ba8cbb7dcf45", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_aws_function.ipynb", "title": "GPT Action Library (Middleware): AWS Lambda", "tokens": 2682, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "f2685b46eb96333d56ae156ba03f1b48296c4071979b6ccdc70fa25e8228b0d4", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "0f5d8cac-92e4-4611-ab83-c6871c5a0036", "node_type": "1", "metadata": {}, "hash": "0e430dfa0eefd597a3637a0a73039bf9512f4671401aba98a6ccdf67e9f9fdf4", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 5990, "end_char_idx": 9039, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
[../../../images/aws_lambda_1.png](../../../images/aws_lambda_1.png)

You can confirm that the function is not reachable unless authenticated by running a curl command without any authentication:

```
curl -d {} <middleware_api_output_url_from_deploy_command>
```

which should return `{"message":"Unauthorized"}`.

## Set up Auth in AWS Cognito

_Optional: do those steps only if you created a user pool and are not using an existing one_ 

Let's create a user in the newly user pool. To do that, fetch the output of CognitoUserPoolArn in the deploy command, and get the value after the "/", which should be in the format of: `your-region_xxxxx`.

```
aws cognito-idp admin-create-user \
    --user-pool-id "your-region_xxxxx" \
    --username johndoe@example.com \
    --user-attributes Name=email,Value=johndoe@example.com \
    --temporary-password "TempPassword123"
```

Let's now make sure we create a webpage/domain on which we can log-in. Go to AWS Cognito, select the newly created user pool & go to App Integration tab:

![../../../images/aws_lambda_3.png](../../../images/aws_lambda_3.png)

Create a Cognito Domain by clicking on "Domains" then "Create Cognito Domain"

![../../../images/aws_lambda_8.png](../../../images/aws_lambda_8.png)

Scroll down to `App client list` on the App Integration page of your User Pool:

![../../../images/aws_lambda_9.png](../../../images/aws_lambda_9.png)

Select your app client and edit the Hosted UI:

![../../../images/aws_lambda_10.png](../../../images/aws_lambda_10.png)

And add a callback URL, Authorization Scheme and OAuth scope:

![../../../images/aws_lambda_11.png](../../../images/aws_lambda_11.png)

_Note that you'll come back to this step when ChatGPT will generate a callback URL for the authentication of your action. The postman URL, should be used only for development purpose._

You can try this connection in Postman, under Authorization for your `<api_url>`, copy/paste the value from AWS for the client_id, client_secret and the URL you set up for the auth domain, make sure to add `openid` in the scope to get a valid access_token:

![../../../images/aws_lambda_12.png](../../../images/aws_lambda_12.png)

![../../../images/aws_lambda_13.png](../../../images/aws_lambda_13.png)

If you're now doing the request on Postman, using the access_token you just retrieve, you'll get a success JSON returned:

![../../../images/aws_lambda_14.png](../../../images/aws_lambda_14.png)

## Create Action in ChatGPT

Now let's integrate this into ChatGPT.

Create an action and copy paste the following spec:

```
openapi: 3.1.0
info:
  title: Success API
  description: API that returns a success message.
  version: 1.0.0
servers:
  - url: https://3ho5n15aef.execute-api.us-east-1.amazonaws.com/Prod
    description: Main production server
paths:
  /my_route:
    post:
      operationId: postSuccess
      summary: Returns a success message.
      description: Endpoint to check the success status.
      responses:
        '200':
          description: A JSON object indicating success.


key: _node_content
string_value:
{"id_": "0f5d8cac-92e4-4611-ab83-c6871c5a0036", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_aws_function.ipynb", "title": "GPT Action Library (Middleware): AWS Lambda", "tokens": 2682, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "48583bac-d554-4e75-88c7-1c063eaeb208", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_aws_function.ipynb", "title": "GPT Action Library (Middleware): AWS Lambda", "tokens": 2682, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "2799badb793ed877e71587d780469e74aa2b306afd6569a043865d031e793118", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "a87bee54-7f65-42ff-a417-339f3d9268d6", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_aws_function.ipynb", "title": "GPT Action Library (Middleware): AWS Lambda", "tokens": 2682, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "1978f21c1cda94a5f0726b9380ad179e3a6d576354eeac27b31b740f2013d406", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "dd215a93-a3b5-4970-9212-52cc9416c530", "node_type": "1", "metadata": {}, "hash": "a65f6a674aa92b759fa53ef12d0f193b3add7702675fcebb795737ad396d2971", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 7443, "end_char_idx": 10702, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
[../../../images/aws_lambda_10.png](../../../images/aws_lambda_10.png)

And add a callback URL, Authorization Scheme and OAuth scope:

![../../../images/aws_lambda_11.png](../../../images/aws_lambda_11.png)

_Note that you'll come back to this step when ChatGPT will generate a callback URL for the authentication of your action. The postman URL, should be used only for development purpose._

You can try this connection in Postman, under Authorization for your `<api_url>`, copy/paste the value from AWS for the client_id, client_secret and the URL you set up for the auth domain, make sure to add `openid` in the scope to get a valid access_token:

![../../../images/aws_lambda_12.png](../../../images/aws_lambda_12.png)

![../../../images/aws_lambda_13.png](../../../images/aws_lambda_13.png)

If you're now doing the request on Postman, using the access_token you just retrieve, you'll get a success JSON returned:

![../../../images/aws_lambda_14.png](../../../images/aws_lambda_14.png)

## Create Action in ChatGPT

Now let's integrate this into ChatGPT.

Create an action and copy paste the following spec:

```
openapi: 3.1.0
info:
  title: Success API
  description: API that returns a success message.
  version: 1.0.0
servers:
  - url: https://3ho5n15aef.execute-api.us-east-1.amazonaws.com/Prod
    description: Main production server
paths:
  /my_route:
    post:
      operationId: postSuccess
      summary: Returns a success message.
      description: Endpoint to check the success status.
      responses:
        '200':
          description: A JSON object indicating success.
          content:
            application/json:
              schema:
                type: object
                properties:
                  success:
                    type: boolean
                    example: true
```

If you try to test the action (you can click the "Test" Button), you'll see that you have a 401 as you're not authenticated.

Let's now add authentication in the action.

Click on Authentication > OAuth.
We'll now need to fetch AWS Cognito's variables. Let's go on your User Pool > User Pool App Client. From there you can retrieve your client ID and client Secret.

![../../../images/aws_lambda_15.png](../../../images/aws_lambda_15.png)

Copy paste those values in ChatGPT. Now let's add the Token URLs.

From your User Pool you'll find the URL you've previously created for the hosted domain.

![../../../images/aws_lambda_16.png](../../../images/aws_lambda_16.png)

We'll take this URL and append [AWS routes for OAuth](https://docs.aws.amazon.com/cognito/latest/developerguide/federation-endpoints.html).

- token: `<your_url>/oauth2/token`
- authorization: `<your_url>/oauth2/authorize`

Copy paste those in ChatGPT.

In scope, add openid and click on Save.

## Configure Cognito with ChatGPT URL

Now go back on your GPT (moving out of the action subview), and you'll see a callback URL provided by ChatGPT for the Authentication:

![../../../images/aws_lambda_17.png](../../../images/aws_lambda_17.png)

Get this URL and edit the hosted UI of your User Pool App client & save the changes:

![../../../images/aws_lambda_18.png](../../../images/aws_lambda_18.png)

## Testing the function

You can now test this action again:

!


key: _node_content
string_value:
{"id_": "dd215a93-a3b5-4970-9212-52cc9416c530", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_aws_function.ipynb", "title": "GPT Action Library (Middleware): AWS Lambda", "tokens": 2682, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "48583bac-d554-4e75-88c7-1c063eaeb208", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_aws_function.ipynb", "title": "GPT Action Library (Middleware): AWS Lambda", "tokens": 2682, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "2799badb793ed877e71587d780469e74aa2b306afd6569a043865d031e793118", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "0f5d8cac-92e4-4611-ab83-c6871c5a0036", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_aws_function.ipynb", "title": "GPT Action Library (Middleware): AWS Lambda", "tokens": 2682, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "51bb5a88a383e3158b7d7a062e7091e72d2e381105f546345439b7d3b02ad4c7", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 8900, "end_char_idx": 11804, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
description: Endpoint to check the success status.
      responses:
        '200':
          description: A JSON object indicating success.
          content:
            application/json:
              schema:
                type: object
                properties:
                  success:
                    type: boolean
                    example: true
```

If you try to test the action (you can click the "Test" Button), you'll see that you have a 401 as you're not authenticated.

Let's now add authentication in the action.

Click on Authentication > OAuth.
We'll now need to fetch AWS Cognito's variables. Let's go on your User Pool > User Pool App Client. From there you can retrieve your client ID and client Secret.

![../../../images/aws_lambda_15.png](../../../images/aws_lambda_15.png)

Copy paste those values in ChatGPT. Now let's add the Token URLs.

From your User Pool you'll find the URL you've previously created for the hosted domain.

![../../../images/aws_lambda_16.png](../../../images/aws_lambda_16.png)

We'll take this URL and append [AWS routes for OAuth](https://docs.aws.amazon.com/cognito/latest/developerguide/federation-endpoints.html).

- token: `<your_url>/oauth2/token`
- authorization: `<your_url>/oauth2/authorize`

Copy paste those in ChatGPT.

In scope, add openid and click on Save.

## Configure Cognito with ChatGPT URL

Now go back on your GPT (moving out of the action subview), and you'll see a callback URL provided by ChatGPT for the Authentication:

![../../../images/aws_lambda_17.png](../../../images/aws_lambda_17.png)

Get this URL and edit the hosted UI of your User Pool App client & save the changes:

![../../../images/aws_lambda_18.png](../../../images/aws_lambda_18.png)

## Testing the function

You can now test this action again:

![../../../images/aws_lambda_19.png](../../../images/aws_lambda_19.png)

You will be redirected to AWS Cognito page, which you can log-in in using the credentials previously set-up.

If you now ask the GPT to run the same action, it will answer correctly as you're now authenticated and able to run this function!

![../../../images/aws_lambda_20.png](../../../images/aws_lambda_20.png)

# Conclusion

You've now set-up an action in ChatGPT that can talk with your applications in AWS, in an authenticated way! This cookbook shows you how to create the Cognito Pool from scratch using username/password, though, we recommend to set-up Cognito based on your needs (for example by plugging your own IDP into Cognito).

Additionally, the function is not connected to any other services, which is the advantage of being able to communicate to an AWS Lambda function in a safe way. You can therefore tweak the code and AWS SAM template to fit your need. An example of a more complex function is Redshift, that follows those steps to create the function and authentication but has a different code/deployment.


key: url
string_value:
https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_action_gmail.ipynb


key: _node_content
string_value:
{"id_": "db1ac6f6-61b2-41c0-97bd-b19964fc2574", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_action_gmail.ipynb", "title": "GPT Action Library: Gmail", "tokens": 3067, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "d3b081e2-0794-4104-9438-9a883cd4dfb4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_action_gmail.ipynb", "title": "GPT Action Library: Gmail", "tokens": 3067, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "4c7a0ed91cae5a94ac501e4cdb5a16d67f5df92023bce088133a45a8487d5cf6", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "ccb89e26-16d1-495b-92ab-8fd009d2d26d", "node_type": "1", "metadata": {}, "hash": "f856140c7b0a5d694bf611feac6f24a0c9e26d03e7eb662989461beffb70ce34", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 0, "end_char_idx": 2242, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
# GPT Action Library: Gmail

## Introduction

This page provides an instruction & guide for developers building a GPT Action for a specific application. Before you proceed, make sure to first familiarize yourself with the following information: 
- [Introduction to GPT Actions](https://platform.openai.com/docs/actions)
- [Introduction to GPT Actions Library](https://platform.openai.com/docs/actions/actions-library)
- [Example of Building a GPT Action from Scratch](https://platform.openai.com/docs/actions/getting-started)

This GPT Action provides an overview of how to connect to Google Gmail, Google’s Private & Secure Email for Personal or Business. This Action is connected to the Google Gmail APIs that can read, send, list, and draft emails in the authorized account. 

### Value + Example Business Use Cases

**Value**: The Gmail GPT will serve as a powerful tool to streamline communication processes, improve customer engagement, and optimize resource allocation.

**Example Use Cases**: 
- Manage internal communications by summarizing lengthy emails and drafting responses based on previous email threads.
- Support agents can provide customers with instant responses adhering to a company’s communication guidelines, tone, and style.
- Reference other GPTs , such as a data analsys GPT, and then ask for a draft/send of the consolidated analysis through email communication.

## Application Information

### Application Key Links

Check out these links from the application before you get started:
- Application Website: https://mail.google.com/mail/u/0/#inbox
- Application API Documentation: https://developers.google.com/gmail/api/guides

### Application Prerequisites

Before you get started, make sure you’ve a Google Cloud account and that the Gmail API is enabled:
- Set up a Google Cloud project
- Enable Gmail API from Google API Library
- If application’s  “Publishing Status” is “Testing”, ensure users are added to your application

## ChatGPT Steps

### Custom GPT Instructions 

Once you've created a Custom GPT, copy the text below in the Instructions panel. Have questions? Check out [Getting Started Example](https://platform.openai.com/docs/actions/getting-started) to see how this step works in more detail.


key: _node_content
string_value:
{"id_": "ccb89e26-16d1-495b-92ab-8fd009d2d26d", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_action_gmail.ipynb", "title": "GPT Action Library: Gmail", "tokens": 3067, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "d3b081e2-0794-4104-9438-9a883cd4dfb4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_action_gmail.ipynb", "title": "GPT Action Library: Gmail", "tokens": 3067, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "4c7a0ed91cae5a94ac501e4cdb5a16d67f5df92023bce088133a45a8487d5cf6", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "db1ac6f6-61b2-41c0-97bd-b19964fc2574", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_action_gmail.ipynb", "title": "GPT Action Library: Gmail", "tokens": 3067, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "57622e903df6f69306d71171947f76d2c7c0a0f3913e789e93d4cb2c4131292a", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "0a64b888-17f2-4ab4-86e9-d741690b41d5", "node_type": "1", "metadata": {}, "hash": "22b1ead1c96e47155276a438d7665db6e13aa297d3bf0698a44b5520b704b4a4", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 2245, "end_char_idx": 5424, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
```python
**Context**
Act as an email assistant designed to enhance user interaction with emails in various ways. This GPT can assist with productivity by summarizing emails/threads, identifying next steps/follow-ups, drafting or sending pre-written responses, and programmatically interacting with third-party tools (e.g., Notion to-dos, Slack channel summaries, data extraction for responses). This GPT has full scope access to the GMAIL OAuth 2.0 API, capable of reading, composing, sending, and permanently deleting emails from Gmail.

**Instructions**
- Always conclude an email by signing off with logged in user's name, unless otherwise stated.
- Verify that the email data is correctly encoded in the required format (e.g., base64 for the message body).
- Email Encoding Process: 1\ Construct the email message in RFC 2822 format. 2\ Base64 encode the email message. 3\Send the encoded message using the API.
- If not specified, sign all emails with the user name.
- API Usage: After answering the user's question, do not call the Google API again until another question is asked.
- All emails created, draft or sent, should be in plain text.
- Ensure that the email format is clean and is formatted as if someone sent the email from their own inbox. Once a draft is created or email sent, display a message to the user confirming that the draft is ready or the email is sent.
- Check that the "to" email address is valid and in the correct format. It should be in the format "recipient@example.com". 
- Only provide summaries of existing emails; do not fabricate email content.
- Professionalism: Behave professionally, providing clear and concise responses.
- Clarification: Ask for clarification when needed to ensure accuracy and completeness in fulfilling user requests.
- Privacy and Security: Respect user privacy and handle all data securely.

```

### OpenAPI Schema 

Once you've created a Custom GPT, copy the text below in the Actions panel. Have questions? Check out [Getting Started Example](https://platform.openai.com/docs/actions/getting-started) to see how this step works in more detail.


```python
openapi: 3.1.0
info:
  title: Gmail Email API
  version: 1.0.0
  description: API to read, write, and send emails in a Gmail account.
servers:
  - url: https://gmail.googleapis.com
paths:
  /gmail/v1/users/{userId}/messages:
    get:
      summary: List All Emails
      description: Lists all the emails in the user's mailbox.
      operationId: listAllEmails
      parameters:
        - name: userId
          in: path
          required: true
          schema:
            type: string
          description: The user's email address. Use "me" to indicate the authenticated user.
        - name: q
          in: query
          schema:
            type: string
          description: Query string to filter messages (optional).
        - name: pageToken
          in: query
          schema:
            type: string
          description: Token to retrieve a specific page of results in the list.
        - name: maxResults
          in: query
          schema:
            type: integer
          description: Maximum number of messages to return.


key: _node_content
string_value:
{"id_": "0a64b888-17f2-4ab4-86e9-d741690b41d5", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_action_gmail.ipynb", "title": "GPT Action Library: Gmail", "tokens": 3067, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "d3b081e2-0794-4104-9438-9a883cd4dfb4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_action_gmail.ipynb", "title": "GPT Action Library: Gmail", "tokens": 3067, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "4c7a0ed91cae5a94ac501e4cdb5a16d67f5df92023bce088133a45a8487d5cf6", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "ccb89e26-16d1-495b-92ab-8fd009d2d26d", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_action_gmail.ipynb", "title": "GPT Action Library: Gmail", "tokens": 3067, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "1e081b02c200f90ea45629eac7f23bf09e673cf3cd5ec0c75e890826b786a381", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "ca6b5a32-18c2-4ceb-a077-1a74a9fe6de1", "node_type": "1", "metadata": {}, "hash": "72b4dfadacf843e7dcde35de009645e362a2b87208f4d0c0ffe9c61210cdf72f", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 4362, "end_char_idx": 7386, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
```python
openapi: 3.1.0
info:
  title: Gmail Email API
  version: 1.0.0
  description: API to read, write, and send emails in a Gmail account.
servers:
  - url: https://gmail.googleapis.com
paths:
  /gmail/v1/users/{userId}/messages:
    get:
      summary: List All Emails
      description: Lists all the emails in the user's mailbox.
      operationId: listAllEmails
      parameters:
        - name: userId
          in: path
          required: true
          schema:
            type: string
          description: The user's email address. Use "me" to indicate the authenticated user.
        - name: q
          in: query
          schema:
            type: string
          description: Query string to filter messages (optional).
        - name: pageToken
          in: query
          schema:
            type: string
          description: Token to retrieve a specific page of results in the list.
        - name: maxResults
          in: query
          schema:
            type: integer
          description: Maximum number of messages to return.
      responses:
        '200':
          description: Successful response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/MessageList'
        '400':
          description: Bad Request
        '401':
          description: Unauthorized
        '403':
          description: Forbidden
        '404':
          description: Not Found
        '500':
          description: Internal Server Error
  /gmail/v1/users/{userId}/messages/send:
    post:
      summary: Send Email
      description: Sends a new email.
      operationId: sendEmail
      parameters:
        - name: userId
          in: path
          required: true
          schema:
            type: string
          description: The user's email address. Use "me" to indicate the authenticated user.
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/Message'
      responses:
        '200':
          description: Email sent successfully
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Message'
        '400':
          description: Bad Request
        '401':
          description: Unauthorized
        '403':
          description: Forbidden
        '500':
          description: Internal Server Error
  /gmail/v1/users/{userId}/messages/{id}:
    get:
      summary: Read Email
      description: Gets the full email content including headers and body.
      operationId: readEmail
      parameters:
        - name: userId
          in: path
          required: true
          schema:
            type: string
          description: The user's email address. Use "me" to indicate the authenticated user.
        - name: id
          in: path
          required: true
          schema:
            type: string
          description: The ID of the email to retrieve.


key: _node_content
string_value:
{"id_": "ca6b5a32-18c2-4ceb-a077-1a74a9fe6de1", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_action_gmail.ipynb", "title": "GPT Action Library: Gmail", "tokens": 3067, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "d3b081e2-0794-4104-9438-9a883cd4dfb4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_action_gmail.ipynb", "title": "GPT Action Library: Gmail", "tokens": 3067, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "4c7a0ed91cae5a94ac501e4cdb5a16d67f5df92023bce088133a45a8487d5cf6", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "0a64b888-17f2-4ab4-86e9-d741690b41d5", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_action_gmail.ipynb", "title": "GPT Action Library: Gmail", "tokens": 3067, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "aff019c9f87635cc675d4aaa6f737c51afef16298148cc3caec1254bae4b16e8", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "1e2d660c-9b1e-4cfc-ab31-1b84608b9039", "node_type": "1", "metadata": {}, "hash": "39eb672ed8cd46e46fae866c265fa5d642f06f7f0f8946f2029ea718714b917f", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 5431, "end_char_idx": 9030, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
responses:
        '200':
          description: Successful response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/MessageList'
        '400':
          description: Bad Request
        '401':
          description: Unauthorized
        '403':
          description: Forbidden
        '404':
          description: Not Found
        '500':
          description: Internal Server Error
  /gmail/v1/users/{userId}/messages/send:
    post:
      summary: Send Email
      description: Sends a new email.
      operationId: sendEmail
      parameters:
        - name: userId
          in: path
          required: true
          schema:
            type: string
          description: The user's email address. Use "me" to indicate the authenticated user.
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/Message'
      responses:
        '200':
          description: Email sent successfully
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Message'
        '400':
          description: Bad Request
        '401':
          description: Unauthorized
        '403':
          description: Forbidden
        '500':
          description: Internal Server Error
  /gmail/v1/users/{userId}/messages/{id}:
    get:
      summary: Read Email
      description: Gets the full email content including headers and body.
      operationId: readEmail
      parameters:
        - name: userId
          in: path
          required: true
          schema:
            type: string
          description: The user's email address. Use "me" to indicate the authenticated user.
        - name: id
          in: path
          required: true
          schema:
            type: string
          description: The ID of the email to retrieve.
      responses:
        '200':
          description: Successful response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/FullMessage'
        '400':
          description: Bad Request
        '401':
          description: Unauthorized
        '403':
          description: Forbidden
        '404':
          description: Not Found
        '500':
          description: Internal Server Error
  /gmail/v1/users/{userId}/messages/{id}/metadata:
    get:
      summary: Get Email Metadata
      description: Gets the metadata of an email, including subject line.
      operationId: getEmailMetadata
      parameters:
        - name: userId
          in: path
          required: true
          schema:
            type: string
          description: The user's email address. Use "me" to indicate the authenticated user.
        - name: id
          in: path
          required: true
          schema:
            type: string
          description: The ID of the email to retrieve metadata for.
      responses:
        '200':
          description: Successful response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/MessageMetadata'
        '400':
          description: Bad Request
        '401':
          description: Unauthorized
        '403':
          description: Forbidden
        '404':
          description: Not Found
        '500':
          description: Internal Server Error
  /gmail/v1/users/{userId}/drafts:
    post:
      summary: Create Draft
      description: Creates a new email draft.


key: _node_content
string_value:
{"id_": "1e2d660c-9b1e-4cfc-ab31-1b84608b9039", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_action_gmail.ipynb", "title": "GPT Action Library: Gmail", "tokens": 3067, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "d3b081e2-0794-4104-9438-9a883cd4dfb4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_action_gmail.ipynb", "title": "GPT Action Library: Gmail", "tokens": 3067, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "4c7a0ed91cae5a94ac501e4cdb5a16d67f5df92023bce088133a45a8487d5cf6", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "ca6b5a32-18c2-4ceb-a077-1a74a9fe6de1", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_action_gmail.ipynb", "title": "GPT Action Library: Gmail", "tokens": 3067, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "c13da90900e71af77aaea87fbbc785558c16b301a50816e43eb54487984a96cc", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "8bc18c25-8679-40e9-802c-4065354f2766", "node_type": "1", "metadata": {}, "hash": "ba4f6997770027cc3c3a2bb5de42b4167878f121df076e68bd44b7d0a58d5114", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 7180, "end_char_idx": 10491, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
Use "me" to indicate the authenticated user.
        - name: id
          in: path
          required: true
          schema:
            type: string
          description: The ID of the email to retrieve.
      responses:
        '200':
          description: Successful response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/FullMessage'
        '400':
          description: Bad Request
        '401':
          description: Unauthorized
        '403':
          description: Forbidden
        '404':
          description: Not Found
        '500':
          description: Internal Server Error
  /gmail/v1/users/{userId}/messages/{id}/metadata:
    get:
      summary: Get Email Metadata
      description: Gets the metadata of an email, including subject line.
      operationId: getEmailMetadata
      parameters:
        - name: userId
          in: path
          required: true
          schema:
            type: string
          description: The user's email address. Use "me" to indicate the authenticated user.
        - name: id
          in: path
          required: true
          schema:
            type: string
          description: The ID of the email to retrieve metadata for.
      responses:
        '200':
          description: Successful response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/MessageMetadata'
        '400':
          description: Bad Request
        '401':
          description: Unauthorized
        '403':
          description: Forbidden
        '404':
          description: Not Found
        '500':
          description: Internal Server Error
  /gmail/v1/users/{userId}/drafts:
    post:
      summary: Create Draft
      description: Creates a new email draft.
      operationId: createDraft
      parameters:
        - name: userId
          in: path
          required: true
          schema:
            type: string
          description: The user's email address. Use "me" to indicate the authenticated user.
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/Draft'
      responses:
        '200':
          description: Draft created successfully
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Draft'
        '400':
          description: Bad Request
        '401':
          description: Unauthorized
        '403':
          description: Forbidden
        '500':
          description: Internal Server Error
components:
  schemas:
    MessageList:
      type: object
      properties:
        messages:
          type: array
          items:
            $ref: '#/components/schemas/Message'
        nextPageToken:
          type: string
    Message:
      type: object
      properties:
        id:
          type: string
        threadId:
          type: string
        labelIds:
          type: array
          items:
            type: string
        snippet:
          type: string
        raw:
          type: string
          format: byte
          description: The entire email message in an RFC 2822 formatted and base64url encoded string.


key: _node_content
string_value:
{"id_": "8bc18c25-8679-40e9-802c-4065354f2766", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_action_gmail.ipynb", "title": "GPT Action Library: Gmail", "tokens": 3067, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "d3b081e2-0794-4104-9438-9a883cd4dfb4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_action_gmail.ipynb", "title": "GPT Action Library: Gmail", "tokens": 3067, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "4c7a0ed91cae5a94ac501e4cdb5a16d67f5df92023bce088133a45a8487d5cf6", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "1e2d660c-9b1e-4cfc-ab31-1b84608b9039", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_action_gmail.ipynb", "title": "GPT Action Library: Gmail", "tokens": 3067, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "e4231ab4130b0ad53107c7628bb2e9dbce50fec9ed0d3e82f9b4e53760463c41", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "9c24b569-462d-4871-ae74-794bb25c4da1", "node_type": "1", "metadata": {}, "hash": "a00ef10ad8a5428dd35989ccc1bdcd60af44b62ac963aa0d103ceeba14ec60ca", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 9037, "end_char_idx": 12907, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
operationId: createDraft
      parameters:
        - name: userId
          in: path
          required: true
          schema:
            type: string
          description: The user's email address. Use "me" to indicate the authenticated user.
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/Draft'
      responses:
        '200':
          description: Draft created successfully
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Draft'
        '400':
          description: Bad Request
        '401':
          description: Unauthorized
        '403':
          description: Forbidden
        '500':
          description: Internal Server Error
components:
  schemas:
    MessageList:
      type: object
      properties:
        messages:
          type: array
          items:
            $ref: '#/components/schemas/Message'
        nextPageToken:
          type: string
    Message:
      type: object
      properties:
        id:
          type: string
        threadId:
          type: string
        labelIds:
          type: array
          items:
            type: string
        snippet:
          type: string
        raw:
          type: string
          format: byte
          description: The entire email message in an RFC 2822 formatted and base64url encoded string.
    FullMessage:
      type: object
      properties:
        id:
          type: string
        threadId:
          type: string
        labelIds:
          type: array
          items:
            type: string
        snippet:
          type: string
        payload:
          type: object
          properties:
            headers:
              type: array
              items:
                type: object
                properties:
                  name:
                    type: string
                  value:
                    type: string
            parts:
              type: array
              items:
                type: object
                properties:
                  mimeType:
                    type: string
                  body:
                    type: object
                    properties:
                      data:
                        type: string
    MessageMetadata:
      type: object
      properties:
        id:
          type: string
        threadId:
          type: string
        labelIds:
          type: array
          items:
            type: string
        snippet:
          type: string
        headers:
          type: array
          items:
            type: object
            properties:
              name:
                type: string
              value:
                type: string
    EmailDraft:
      type: object
      properties:
        to:
          type: array
          items:
            type: string
        cc:
          type: array
          items:
            type: string
        bcc:
          type: array
          items:
            type: string
        subject:
          type: string
        body:
          type: object
          properties:
            mimeType:
              type: string
              enum: [text/plain, text/html]
            content:
              type: string
    Draft:
      type: object
      properties:
        id:
          type: string
        message:
          $ref: '#/components/schemas/Message'
```

## Authentication Instructions

Below are instructions on setting up authentication with this 3rd party application. Have questions? Check out [Getting Started Example](https://platform.openai.com/docs/actions/getting-started) to see how this step works in more detail.

### Pre-Action Steps

Before you set up authentication in ChatGPT, please take the following steps in the application.


key: _node_content
string_value:
{"id_": "9c24b569-462d-4871-ae74-794bb25c4da1", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_action_gmail.ipynb", "title": "GPT Action Library: Gmail", "tokens": 3067, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "d3b081e2-0794-4104-9438-9a883cd4dfb4", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_action_gmail.ipynb", "title": "GPT Action Library: Gmail", "tokens": 3067, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "4c7a0ed91cae5a94ac501e4cdb5a16d67f5df92023bce088133a45a8487d5cf6", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "8bc18c25-8679-40e9-802c-4065354f2766", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_action_gmail.ipynb", "title": "GPT Action Library: Gmail", "tokens": 3067, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "326611deef6469861efd2a34d7f31567d337be9aa58be8a350e2da39305ba00c", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 12635, "end_char_idx": 14819, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
Have questions? Check out [Getting Started Example](https://platform.openai.com/docs/actions/getting-started) to see how this step works in more detail.

### Pre-Action Steps

Before you set up authentication in ChatGPT, please take the following steps in the application.


- Go to the Google Cloud Console
- Navigate to API & Services > Credentials

![gptactions_BigQuery_auth.png](../../../images/gptactions_Gmail_enableAPIs.png)

![gptactions_BigQuery_auth.png](../../../images/gptactions_Gmail_gmailApiTile.png)

- Create new OAuth credentials (or use an existing one)

![gptactions_BigQuery_auth.png](../../../images/gptactions_Gmail_apikey.png)

- Locate your OAuth Client ID & Client Secret and store both values securely (see screenshot below)

![gptactions_BigQuery_auth.png](../../../images/gptactions_Gmail_clientidsecret.png)

### In ChatGPT

In ChatGPT, click on "Authentication" and choose **"OAuth"**. Enter in the information below. 

- **Client ID**: use Client ID from steps above 
- **Client Secret**: use Client Secret from steps above
- **Authorization URL**: https://accounts.google.com/o/oauth2/auth
- **Token URL**: https://oauth2.googleapis.com/token 
- **Scope**: https://mail.google.com/
- **Token**: Default (POST)

### Post-Action Steps

Once you've set up authentication in ChatGPT, follow the steps below in the application to finalize the Action. 

- Copy the callback URL from the GPT Action
- In the “Authorized redirect URIs” (see screenshot above), add your callback URL 


### FAQ & Troubleshooting

- *Callback URL Error:* If you get a callback URL error in ChatGPT, pay close attention to the screenshot above. You need to add the callback URL directly into GCP for the action to authenticate correctly
- *Schema calls the wrong project or dataset:* If ChatGPT calls the wrong project or dataset, consider updating your instructions to make it more explicit either (a) which project / dataset should be called or (b) to require the user provide those exact details before it runs the query

*Are there integrations that you’d like us to prioritize? Are there errors in our integrations? File a PR or issue in our github, and we’ll take a look.*


key: url
string_value:
https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_google_cloud_function.ipynb


key: _node_content
string_value:
{"id_": "72fbf432-0ade-4456-a574-fab59f875a70", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_google_cloud_function.ipynb", "title": "GPT Action Library (Middleware): Google Cloud Function", "tokens": 2073, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "b6e84051-e94a-465d-8b54-d0e7f21071d2", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_google_cloud_function.ipynb", "title": "GPT Action Library (Middleware): Google Cloud Function", "tokens": 2073, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "c6432ce6b74d1b2e9fb73034623d53f2dd62f870e48dd0a4e16ea79a2cc5f540", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "676018e1-565b-4bf9-a747-6111ca807ed8", "node_type": "1", "metadata": {}, "hash": "e4420dea39feb63efc35bec88198ad566697fbeac63c80dcff727c639be41e4e", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 0, "end_char_idx": 3555, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
# GPT Action Library (Middleware): Google Cloud Function

## Introduction



This page provides an instruction & guide for developers building middleware to connect a GPT Action to a specific application. Before you proceed, make sure to first familiarize yourself with the following information:
- [Introduction to GPT Actions](https://platform.openai.com/docs/actions)
- [Introduction to GPT Actions Library](https://platform.openai.com/docs/actions/actions-library)
- [Example of Building a GPT Action from Scratch](https://platform.openai.com/docs/actions/getting-started)

This particular GPT Action provides an overview of how to build an **Google Cloud Function**, Google's cloud-based function builder. This documentation helps a user set up an OAuth-protected Google Cloud Function to connect to a GPT Action, and to a sample application.

### Value + Example Business Use Cases

**Value**: Users can now leverage ChatGPT's natural language capability to connect directly to Google Cloud Function. This can in a few ways:

- 100k character limit in GPT Actions: users can use the middleware to pre-process the text response from an API. For example, you can use OpenAI’s API in the middleware to summarize the text before sending it back to ChatGPT.
- Typically for actions, users are relying on the SaaS API to return text. You can convert the response for the vendor API into easily digestible text, and it can handle different data types such as structured and unstructured data.
- It can return files instead of just text. This can be useful to surface CSV files for Data Analysis, or bring back an PDF file and ChatGPT will treat it like an upload.


**Example Use Cases**:
- A user needs to look up query Google Cloud SQL, but needs a middleware app between ChatGPT and Google Cloud SQL
- A user has built several steps in a row in a Google Cloud function, and needs to be able to kick off that process using ChatGPT

## Application Information

### Application Key Links

Check out these links from the application before you get started:
- Application Website: https://cloud.google.com/functions/docs
- Application API Documentation: https://cloud.google.com/functions/docs/writing/write-http-functions

### Application Prerequisites

Before you get started, make sure you go through the following steps in your application environment:
- Google Cloud Console with access to create Google Cloud Functions and Google Cloud APIs (you will need this to set up the OAuth Client)

## Application Setup

### Installing the app

There are 3 options to create and deploy the Google Cloud Functions

*   IDE - create using your favorite IDE, e.g. VS Code
*   Google Cloud Console - create using your browser
*   Google Cloud CLI (gcloud) - create through command line

You can read up on the supported runtimes [here](https://cloud.google.com/functions/docs/concepts/execution-environment)

#### Option 1: Use IDE (VSCode)

See Google's documentation [here](https://cloud.google.com/functions/docs/create-deploy-ide) for how to deploy using VSCode. If you have familiarity with this approach, feel free to use it.


#### Option 2: Directly in Google Cloud Console

See the documentation [here](https://cloud.google.com/functions/docs/console-quickstart) for how to deploy using the Google Cloud Console. 

#### Option 3: Use the Google Cloud CLI (`gcloud`)

See the documentation [here](https://cloud.google.com/functions/docs/create-deploy-gcloud) for how to deploy using the Google Cloud Console. We’ll walk through an example here step by step.


key: _node_content
string_value:
{"id_": "676018e1-565b-4bf9-a747-6111ca807ed8", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_google_cloud_function.ipynb", "title": "GPT Action Library (Middleware): Google Cloud Function", "tokens": 2073, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "b6e84051-e94a-465d-8b54-d0e7f21071d2", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_google_cloud_function.ipynb", "title": "GPT Action Library (Middleware): Google Cloud Function", "tokens": 2073, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "c6432ce6b74d1b2e9fb73034623d53f2dd62f870e48dd0a4e16ea79a2cc5f540", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "72fbf432-0ade-4456-a574-fab59f875a70", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_google_cloud_function.ipynb", "title": "GPT Action Library (Middleware): Google Cloud Function", "tokens": 2073, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "68c4801f7c456ab448818867c55e516dbebc78df8ac93bcead4b76aaa5eb022f", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "61f782d2-4867-4f3a-93cd-c0fcb25a2fa0", "node_type": "1", "metadata": {}, "hash": "098a72565de8f02af1b8a1e3086a8f0944f796af6c9edd38a083aa2d9462de5a", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 3124, "end_char_idx": 6309, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
#### Option 2: Directly in Google Cloud Console

See the documentation [here](https://cloud.google.com/functions/docs/console-quickstart) for how to deploy using the Google Cloud Console. 

#### Option 3: Use the Google Cloud CLI (`gcloud`)

See the documentation [here](https://cloud.google.com/functions/docs/create-deploy-gcloud) for how to deploy using the Google Cloud Console. We’ll walk through an example here step by step.


##### Part 1: Install and initialize Google Cloud CLI (`gcloud`)
Follow the steps [here](https://cloud.google.com/sdk/docs/install) that are relevant to the OS you are runnning. The last step of this process is for you to run `gcloud init` and sign in to your Google account

##### Part 2: Setup local development environment
In this example, we will be setting up a Node.js environment.

```
mkdir <directory_name>
cd <directory_name>
```

Initialize the Node.js project

```
npm init
```
Accept the default values for `npm init`

##### Part 3: Create Function
Create the `index.js` file

```
const functions = require('@google-cloud/functions-framework');
const axios = require('axios');

const TOKENINFO_URL = 'https://oauth2.googleapis.com/tokeninfo';

// Register an HTTP function with the Functions Framework that will be executed
// when you make an HTTP request to the deployed function's endpoint.
functions.http('executeGCPFunction', async (req, res) => {
  const authHeader = req.headers.authorization;

  if (!authHeader) {
    return res.status(401).send('Unauthorized: No token provided');
  }

  const token = authHeader.split(' ')[1];
  if (!token) {
    return res.status(401).send('Unauthorized: No token provided');
  }

  try {
    const tokenInfo = await validateAccessToken(token);            
    res.json("You have connected as an authenticated user to Google Functions");
  } catch (error) {
    res.status(401).send('Unauthorized: Invalid token');
  }  
});

async function validateAccessToken(token) {
  try {
    const response = await axios.get(TOKENINFO_URL, {
      params: {
        access_token: token,
      },
    });
    return response.data;
  } catch (error) {
    throw new Error('Invalid token');
  }
}
```
##### Part 4: Deploy Function

This step below will install and add the necessary dependencies in your `package.json` file

```
npm install @google-cloud/functions-framework
npm install axios
```

```
npx @google-cloud/functions-framework --target=executeGCPFunction
```

```
gcloud functions deploy gcp-function-for-chatgpt \
  --gen2 \
  --runtime=nodejs20 \
  --region=us-central1 \
  --source=. \
  --entry-point=executeGCPFunction \
  --trigger-http \
  --allow-unauthenticated
```

## ChatGPT Steps

### Custom GPT Instructions

Once you've created a Custom GPT, copy the text below in the Instructions panel. Have questions? Check out [Getting Started Example](https://platform.openai.com/docs/actions/getting-started) to see how this step works in more detail.

```
When the user asks you to test the integration, you will make a call to the custom action and display the results
```

### OpenAPI Schema

Once you've created a Custom GPT, copy the text below in the Actions panel. Have questions?


key: _node_content
string_value:
{"id_": "61f782d2-4867-4f3a-93cd-c0fcb25a2fa0", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_google_cloud_function.ipynb", "title": "GPT Action Library (Middleware): Google Cloud Function", "tokens": 2073, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "b6e84051-e94a-465d-8b54-d0e7f21071d2", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_google_cloud_function.ipynb", "title": "GPT Action Library (Middleware): Google Cloud Function", "tokens": 2073, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "c6432ce6b74d1b2e9fb73034623d53f2dd62f870e48dd0a4e16ea79a2cc5f540", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "676018e1-565b-4bf9-a747-6111ca807ed8", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_google_cloud_function.ipynb", "title": "GPT Action Library (Middleware): Google Cloud Function", "tokens": 2073, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "77e943f53ec26c9ce9b3b8b1dc553bb6611a8e6ef08b9c5326953514b4867354", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "b417d71b-fbea-4267-90eb-18ff2873bcb1", "node_type": "1", "metadata": {}, "hash": "eccca45ba1b916644061d13e95594267cad7ebff689ae4c0b492194108b9c259", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 5704, "end_char_idx": 8126, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
\
  --entry-point=executeGCPFunction \
  --trigger-http \
  --allow-unauthenticated
```

## ChatGPT Steps

### Custom GPT Instructions

Once you've created a Custom GPT, copy the text below in the Instructions panel. Have questions? Check out [Getting Started Example](https://platform.openai.com/docs/actions/getting-started) to see how this step works in more detail.

```
When the user asks you to test the integration, you will make a call to the custom action and display the results
```

### OpenAPI Schema

Once you've created a Custom GPT, copy the text below in the Actions panel. Have questions? Check out [Getting Started Example](https://platform.openai.com/docs/actions/getting-started) to see how this step works in more detail.

Below is an example of what connecting to this Middlware might look like. You'll need to insert your application's & function's information in this section.

```javascript
openapi: 3.1.0
info:
  title: {insert title}
  description: {insert description}
  version: 1.0.0
servers:
  - url: {url of your Google Cloud Function}
    description: {insert description}
paths:
  /{your_function_name}:
    get:
      operationId: {create an operationID}
      summary: {insert summary}
      responses:
        '200':
          description: {insert description}
          content:
            text/plain:
              schema:
                type: string
                example: {example of response}
```

## Authentication Instructions

Below are instructions on setting up authentication with this 3rd party application. Have questions? Check out [Getting Started Example](https://platform.openai.com/docs/actions/getting-started) to see how this step works in more detail.


### In Google Cloud Console
In Google Cloud Console, you need to create OAuth client ID credentials. To navigate to the right page search for "Credentials" in Google Cloud Console or enter `https://console.cloud.google.com/apis/credentials?project=<your_project_id>` in your browser. You can read more about it [here](https://developers.google.com/workspace/guides/create-credentials).

Click on "CREATE CREDENTIALS" and select "Oauth client ID". Select "Web Application" for "Application type" and enter the name of your application (see below).

![](../../../images/gcp-function-middleware-oauthclient.png)

In the "OAuth client created" modal dialog, please take note of the

* Client ID
* Client secret


key: _node_content
string_value:
{"id_": "b417d71b-fbea-4267-90eb-18ff2873bcb1", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_google_cloud_function.ipynb", "title": "GPT Action Library (Middleware): Google Cloud Function", "tokens": 2073, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "b6e84051-e94a-465d-8b54-d0e7f21071d2", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_google_cloud_function.ipynb", "title": "GPT Action Library (Middleware): Google Cloud Function", "tokens": 2073, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "c6432ce6b74d1b2e9fb73034623d53f2dd62f870e48dd0a4e16ea79a2cc5f540", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "61f782d2-4867-4f3a-93cd-c0fcb25a2fa0", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_google_cloud_function.ipynb", "title": "GPT Action Library (Middleware): Google Cloud Function", "tokens": 2073, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "9325819e52073f84a8351e180c58183bb6975634e759f753fe5e1015fa6042cc", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 6448, "end_char_idx": 9352, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
Below is an example of what connecting to this Middlware might look like. You'll need to insert your application's & function's information in this section.

```javascript
openapi: 3.1.0
info:
  title: {insert title}
  description: {insert description}
  version: 1.0.0
servers:
  - url: {url of your Google Cloud Function}
    description: {insert description}
paths:
  /{your_function_name}:
    get:
      operationId: {create an operationID}
      summary: {insert summary}
      responses:
        '200':
          description: {insert description}
          content:
            text/plain:
              schema:
                type: string
                example: {example of response}
```

## Authentication Instructions

Below are instructions on setting up authentication with this 3rd party application. Have questions? Check out [Getting Started Example](https://platform.openai.com/docs/actions/getting-started) to see how this step works in more detail.


### In Google Cloud Console
In Google Cloud Console, you need to create OAuth client ID credentials. To navigate to the right page search for "Credentials" in Google Cloud Console or enter `https://console.cloud.google.com/apis/credentials?project=<your_project_id>` in your browser. You can read more about it [here](https://developers.google.com/workspace/guides/create-credentials).

Click on "CREATE CREDENTIALS" and select "Oauth client ID". Select "Web Application" for "Application type" and enter the name of your application (see below).

![](../../../images/gcp-function-middleware-oauthclient.png)

In the "OAuth client created" modal dialog, please take note of the

* Client ID
* Client secret


### In ChatGPT (refer to Step 2 in the Getting Started Example)

In ChatGPT, click on "Authentication" and choose **"OAuth"**. Enter in the information below.

- **Client ID**: *see step above*
- **Client Secret**: *see step above*
- **Authorization URL**: `https://accounts.google.com/o/oauth2/auth`
- **Token URL**: `https://oauth2.googleapis.com/token`
- **Scope**: `https://www.googleapis.com/auth/userinfo.email`

### Back in Google Cloud Console (while referring to Step 4 in the Getting Started Example)

Edit the OAuth 2.0 Client ID you create in Google Cloud earlier and add the callback URL you received after creating your custom action.

![](../../../images/gcp-function-middleware-oauthcallback.png)

### Test the GPT

You are now ready to test out the GPT. You can enter a simple prompt like "Test Integration" and expect to see the following:

1. Request to sign into Google
2. Allow request to your Google Function
3. Response from ChatGPT showing the response from your function - e.g. "You have connected as an authenticated user to Google Functions"

*Are there integrations that you’d like us to prioritize? Are there errors in our integrations? File a PR or issue in our github, and we’ll take a look.*


key: url
string_value:
https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_azure_function.ipynb


key: _node_content
string_value:
{"id_": "32d5306b-0f77-4f34-88c8-cebd0be4599c", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_azure_function.ipynb", "title": "GPT Action Library (Middleware): Azure Function", "tokens": 2978, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7a56ad3c-a24f-4bb7-b1e9-3db7ddbe9947", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_azure_function.ipynb", "title": "GPT Action Library (Middleware): Azure Function", "tokens": 2978, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "d47990d53172b6ae9aaafd601b4733c69d4c6f7526745cf1ae404aa226856417", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "e4962d74-7daa-48f2-a05b-830b3a8f02da", "node_type": "1", "metadata": {}, "hash": "07d0ef381472c3aac902c12d8867a2bbcf9a254116916977f9b916c778251b20", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 0, "end_char_idx": 3560, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
# GPT Action Library (Middleware): Azure Function

## Introduction

This page provides an instruction & guide for developers building middleware to connect a GPT Action to a specific application. Before you proceed, make sure to first familiarize yourself with the following information: 
- [Introduction to GPT Actions](https://platform.openai.com/docs/actions)
- [Introduction to GPT Actions Library](https://platform.openai.com/docs/actions/actions-library)
- [Example of Building a GPT Action from Scratch](https://platform.openai.com/docs/actions/getting-started)

This particular GPT Action provides an overview of how to build an **Azure Function**, MSFT's cloud-based function builder. This documentation helps a user set up an OAuth-protected Azure Function to connect to a GPT Action, and to a sample application. 

### Value + Example Business Use Cases

**Value**: Users can now leverage ChatGPT's natural language capability to connect directly to Azure Function. This can in a few ways:

- 100k character limit in GPT Actions: users can use the middleware to pre-process the text response from an API. For example, you can use OpenAI’s API in the middleware to summarize the text before sending it back to ChatGPT.
- Typically for actions, users are relying on the SaaS API to return text. You can convert the response for the vendor API into easily digestible text, and it can handle different data types such as structured and unstructured data.
- It can return files instead of just text. This can be useful to surface CSV files for Data Analysis, or bring back an PDF file and ChatGPT will treat it like an upload. 


**Example Use Cases**: 
- A user needs to look up files in Sharepoint, but needs a middleware app between ChatGPT and Sharepoint
- A user has built several steps in a row in an Azure function, and needs to be able to kick off that process using ChatGPT

## Application Information

### Application Key Links

Check out these links from the application before you get started:
- Application Website: https://learn.microsoft.com/en-us/azure/azure-functions/
- Application API Documentation: https://learn.microsoft.com/en-us/azure/azure-functions/functions-reference/

### Application Prerequisites

Before you get started, make sure you go through the following steps in your application environment:
- Azure Portal with access to create Azure Function Apps and Azure Entra App Registrations

## Application Setup

### Installing the app

You can read more about languages and deployment options for Azure Functions on the left hand side of the documentation [here](https://learn.microsoft.com/en-us/azure/azure-functions/functions-overview?pivots=programming-language-csharp). 

#### Option 1: Use VSCode

See Microsoft’s documentation [here](https://learn.microsoft.com/en-us/azure/azure-functions/functions-develop-vs-code?tabs=node-v4,python-v2,isolated-process\&pivots=programming-language-javascript) for how to deploy using VSCode. If you have familiarity with this approach, feel free to use it. 


#### Option 2: Directly in Azure Portal

See the documentation [here](https://learn.microsoft.com/en-us/azure/azure-functions/functions-create-function-app-portal?pivots=programming-language-javascript) for how to deploy using the Azure portal. We’ll walk through an example here step by step.


##### Part 1: Create Function

![](../../../images/create_function_app.png)

1. Create an [Azure Function app](https://learn.microsoft.com/en-us/azure/azure-functions/functions-overview?pivots=programming-language-csharp).


key: _node_content
string_value:
{"id_": "e4962d74-7daa-48f2-a05b-830b3a8f02da", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_azure_function.ipynb", "title": "GPT Action Library (Middleware): Azure Function", "tokens": 2978, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7a56ad3c-a24f-4bb7-b1e9-3db7ddbe9947", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_azure_function.ipynb", "title": "GPT Action Library (Middleware): Azure Function", "tokens": 2978, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "d47990d53172b6ae9aaafd601b4733c69d4c6f7526745cf1ae404aa226856417", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "32d5306b-0f77-4f34-88c8-cebd0be4599c", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_azure_function.ipynb", "title": "GPT Action Library (Middleware): Azure Function", "tokens": 2978, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "24fa7c59bea4d90738fb0569f358c7dd2111dd9c3f2192cfd368bd7888fe682f", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "6db4c596-ec20-4de1-9aa2-49e995381606", "node_type": "1", "metadata": {}, "hash": "a09cb77d51b804c5871ae8a0c0d7b2226feca22395a8c3dba5fdcb506a40a03c", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 3042, "end_char_idx": 6299, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
#### Option 2: Directly in Azure Portal

See the documentation [here](https://learn.microsoft.com/en-us/azure/azure-functions/functions-create-function-app-portal?pivots=programming-language-javascript) for how to deploy using the Azure portal. We’ll walk through an example here step by step.


##### Part 1: Create Function

![](../../../images/create_function_app.png)

1. Create an [Azure Function app](https://learn.microsoft.com/en-us/azure/azure-functions/functions-overview?pivots=programming-language-csharp). I used the following settings but you can use anything you are comfortable with. Note that not every language / operating system allows for editing the functions in the console directly - the combination I chose below does. For my walkthrough, I left everything as default and made the selections below. The below settings work out of the box for the SharePoint Node.js solutions [here](https://cookbook.openai.com/examples/chatgpt/gpt_actions_library/gpt_action_sharepoint_doc) and [here](https://cookbook.openai.com/examples/chatgpt/gpt_actions_library/gpt_action_sharepoint_text). 

   1. Basics

      1. _Do you want to deploy code or container image?:_  **Code**

      2. _Runtime stack:_ **Node.js**

      3. _Operating system:_ **Windows**

   2. Networking

      1. _Enable public access_: **on (need this on to connect to the GPT)**

2. After completing the above, you’ll land on the “Deployments” page. Once the deployment completes (which should only take a few minutes) click on **“Go to Resource”** to go back to the Function App

  > You may get an error the first time you attempt this, click create again and it will likely work. 

##### Part 2: Set up Auth

3. On the left-hand side menu of the Azure Function App, click on **Authentication** under the **Settings** menu. 

   1. Add identity provider

   2. Select **Microsoft** as identity provider. 

   3. **Workforce** as tenant type

   4. **Create a new application.** The instructions are fairly similar if you are using an existing application, but it is easier to create a new application as it will have the callback URLs and the API exposed automatically using “Easy Auth”. You can read more about that [**here**](https://learn.microsoft.com/en-us/azure/app-service/overview-authentication-authorization).

   5. Leave all the other settings on this page as the default, but feel free to change based on your internal guidelines.

   6. On the **permissions** tab, click **Add Permission** and add **Files.Read.All** and **Sites.ReadAll**, then **Add.** This allows this application to read files which is important in order to use the Microsoft Graph Search API. If you are not using this for the SharePoint solution [here](https://cookbook.openai.com/examples/chatgpt/gpt_actions_library/gpt_action_sharepoint_doc) and [here](https://cookbook.openai.com/examples/chatgpt/gpt_actions_library/gpt_action_sharepoint_text) you can skip this.

4. Once it is created, **click on the enterprise application you just created** (so, leave the Function App page and land on the Enterprise Application that you just spun up)**.** We are now going to give it one more permission, to execute the Azure Function by impersonating the user logging into the application.


key: _node_content
string_value:
{"id_": "6db4c596-ec20-4de1-9aa2-49e995381606", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_azure_function.ipynb", "title": "GPT Action Library (Middleware): Azure Function", "tokens": 2978, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7a56ad3c-a24f-4bb7-b1e9-3db7ddbe9947", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_azure_function.ipynb", "title": "GPT Action Library (Middleware): Azure Function", "tokens": 2978, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "d47990d53172b6ae9aaafd601b4733c69d4c6f7526745cf1ae404aa226856417", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "e4962d74-7daa-48f2-a05b-830b3a8f02da", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_azure_function.ipynb", "title": "GPT Action Library (Middleware): Azure Function", "tokens": 2978, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "243329d3a65de4002c2fd0c3868e759df6acb632edc6bf82ab8626ade3e7e31b", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "02fcfde7-0c04-406a-8263-7460cd2d3be2", "node_type": "1", "metadata": {}, "hash": "92bcf7b9e4ec0765c5598717fcb5aee4c88a568626eacf600dc111cf73299949", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 4713, "end_char_idx": 7788, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
##### Part 2: Set up Auth

3. On the left-hand side menu of the Azure Function App, click on **Authentication** under the **Settings** menu. 

   1. Add identity provider

   2. Select **Microsoft** as identity provider. 

   3. **Workforce** as tenant type

   4. **Create a new application.** The instructions are fairly similar if you are using an existing application, but it is easier to create a new application as it will have the callback URLs and the API exposed automatically using “Easy Auth”. You can read more about that [**here**](https://learn.microsoft.com/en-us/azure/app-service/overview-authentication-authorization).

   5. Leave all the other settings on this page as the default, but feel free to change based on your internal guidelines.

   6. On the **permissions** tab, click **Add Permission** and add **Files.Read.All** and **Sites.ReadAll**, then **Add.** This allows this application to read files which is important in order to use the Microsoft Graph Search API. If you are not using this for the SharePoint solution [here](https://cookbook.openai.com/examples/chatgpt/gpt_actions_library/gpt_action_sharepoint_doc) and [here](https://cookbook.openai.com/examples/chatgpt/gpt_actions_library/gpt_action_sharepoint_text) you can skip this.

4. Once it is created, **click on the enterprise application you just created** (so, leave the Function App page and land on the Enterprise Application that you just spun up)**.** We are now going to give it one more permission, to execute the Azure Function by impersonating the user logging into the application. See [here](https://learn.microsoft.com/en-us/azure/app-service/configure-authentication-provider-aad?tabs=workforce-tenant) for more details.

   1. On the main page, click “**View API Permissions”**

   2. Search for **Microsoft Azure App Service** in the **APIs my organization uses** and find **user\_impersonation** 

   3. Add it, then you’ll need an Admin on Azure Portal to **Grant Admin Consent.**

5) **Within that enterprise application**, Click on **“Expose an API”** on the left hand menu under **Manage,** then copy the **scope** that was created using the **Copy to Clipboard** button. The scope should look like “api://\<insert-uuid>/user\_impersonation”. **Save this for later as** `SCOPE`**.**

6) Click on **“Authentication”** on the left hand menu under **Manage**

   1. Under the **Web** section, you’ll notice one callback URI was added automatically. Add the Postman redirect URI (<https://oauth.pstmn.io/v1/callback>) for testing.

7) On the left-hand side, go to **Overview**. Copy the **application (client) ID** and and the **directory (tenant) ID** and **save for later as** `CLIENT_ID` **and** `TENANT_ID`**.**

##### Part 3: Set up Test Function

8. Leave the page by going home and then back to your **Function App.**

9. Click on **Create Function.** For this example, I’m going to develop it in the portal, but you can also use VSCode or another IDE.

   1. Choose **HTTP trigger**

   2. For **Authorization Level,** you can choose any key type you want.


key: _node_content
string_value:
{"id_": "02fcfde7-0c04-406a-8263-7460cd2d3be2", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_azure_function.ipynb", "title": "GPT Action Library (Middleware): Azure Function", "tokens": 2978, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7a56ad3c-a24f-4bb7-b1e9-3db7ddbe9947", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_azure_function.ipynb", "title": "GPT Action Library (Middleware): Azure Function", "tokens": 2978, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "d47990d53172b6ae9aaafd601b4733c69d4c6f7526745cf1ae404aa226856417", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "6db4c596-ec20-4de1-9aa2-49e995381606", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_azure_function.ipynb", "title": "GPT Action Library (Middleware): Azure Function", "tokens": 2978, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "7233fca013bfa6b397cfd0d8fff7babfaefa039849096dc607529aebeda96fe0", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "e10c607f-1f89-4474-a067-34599cf2e8a7", "node_type": "1", "metadata": {}, "hash": "c47f4d62c5ae50566c3f79e698651cf395dd143b6a33fa33ef126c73eaafa630", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 6300, "end_char_idx": 9111, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
See [here](https://learn.microsoft.com/en-us/azure/app-service/configure-authentication-provider-aad?tabs=workforce-tenant) for more details.

   1. On the main page, click “**View API Permissions”**

   2. Search for **Microsoft Azure App Service** in the **APIs my organization uses** and find **user\_impersonation** 

   3. Add it, then you’ll need an Admin on Azure Portal to **Grant Admin Consent.**

5) **Within that enterprise application**, Click on **“Expose an API”** on the left hand menu under **Manage,** then copy the **scope** that was created using the **Copy to Clipboard** button. The scope should look like “api://\<insert-uuid>/user\_impersonation”. **Save this for later as** `SCOPE`**.**

6) Click on **“Authentication”** on the left hand menu under **Manage**

   1. Under the **Web** section, you’ll notice one callback URI was added automatically. Add the Postman redirect URI (<https://oauth.pstmn.io/v1/callback>) for testing.

7) On the left-hand side, go to **Overview**. Copy the **application (client) ID** and and the **directory (tenant) ID** and **save for later as** `CLIENT_ID` **and** `TENANT_ID`**.**

##### Part 3: Set up Test Function

8. Leave the page by going home and then back to your **Function App.**

9. Click on **Create Function.** For this example, I’m going to develop it in the portal, but you can also use VSCode or another IDE.

   1. Choose **HTTP trigger**

   2. For **Authorization Level,** you can choose any key type you want.

      1. Note this may error out the first time, but it is likely the Function did create, do a refresh of the page to check.

10. Click on the function you just created (You may need to click refresh to see it). Click on **Get Function URL** and save it to test in Postman. You will also use this when creating the OpenAPI spec later when you put it into the GPT. 

![](../../../images/get_function_url.png)

11. Go back to the function app and click on **Configuration.** Show the value for the `MICROSOFT_PROVIDER_AUTHENTICATION_SECRET` variable, copy it (click advanced edit to copy it), and **save it for later.**  

At this point, you should have a test function created, and you should have saved a **client id, tenant id, secret, scope, and function URL**. You are now ready to test out the authentication in Postman

##### Part 4: Test Authentication in Postman

12. Try to hit endpoint you created in Postman using those OAuth settings:

    1. **Grant Type:** Authorization Code

    2. **Auth URL**: https://login.microsoftonline.com/`TENANT_ID`/oauth2/v2.0/authorize

    3. **Auth Token URL**: https://login.microsoftonline.com/`TENANT_ID`/oauth2/v2.0/token

    4. **Client ID:** `CLIENT_ID` from step 7 above

    5. **Client secret:** `MICROSOFT_PROVIDER_AUTHENTICATION_SECRET `from step 11 above

    6.


key: _node_content
string_value:
{"id_": "e10c607f-1f89-4474-a067-34599cf2e8a7", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_azure_function.ipynb", "title": "GPT Action Library (Middleware): Azure Function", "tokens": 2978, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7a56ad3c-a24f-4bb7-b1e9-3db7ddbe9947", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_azure_function.ipynb", "title": "GPT Action Library (Middleware): Azure Function", "tokens": 2978, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "d47990d53172b6ae9aaafd601b4733c69d4c6f7526745cf1ae404aa226856417", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "02fcfde7-0c04-406a-8263-7460cd2d3be2", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_azure_function.ipynb", "title": "GPT Action Library (Middleware): Azure Function", "tokens": 2978, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "0cc9fec501e7213f97ae768597887429cbd59deb6a5d01dfee75be5945475927", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "c6b5dba0-2008-4178-bf44-0ac281804835", "node_type": "1", "metadata": {}, "hash": "6041308991761ca8874cb53a990c89a7d9034bb92959e71f02b3d170cf65225c", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 7688, "end_char_idx": 10648, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
1. Choose **HTTP trigger**

   2. For **Authorization Level,** you can choose any key type you want.

      1. Note this may error out the first time, but it is likely the Function did create, do a refresh of the page to check.

10. Click on the function you just created (You may need to click refresh to see it). Click on **Get Function URL** and save it to test in Postman. You will also use this when creating the OpenAPI spec later when you put it into the GPT. 

![](../../../images/get_function_url.png)

11. Go back to the function app and click on **Configuration.** Show the value for the `MICROSOFT_PROVIDER_AUTHENTICATION_SECRET` variable, copy it (click advanced edit to copy it), and **save it for later.**  

At this point, you should have a test function created, and you should have saved a **client id, tenant id, secret, scope, and function URL**. You are now ready to test out the authentication in Postman

##### Part 4: Test Authentication in Postman

12. Try to hit endpoint you created in Postman using those OAuth settings:

    1. **Grant Type:** Authorization Code

    2. **Auth URL**: https://login.microsoftonline.com/`TENANT_ID`/oauth2/v2.0/authorize

    3. **Auth Token URL**: https://login.microsoftonline.com/`TENANT_ID`/oauth2/v2.0/token

    4. **Client ID:** `CLIENT_ID` from step 7 above

    5. **Client secret:** `MICROSOFT_PROVIDER_AUTHENTICATION_SECRET `from step 11 above

    6. **Scope**: `SCOPE` from step 5 above

    7. **Client credentials**: Send client credentials in body

13. You will need to click **Get New Access Token**, and then hit the endpoint you saved in step 10 above. If it was successful, you should get this response: `”This HTTP triggered function executed successfully. Pass a name in the query string or in the request body for a personalized response.”`

##### Part 5: Set up your Application on an Azure Function

This should be done separately and is specific to your app. See the [Sharepoint Cookbook]((https://cookbook.openai.com/examples/chatgpt/gpt_actions_library/gpt_action_sharepoint_doc)) for an example of that. 

##### Part 6: Set up ChatGPT

14. Generate an OpenAPI spec for your endpoint. 

15. Paste that into the Actions section of a GPT, and choose OAuth as the authentication type. Fill out the OAuth settings the same way you did for Postman above. 

16. Once you save the action, you will see a callback URI at the bottom of the GPT configuration. Copy that URL, then go **back to your Function App in the Azure Portal**.

17. Click on **Authentication** under **Settings**, then click on your Entra application.

18. Once you are there, then click **Authentication** under the **Manage** section.

19. Add a new Redirect URI under the **Web** section of that page, and paste in the Callback URI you got from step 16, then click Save. 

20. Test out the GPT and it should work as expected.

## ChatGPT Steps

### Custom GPT Instructions 

*This is application specific.


key: _node_content
string_value:
{"id_": "c6b5dba0-2008-4178-bf44-0ac281804835", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_azure_function.ipynb", "title": "GPT Action Library (Middleware): Azure Function", "tokens": 2978, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7a56ad3c-a24f-4bb7-b1e9-3db7ddbe9947", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_azure_function.ipynb", "title": "GPT Action Library (Middleware): Azure Function", "tokens": 2978, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "d47990d53172b6ae9aaafd601b4733c69d4c6f7526745cf1ae404aa226856417", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "e10c607f-1f89-4474-a067-34599cf2e8a7", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_azure_function.ipynb", "title": "GPT Action Library (Middleware): Azure Function", "tokens": 2978, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "c94d38f67b258f9896c8cff8762ac8751dc310a0180b7e5e7dfef01098e2583e", "class_name": "RelatedNodeInfo"}, "3": {"node_id": "39ae4496-3b92-4713-95a7-9462ec7b28ff", "node_type": "1", "metadata": {}, "hash": "3874937e35ba6f2e76bd388e7efaf81022ee3dc004e4e10ded0f1060286d7de7", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 9112, "end_char_idx": 11193, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
**Scope**: `SCOPE` from step 5 above

    7. **Client credentials**: Send client credentials in body

13. You will need to click **Get New Access Token**, and then hit the endpoint you saved in step 10 above. If it was successful, you should get this response: `”This HTTP triggered function executed successfully. Pass a name in the query string or in the request body for a personalized response.”`

##### Part 5: Set up your Application on an Azure Function

This should be done separately and is specific to your app. See the [Sharepoint Cookbook]((https://cookbook.openai.com/examples/chatgpt/gpt_actions_library/gpt_action_sharepoint_doc)) for an example of that. 

##### Part 6: Set up ChatGPT

14. Generate an OpenAPI spec for your endpoint. 

15. Paste that into the Actions section of a GPT, and choose OAuth as the authentication type. Fill out the OAuth settings the same way you did for Postman above. 

16. Once you save the action, you will see a callback URI at the bottom of the GPT configuration. Copy that URL, then go **back to your Function App in the Azure Portal**.

17. Click on **Authentication** under **Settings**, then click on your Entra application.

18. Once you are there, then click **Authentication** under the **Manage** section.

19. Add a new Redirect URI under the **Web** section of that page, and paste in the Callback URI you got from step 16, then click Save. 

20. Test out the GPT and it should work as expected.

## ChatGPT Steps

### Custom GPT Instructions 

*This is application specific. See [Sharepoint Cookbook]((https://cookbook.openai.com/examples/chatgpt/gpt_actions_library/gpt_action_sharepoint_doc)) for an example*

### OpenAPI Schema 

Once you've created a Custom GPT, copy the text below in the Actions panel. Have questions? Check out [Getting Started Example](https://platform.openai.com/docs/actions/getting-started) to see how this step works in more detail.

Below is an example of what connecting to this Middlware might look like. You'll need to insert your application's & function's information in this section.


key: _node_content
string_value:
{"id_": "39ae4496-3b92-4713-95a7-9462ec7b28ff", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_azure_function.ipynb", "title": "GPT Action Library (Middleware): Azure Function", "tokens": 2978, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "7a56ad3c-a24f-4bb7-b1e9-3db7ddbe9947", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_azure_function.ipynb", "title": "GPT Action Library (Middleware): Azure Function", "tokens": 2978, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "d47990d53172b6ae9aaafd601b4733c69d4c6f7526745cf1ae404aa226856417", "class_name": "RelatedNodeInfo"}, "2": {"node_id": "c6b5dba0-2008-4178-bf44-0ac281804835", "node_type": "1", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/chatgpt/gpt_actions_library/gpt_middleware_azure_function.ipynb", "title": "GPT Action Library (Middleware): Azure Function", "tokens": 2978, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "03cdd672bdfd697167c5dde7a516dbe6a63fb42e285d6ab2828527e570f81897", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 9634, "end_char_idx": 12713, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
See the [Sharepoint Cookbook]((https://cookbook.openai.com/examples/chatgpt/gpt_actions_library/gpt_action_sharepoint_doc)) for an example of that. 

##### Part 6: Set up ChatGPT

14. Generate an OpenAPI spec for your endpoint. 

15. Paste that into the Actions section of a GPT, and choose OAuth as the authentication type. Fill out the OAuth settings the same way you did for Postman above. 

16. Once you save the action, you will see a callback URI at the bottom of the GPT configuration. Copy that URL, then go **back to your Function App in the Azure Portal**.

17. Click on **Authentication** under **Settings**, then click on your Entra application.

18. Once you are there, then click **Authentication** under the **Manage** section.

19. Add a new Redirect URI under the **Web** section of that page, and paste in the Callback URI you got from step 16, then click Save. 

20. Test out the GPT and it should work as expected.

## ChatGPT Steps

### Custom GPT Instructions 

*This is application specific. See [Sharepoint Cookbook]((https://cookbook.openai.com/examples/chatgpt/gpt_actions_library/gpt_action_sharepoint_doc)) for an example*

### OpenAPI Schema 

Once you've created a Custom GPT, copy the text below in the Actions panel. Have questions? Check out [Getting Started Example](https://platform.openai.com/docs/actions/getting-started) to see how this step works in more detail.

Below is an example of what connecting to this Middlware might look like. You'll need to insert your application's & function's information in this section. 


```python
openapi: 3.1.0
info:
  title: {insert title}
  description: {insert description}
  version: 1.0.0
servers:
  - url: https://{your_function_app_name}.azurewebsites.net/api
    description: {insert description}
paths:
  /{your_function_name}?code={enter your specific endpoint id here}:
    post:
      operationId: {insert operationID}
      summary: {insert summary}
      requestBody: 
{the rest of this is specific to your application}
```

## Authentication Instructions

Below are instructions on setting up authentication with this 3rd party application. Have questions? Check out [Getting Started Example](https://platform.openai.com/docs/actions/getting-started) to see how this step works in more detail.

### Pre-Action Steps

Before you set up authentication in ChatGPT, please take the following steps in the application.

*Follow steps 2 & 4 above to setting up authentication*

### In ChatGPT

In ChatGPT, click on "Authentication" and choose **"OAuth"**. Enter in the information below. 

- **Client ID**: *see step 12 above*
- **Client Secret**: *ditto*
- **Authorization URL**: *ditto*
- **Token URL**: *ditto*
- **Scope**: *ditto*
- **Token**: *ditto*

### Post-Action Steps

Once you've set up authentication in ChatGPT, follow the steps below in the application to finalize the Action. 

*See above for testing out this application*

*Are there integrations that you’d like us to prioritize? Are there errors in our integrations? File a PR or issue in our github, and we’ll take a look.*


key: url
string_value:
https://github.com/openai/openai-cookbook/blob/main/examples/data/structured_outputs_articles/llms.ipynb


key: _node_content
string_value:
{"id_": "f10f5af5-02bd-4598-ac54-8bbc92a7c25c", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/data/structured_outputs_articles/llms.ipynb", "title": "### Large Language Models (LLMs)", "tokens": 721, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "075b62b3-fb59-4d2e-9674-036ea9b8e9ba", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/data/structured_outputs_articles/llms.ipynb", "title": "### Large Language Models (LLMs)", "tokens": 721, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "2e805eb49bd35b90bfcf0b5b2084234f400aa5e598146ff9b41f517aa8f75e8c", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 0, "end_char_idx": 3886, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
### Large Language Models (LLMs)

Large Language Models (LLMs) are a type of artificial intelligence model designed to understand and generate human language. The development of LLMs has been driven by advances in deep learning and natural language processing. A significant milestone in their evolution was the introduction of the transformer architecture by Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin in 2017. LLMs have significantly advanced the fields of natural language processing (NLP) and understanding, enabling applications such as machine translation, text summarization, and conversational agents.

#### Architecture

LLMs are typically based on transformer architecture, which allows them to process and generate text in a highly parallelized manner. Key components of LLM architecture include:

- **Embeddings:** Input text is converted into a continuous vector space using embedding layers. This step transforms discrete words or subwords into numerical representations that capture semantic relationships.

- **Transformer Blocks:** LLMs consist of multiple stacked transformer blocks. Each block contains self-attention mechanisms and feed-forward neural networks. The self-attention mechanism allows the model to weigh the importance of different words in a context, capturing long-range dependencies and relationships within the text.

- **Attention Mechanisms:** Attention mechanisms enable the model to focus on relevant parts of the input text when generating output. This is crucial for tasks like translation, where the model needs to align source and target language elements accurately.

- **Decoder:** In generative models, a decoder is used to generate text from the encoded representations. The decoder uses masked self-attention to ensure that predictions for each word depend only on previously generated words.

#### Training

Training LLMs involves pre-training and fine-tuning stages:

- **Pre-training:** During this phase, the model is trained on a large corpus of text data using unsupervised learning objectives, such as predicting masked words or the next word in a sequence. This helps the model learn language patterns, grammar, and context.

- **Fine-tuning:** After pre-training, the model is fine-tuned on specific tasks using supervised learning. This stage involves training the model on labeled datasets to perform tasks like sentiment analysis, question answering, or text classification.

#### Applications

LLMs have a wide range of applications across various domains:

- **Text Generation:** LLMs can generate coherent and contextually relevant text, useful for creative writing, content creation, and dialogue generation in chatbots.

- **Machine Translation:** LLMs power modern translation systems, providing accurate translations between multiple languages by understanding the nuances and context of the source text.

- **Summarization:** LLMs can condense long documents into concise summaries, aiding in information retrieval and reducing the time required to understand large volumes of text.

- **Sentiment Analysis:** LLMs can analyze text to determine the sentiment expressed, valuable for market analysis, customer feedback, and social media monitoring.

- **Conversational Agents:** LLMs enable the development of advanced chatbots and virtual assistants that can understand and respond to user queries naturally and contextually.

Overall, LLMs have transformed the field of NLP, enabling more sophisticated and human-like interactions between machines and users, and continue to evolve with ongoing research and technological advancements. The introduction of the transformer architecture by Vaswani et al. has been instrumental in this transformation, providing a foundation for the development of increasingly powerful language models.


key: url
string_value:
https://github.com/openai/openai-cookbook/blob/main/examples/data/structured_outputs_articles/cnns.ipynb


key: _node_content
string_value:
{"id_": "534eee51-b204-4bc5-b1eb-ab351d6f9211", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/data/structured_outputs_articles/cnns.ipynb", "title": "### Convolutional Neural Networks (CNNs)", "tokens": 567, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "f14b9300-c6e1-4f2e-9e39-29bf188b7f98", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/data/structured_outputs_articles/cnns.ipynb", "title": "### Convolutional Neural Networks (CNNs)", "tokens": 567, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "1025ec27b0c3f34ef0b3229b6e8b725905d29acc84336a1b426e348c5df5d3ee", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 0, "end_char_idx": 3076, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
### Convolutional Neural Networks (CNNs)

Convolutional Neural Networks (CNNs) are a class of deep neural networks primarily used for processing structured grid data like images. They were invented by Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner in 1989. CNNs have revolutionized the field of computer vision, enabling advancements in areas such as image classification, object detection, and image segmentation.

#### Architecture

A typical CNN architecture consists of multiple layers, including convolutional layers, pooling layers, and fully connected layers.

- **Convolutional Layers:** These layers apply a set of learnable filters (kernels) to the input data. Each filter convolves across the input data, producing a feature map that detects specific features such as edges, textures, and patterns. The parameters of these filters are learned during training.

- **Pooling Layers:** Also known as subsampling or downsampling layers, pooling layers reduce the spatial dimensions of the feature maps. The most common type is max pooling, which selects the maximum value from each region of the feature map, thereby reducing its size and computation requirements while retaining important features.

- **Fully Connected Layers:** After several convolutional and pooling layers, the output is flattened and fed into one or more fully connected layers, which perform the final classification or regression task. These layers resemble those in traditional neural networks, connecting every neuron in one layer to every neuron in the next.

#### Training

CNNs are trained using backpropagation and gradient descent. During training, the network learns the optimal filter values that minimize the loss function, typically a measure of the difference between the predicted and actual labels. The training process involves adjusting the weights of the filters and the fully connected layers through iterative updates.

#### Applications

CNNs have become the cornerstone of many state-of-the-art systems in computer vision. Some notable applications include:

- **Image Classification:** CNNs can classify images into various categories with high accuracy. They have been used in systems like Google Photos and Facebook's automatic tagging.

- **Object Detection:** CNNs can detect and localize objects within an image, which is essential for tasks like autonomous driving and facial recognition.

- **Medical Image Analysis:** CNNs assist in diagnosing diseases by analyzing medical images such as X-rays, MRIs, and CT scans.

- **Image Segmentation:** CNNs are used to partition an image into meaningful segments, useful in applications such as scene understanding and medical image analysis.

Overall, CNNs have significantly advanced the field of artificial intelligence, particularly in tasks that involve visual data, and continue to be an area of active research and development. The pioneering work of LeCun and his colleagues laid the foundation for these transformative technologies, which have since become integral to modern computer vision systems.


key: url
string_value:
https://github.com/openai/openai-cookbook/blob/main/examples/data/structured_outputs_articles/moe.ipynb


key: _node_content
string_value:
{"id_": "1e64bc16-c683-4401-b0f9-6c941e87598e", "embedding": null, "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/data/structured_outputs_articles/moe.ipynb", "title": "### Mixture of Experts (MoE)", "tokens": 593, "retrieve_doc": true, "source": "openai_cookbooks"}, "excluded_embed_metadata_keys": ["url", "tokens", "retrieve_doc", "source"], "excluded_llm_metadata_keys": ["title", "tokens", "retrieve_doc", "source"], "relationships": {"1": {"node_id": "201c73ad-10aa-4e9d-b25c-3eec2013a5f8", "node_type": "4", "metadata": {"url": "https://github.com/openai/openai-cookbook/blob/main/examples/data/structured_outputs_articles/moe.ipynb", "title": "### Mixture of Experts (MoE)", "tokens": 593, "retrieve_doc": true, "source": "openai_cookbooks"}, "hash": "a6ac7518aaa91b81ce8c7cf81fbec4109ba55ccda3900dff3a8fd6fd82f6d2c4", "class_name": "RelatedNodeInfo"}}, "text": "", "mimetype": "text/plain", "start_char_idx": 0, "end_char_idx": 3331, "text_template": "{metadata_str}\n\n{content}", "metadata_template": "{key}: {value}", "metadata_seperator": "\n", "class_name": "TextNode"}


key: chroma:document
string_value:
### Mixture of Experts (MoE)

Mixture of Experts (MoE) is a machine learning technique designed to enhance model performance by combining the predictions of multiple specialized models, or "experts." The concept was introduced by Michael I. Jordan and Robert A. Jacobs in 1991. MoE models have been applied in various fields, including natural language processing, computer vision, and speech recognition, to improve accuracy and efficiency.

#### Architecture

A typical MoE architecture consists of several key components:

- **Experts:** These are individual models, each trained to specialize in different parts of the input space or specific aspects of the task. Each expert might be a neural network trained to focus on particular features or patterns within the data.

- **Gating Network:** The gating network is responsible for dynamically selecting which experts should be consulted for a given input. It assigns weights to each expert's output, determining their contribution to the final prediction. The gating network typically uses a softmax function to produce these weights, ensuring they sum to one.

- **Combiner:** The combiner aggregates the outputs from the selected experts, weighted by the gating network. This combination can be a weighted sum or another aggregation method, producing the final output of the MoE model.

#### Training

Training an MoE model involves two main stages:

- **Expert Training:** Each expert model is trained on a subset of the data or a specific aspect of the task. This training can be done independently, with each expert focusing on maximizing performance within its specialized domain.

- **Gating Network Training:** The gating network is trained to learn the optimal combination of experts for different inputs. It uses the loss from the final combined output to adjust its parameters, learning which experts are most relevant for various parts of the input space.

#### Applications

MoE models have a wide range of applications across different domains:

- **Natural Language Processing:** In tasks like machine translation and language modeling, MoE models can dynamically allocate computational resources, allowing different experts to handle different linguistic features or contexts.

- **Computer Vision:** MoE models can be used for image classification and object detection, where different experts specialize in recognizing specific types of objects or features within images.

- **Speech Recognition:** In speech recognition systems, MoE models can improve accuracy by assigning different experts to handle variations in speech, such as accents, intonations, or background noise.

- **Recommendation Systems:** MoE models can enhance recommendation engines by using different experts to analyze various aspects of user behavior and preferences, providing more personalized recommendations.

Overall, Mixture of Experts models offer a powerful framework for improving the performance and efficiency of machine learning systems. By leveraging specialized experts and dynamically selecting the most relevant ones for each input, MoE models can achieve superior results in a variety of complex tasks. The pioneering work of Jordan and Jacobs laid the foundation for this innovative approach, which continues to evolve and find new applications in modern AI research.


