prompt: We are currently solving the following issue within our repository. Here is the issue text:
  --- BEGIN ISSUE ---
  [BUG] Getting error when loading the TF4Rec PyTorch model  to the TIS
  **Describe the bug**
  I am getting the following error when I load a trained TF4Rec PyTorch to TIS:
  
  ```
   | t4r_pytorch_pt  | 1       | UNAVAILABLE: Internal: ImportError: cannot import name '_convert_string2pytorch_dty |
  |                 |         | pe' from 'nvtabular.inference.triton' (/nvtabular/nvtabular/inference/triton/__init |
  |                 |         | __.py)                                                                              |
  |                 |         |                                                                                     |
  |                 |         | At:                                                                                 |
  |                 |         |   /workspace/models/t4r_pytorch_pt/1/model.py(42): <module>                         |
  |                 |         |   <frozen importlib._bootstrap>(219): _call_with_frames_removed                     |
  |                 |         |   <frozen importlib._bootstrap_external>(848): exec_module                          |
  |                 |         |   <frozen importlib._bootstrap>(686): _load_unlocked                                |
  |                 |         |   <frozen importlib._bootstrap>(975): _find_and_load_unlocked                       |
  |                 |         |   <frozen importlib._bootstrap>(991): _find_and_load                                |
  +-----------------+---------+---------------------------------------------------------
  ```
  
  **Steps/Code to reproduce bug**
  
  Run the 02 and 03 notebooks Transformers4Rec tutorial [notebooks](https://github.com/NVIDIA-Merlin/Transformers4Rec/tree/main/examples/tutorial) to train the model. Then serve the model to TIS based on the instructions  given on the [inference notebook](https://github.com/NVIDIA-Merlin/Transformers4Rec/blob/main/examples/tutorial/04-Inference-with-Triton.ipynb).
  
  `Oct-2019.parquet` Dataset can be downloaded from here: https://drive.google.com/drive/u/0/folders/1nTuG6UHWOEaZnBJj7YSIVvnphE1zGc1h
  
  **Expected behavior**
  Model should be loaded to the TIS without issue.
  
  **Environment details (please complete the following information):**
   - Environment location: [Bare-metal, Docker, Cloud(specify cloud provider)] : Docker
   - Method of NVTabular install: [conda, Docker, or from source]: Docker `merlin-inference:21.11` and `merlin-pytoch-training:21.11` `
   Please do  `git pull origin main` && `pip install -e .` to pull the latest main branch.
     - If method of install is [Docker], provide `docker pull` & `docker run` commands used
   
  This issue was also submitted by a user on TF4Rec GH repo- https://github.com/NVIDIA-Merlin/Transformers4Rec/issues/339
  
  
  --- END ISSUE ---
  
  Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
  --- BEGIN FILES ---
  Path: `nvtabular/inference/triton/__init__.py`
  Content:
  ```
  1 # Copyright (c) 2021, NVIDIA CORPORATION.
  2 #
  3 # Licensed under the Apache License, Version 2.0 (the "License");
  4 # you may not use this file except in compliance with the License.
  5 # You may obtain a copy of the License at
  6 #
  7 #     http://www.apache.org/licenses/LICENSE-2.0
  8 #
  9 # Unless required by applicable law or agreed to in writing, software
  10 # distributed under the License is distributed on an "AS IS" BASIS,
  11 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  12 # See the License for the specific language governing permissions and
  13 # limitations under the License.
  14 #
  15 import json
  16 import os
  17 
  18 import pandas as pd
  19 
  20 # this needs to be before any modules that import protobuf
  21 os.environ["PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION"] = "python"
  22 
  23 import tritonclient.grpc as grpcclient  # noqa
  24 from tritonclient.utils import np_to_triton_dtype  # noqa
  25 
  26 from nvtabular.dispatch import _is_list_dtype, _is_string_dtype, _make_df  # noqa
  27 from nvtabular.inference.triton.ensemble import (  # noqa
  28     export_hugectr_ensemble,
  29     export_pytorch_ensemble,
  30     export_tensorflow_ensemble,
  31     generate_hugectr_model,
  32     generate_nvtabular_model,
  33 )
  34 
  35 
  36 def convert_df_to_triton_input(column_names, batch, input_class=grpcclient.InferInput):
  37     columns = [(col, batch[col]) for col in column_names]
  38     inputs = []
  39     for i, (name, col) in enumerate(columns):
  40         if _is_list_dtype(col):
  41             if isinstance(col, pd.Series):
  42                 raise ValueError("this function doesn't support CPU list values yet")
  43             inputs.append(
  44                 _convert_column_to_triton_input(
  45                     col._column.offsets.values_host.astype("int64"), name + "__nnzs", input_class
  46                 )
  47             )
  48             inputs.append(
  49                 _convert_column_to_triton_input(
  50                     col.list.leaves.values_host.astype("int64"), name + "__values", input_class
  51                 )
  52             )
  53         else:
  54             values = col.values if isinstance(col, pd.Series) else col.values_host
  55             inputs.append(_convert_column_to_triton_input(values, name, input_class))
  56     return inputs
  57 
  58 
  59 def _convert_column_to_triton_input(col, name, input_class=grpcclient.InferInput):
  60     col = col.reshape(len(col), 1)
  61     input_tensor = input_class(name, col.shape, np_to_triton_dtype(col.dtype))
  62     input_tensor.set_data_from_numpy(col)
  63     return input_tensor
  64 
  65 
  66 def convert_triton_output_to_df(columns, response):
  67     return _make_df({col: response.as_numpy(col) for col in columns})
  68 
  69 
  70 def get_column_types(path):
  71     return json.load(open(os.path.join(path, "column_types.json")))
  72 
  73 
  74 def _convert_tensor(t):
  75     out = t.as_numpy()
  76     if len(out.shape) == 2:
  77         out = out[:, 0]
  78     # cudf doesn't seem to handle dtypes like |S15 or object that well
  79     if _is_string_dtype(out.dtype):
  80         out = out.astype("str")
  81     return out
  82
  ```
  --- END FILES ---
  
  Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.
  
  Here is an example:
  
  ```diff
  diff --git a/examples/server_async.py b/examples/server_async.py
  --- a/examples/server_async.py
  +++ b/examples/server_async.py
  @@ -313,4 +313,4 @@
   
   
   if __name__ == "__main__":
  -    asyncio.run(run_async_server("."), debug=True)
  +    asyncio.run(run_async_server(), debug=True)
  diff --git a/examples/server_sync.py b/examples/server_sync.py
  --- a/examples/server_sync.py
  +++ b/examples/server_sync.py
  @@ -313,5 +313,5 @@
   
   
   if __name__ == "__main__":
  -    server = run_sync_server(".")
  +    server = run_sync_server()
       server.shutdown()
  
  ```
  
golden_diff: diff --git a/nvtabular/inference/triton/__init__.py b/nvtabular/inference/triton/__init__.py
  --- a/nvtabular/inference/triton/__init__.py
  +++ b/nvtabular/inference/triton/__init__.py
  @@ -25,6 +25,7 @@
   
   from nvtabular.dispatch import _is_list_dtype, _is_string_dtype, _make_df  # noqa
   from nvtabular.inference.triton.ensemble import (  # noqa
  +    _convert_string2pytorch_dtype,
       export_hugectr_ensemble,
       export_pytorch_ensemble,
       export_tensorflow_ensemble,
  
verification_info: {"golden_diff": "diff --git a/nvtabular/inference/triton/__init__.py b/nvtabular/inference/triton/__init__.py\n--- a/nvtabular/inference/triton/__init__.py\n+++ b/nvtabular/inference/triton/__init__.py\n@@ -25,6 +25,7 @@\n \n from nvtabular.dispatch import _is_list_dtype, _is_string_dtype, _make_df  # noqa\n from nvtabular.inference.triton.ensemble import (  # noqa\n+    _convert_string2pytorch_dtype,\n     export_hugectr_ensemble,\n     export_pytorch_ensemble,\n     export_tensorflow_ensemble,\n", "issue": "[BUG] Getting error when loading the TF4Rec PyTorch model  to the TIS\n**Describe the bug**\r\nI am getting the following error when I load a trained TF4Rec PyTorch to TIS:\r\n\r\n```\r\n | t4r_pytorch_pt  | 1       | UNAVAILABLE: Internal: ImportError: cannot import name '_convert_string2pytorch_dty |\r\n|                 |         | pe' from 'nvtabular.inference.triton' (/nvtabular/nvtabular/inference/triton/__init |\r\n|                 |         | __.py)                                                                              |\r\n|                 |         |                                                                                     |\r\n|                 |         | At:                                                                                 |\r\n|                 |         |   /workspace/models/t4r_pytorch_pt/1/model.py(42): <module>                         |\r\n|                 |         |   <frozen importlib._bootstrap>(219): _call_with_frames_removed                     |\r\n|                 |         |   <frozen importlib._bootstrap_external>(848): exec_module                          |\r\n|                 |         |   <frozen importlib._bootstrap>(686): _load_unlocked                                |\r\n|                 |         |   <frozen importlib._bootstrap>(975): _find_and_load_unlocked                       |\r\n|                 |         |   <frozen importlib._bootstrap>(991): _find_and_load                                |\r\n+-----------------+---------+---------------------------------------------------------\r\n```\r\n\r\n**Steps/Code to reproduce bug**\r\n\r\nRun the 02 and 03 notebooks Transformers4Rec tutorial [notebooks](https://github.com/NVIDIA-Merlin/Transformers4Rec/tree/main/examples/tutorial) to train the model. Then serve the model to TIS based on the instructions  given on the [inference notebook](https://github.com/NVIDIA-Merlin/Transformers4Rec/blob/main/examples/tutorial/04-Inference-with-Triton.ipynb).\r\n\r\n`Oct-2019.parquet` Dataset can be downloaded from here: https://drive.google.com/drive/u/0/folders/1nTuG6UHWOEaZnBJj7YSIVvnphE1zGc1h\r\n\r\n**Expected behavior**\r\nModel should be loaded to the TIS without issue.\r\n\r\n**Environment details (please complete the following information):**\r\n - Environment location: [Bare-metal, Docker, Cloud(specify cloud provider)] : Docker\r\n - Method of NVTabular install: [conda, Docker, or from source]: Docker `merlin-inference:21.11` and `merlin-pytoch-training:21.11` `\r\n Please do  `git pull origin main` && `pip install -e .` to pull the latest main branch.\r\n   - If method of install is [Docker], provide `docker pull` & `docker run` commands used\r\n \r\nThis issue was also submitted by a user on TF4Rec GH repo- https://github.com/NVIDIA-Merlin/Transformers4Rec/issues/339\r\n\n", "before_files": [{"content": "# Copyright (c) 2021, NVIDIA CORPORATION.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n#\nimport json\nimport os\n\nimport pandas as pd\n\n# this needs to be before any modules that import protobuf\nos.environ[\"PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION\"] = \"python\"\n\nimport tritonclient.grpc as grpcclient  # noqa\nfrom tritonclient.utils import np_to_triton_dtype  # noqa\n\nfrom nvtabular.dispatch import _is_list_dtype, _is_string_dtype, _make_df  # noqa\nfrom nvtabular.inference.triton.ensemble import (  # noqa\n    export_hugectr_ensemble,\n    export_pytorch_ensemble,\n    export_tensorflow_ensemble,\n    generate_hugectr_model,\n    generate_nvtabular_model,\n)\n\n\ndef convert_df_to_triton_input(column_names, batch, input_class=grpcclient.InferInput):\n    columns = [(col, batch[col]) for col in column_names]\n    inputs = []\n    for i, (name, col) in enumerate(columns):\n        if _is_list_dtype(col):\n            if isinstance(col, pd.Series):\n                raise ValueError(\"this function doesn't support CPU list values yet\")\n            inputs.append(\n                _convert_column_to_triton_input(\n                    col._column.offsets.values_host.astype(\"int64\"), name + \"__nnzs\", input_class\n                )\n            )\n            inputs.append(\n                _convert_column_to_triton_input(\n                    col.list.leaves.values_host.astype(\"int64\"), name + \"__values\", input_class\n                )\n            )\n        else:\n            values = col.values if isinstance(col, pd.Series) else col.values_host\n            inputs.append(_convert_column_to_triton_input(values, name, input_class))\n    return inputs\n\n\ndef _convert_column_to_triton_input(col, name, input_class=grpcclient.InferInput):\n    col = col.reshape(len(col), 1)\n    input_tensor = input_class(name, col.shape, np_to_triton_dtype(col.dtype))\n    input_tensor.set_data_from_numpy(col)\n    return input_tensor\n\n\ndef convert_triton_output_to_df(columns, response):\n    return _make_df({col: response.as_numpy(col) for col in columns})\n\n\ndef get_column_types(path):\n    return json.load(open(os.path.join(path, \"column_types.json\")))\n\n\ndef _convert_tensor(t):\n    out = t.as_numpy()\n    if len(out.shape) == 2:\n        out = out[:, 0]\n    # cudf doesn't seem to handle dtypes like |S15 or object that well\n    if _is_string_dtype(out.dtype):\n        out = out.astype(\"str\")\n    return out\n", "path": "nvtabular/inference/triton/__init__.py"}], "after_files": [{"content": "# Copyright (c) 2021, NVIDIA CORPORATION.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n#\nimport json\nimport os\n\nimport pandas as pd\n\n# this needs to be before any modules that import protobuf\nos.environ[\"PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION\"] = \"python\"\n\nimport tritonclient.grpc as grpcclient  # noqa\nfrom tritonclient.utils import np_to_triton_dtype  # noqa\n\nfrom nvtabular.dispatch import _is_list_dtype, _is_string_dtype, _make_df  # noqa\nfrom nvtabular.inference.triton.ensemble import (  # noqa\n    _convert_string2pytorch_dtype,\n    export_hugectr_ensemble,\n    export_pytorch_ensemble,\n    export_tensorflow_ensemble,\n    generate_hugectr_model,\n    generate_nvtabular_model,\n)\n\n\ndef convert_df_to_triton_input(column_names, batch, input_class=grpcclient.InferInput):\n    columns = [(col, batch[col]) for col in column_names]\n    inputs = []\n    for i, (name, col) in enumerate(columns):\n        if _is_list_dtype(col):\n            if isinstance(col, pd.Series):\n                raise ValueError(\"this function doesn't support CPU list values yet\")\n            inputs.append(\n                _convert_column_to_triton_input(\n                    col._column.offsets.values_host.astype(\"int64\"), name + \"__nnzs\", input_class\n                )\n            )\n            inputs.append(\n                _convert_column_to_triton_input(\n                    col.list.leaves.values_host.astype(\"int64\"), name + \"__values\", input_class\n                )\n            )\n        else:\n            values = col.values if isinstance(col, pd.Series) else col.values_host\n            inputs.append(_convert_column_to_triton_input(values, name, input_class))\n    return inputs\n\n\ndef _convert_column_to_triton_input(col, name, input_class=grpcclient.InferInput):\n    col = col.reshape(len(col), 1)\n    input_tensor = input_class(name, col.shape, np_to_triton_dtype(col.dtype))\n    input_tensor.set_data_from_numpy(col)\n    return input_tensor\n\n\ndef convert_triton_output_to_df(columns, response):\n    return _make_df({col: response.as_numpy(col) for col in columns})\n\n\ndef get_column_types(path):\n    return json.load(open(os.path.join(path, \"column_types.json\")))\n\n\ndef _convert_tensor(t):\n    out = t.as_numpy()\n    if len(out.shape) == 2:\n        out = out[:, 0]\n    # cudf doesn't seem to handle dtypes like |S15 or object that well\n    if _is_string_dtype(out.dtype):\n        out = out.astype(\"str\")\n    return out\n", "path": "nvtabular/inference/triton/__init__.py"}]}