code: def _get_unique_invidx(srcids, dstids, nids, low_mem=True):
      """This function is used to compute a list of unique elements,
      and their indices in the input list, which is the concatenation
      of srcids, dstids and uniq_nids. In addition, this function will also
      compute inverse indices, in the list of unique elements, for the
      elements in srcids, dstids and nids arrays. srcids, dstids will be
      over-written to contain the inverse indices. Basically, this function
      is mimicing the functionality of numpy's unique function call.
      The problem with numpy's unique function call is its high memory
      requirement. For an input list of 3 billion edges it consumes about
      550GB of systems memory, which is limiting the capability of the
      partitioning pipeline.
  
      Note: This function is a workaround solution for the high memory requirement
      of numpy's unique function call. This function is not a general purpose
      function and is only used in the context of the partitioning pipeline.
      What's more, this function does not behave exactly the same as numpy's
      unique function call. Namely, this function does not return the exact same
      inverse indices as numpy's unique function call. However, for the current
      use case, this function is sufficient.
  
      Current numpy uniques function returns 3 return parameters, which are
          . list of unique elements
          . list of indices, in the input argument list, which are first
              occurance of the corresponding element in the uniques list
          . list of inverse indices, which are indices from the uniques list
              and can be used to rebuild the original input array
      Compared to the above numpy's return parameters, this work around
      solution returns 4 values
          . list of unique elements,
          . list of indices, which may not be the first occurance of the
              corresponding element from the uniques
          . list of inverse indices, here we only build the inverse indices
              for srcids and dstids input arguments. For the current use case,
              only these two inverse indices are needed.
  
      Parameters:
      -----------
      srcids : numpy array
          a list of numbers, which are the src-ids of the edges
      dstids : numpy array
          a list of numbers, which are the dst-ids of the edges
      nids : numpy array
          a list of numbers, a list of unique shuffle-global-nids.
          This list is guaranteed to be a list of sorted consecutive unique
          list of numbers. Also, this list will be a `super set` for the
          list of dstids. Current implementation of the pipeline guarantees
          this assumption and is used to simplify the current implementation
          of the workaround solution.
      low_mem : bool, optional
          Indicates whether to use the low memory version of the function. If
          ``False``, the function will use numpy's native ``unique`` function.
          Otherwise, the function will use the low memory version of the
          function.
  
      Returns:
      --------
      numpy array :
          a list of unique, sorted elements, computed from the input arguments
      numpy array :
          a list of integers. These are indices in the concatenated list
          [srcids, dstids, uniq_nids], which are the input arguments to this function
      numpy array :
          a list of integers. These are inverse indices, which will be indices
          from the unique elements list specifying the elements from the
          input array, srcids
      numpy array :
          a list of integers. These are inverse indices, which will be indices
          from the unique elements list specifying the elements from the
          input array, dstids
      """
      assert len(srcids) == len(
          dstids
      ), f"Please provide the correct input parameters"
      assert len(srcids) != 0, f"Please provide a non-empty edge-list."
  
      if not low_mem:
          logging.warning(
              "Calling numpy's native function unique. This functions memory "
              "overhead will limit size of the partitioned graph objects "
              "processed by each node in the cluster."
          )
          uniques, idxes, inv_idxes = np.unique(
              np.concatenate([srcids, dstids, nids]),
              return_index=True,
              return_inverse=True,
          )
          src_len = len(srcids)
          dst_len = len(dstids)
          return (
              uniques,
              idxes,
              inv_idxes[:src_len],
              inv_idxes[src_len : (src_len + dst_len)],
          )
  
      # find uniqes which appear only in the srcids list
      mask = np.isin(srcids, nids, invert=True, kind="table")
      srcids_only = srcids[mask]
      srcids_idxes = np.where(mask == 1)[0]
  
      # sort
      uniques, unique_srcids_idx = np.unique(
          srcids_only, return_index=True
      )
      idxes = srcids_idxes[unique_srcids_idx]
  
      # build uniques and idxes, first and second return parameters
      uniques = np.concatenate([uniques, nids])
      idxes = np.concatenate(
          [idxes, len(srcids) + len(dstids) + np.arange(len(nids))]
      )
  
      # sort and idxes
      sort_idx = np.argsort(uniques)
      uniques = uniques[sort_idx]
      idxes = idxes[sort_idx]
  
      # uniques and idxes are built
      assert len(uniques) == len(
          idxes
      ), f"Error building the idxes array."
  
      srcids = np.searchsorted(uniques, srcids, side="left")
  
      # process dstids now.
      # dstids is guaranteed to be a subset of the `nids` list
      # here we are computing index in the list of uniqes for
      # each element in the list of dstids, in a two step process
      # 1. locate the position of first element from nids in the
      #       list of uniques - dstids cannot appear to the left
      #       of this number, they are guaranteed to be on the right
      #       side of this number.
      # 2. dstids = dstids - nids[0]
      #       By subtracting nids[0] from the list of dstids will make
      #       the list of dstids to be in the range of [0, max(nids)-1]
      # 3. dstids = dstids - nids[0] + offset
      #       Now we move the list of dstids by `offset` which will be
      #       the starting position of the nids[0] element. Note that
      #       nids will ALWAYS be a SUPERSET of dstids.
      offset = np.searchsorted(uniques, nids[0], side="left")
      dstids = dstids - nids[0] + offset
  
      # return the values
      return uniques, idxes, srcids, dstids
  
file_path: dgl/tools/distpartitioning/convert_partition.py
imports: import argparse
  import gc
  import json
  import logging
  import os
  import time
  import constants
  import dgl
  import numpy
  import pandas
  import pyarrow
  import torch
  from dgl.distributed.partition import _etype_str_to_tuple, _etype_tuple_to_str, RESERVED_FIELD_DTYPE
  from pyarrow import csv
  from utils import get_idranges, memory_snapshot, read_json
label_desc: Use image node_id 1 for calling a global function with example usage: _get_unique_invidx(srcids, dstids, nids, low_mem) and returns: uniques,  idxes,  srcids,  dstids, uniques,  idxes,  inv_idxes,  inv_idxes