class_docstr: The shard sync layer is a synchronization point between model shards.
  - In the forward pass, it drops parameters in the previous shard and
  loads parameters for the next shard.
  - In the backward pass, it does the reverse.
  It does not change or create any outputs at all, instead it just
  forwards the input as the output.
  NOTE: see https://pytorch.org/docs/stable/autograd.html#torch.autograd.Function
code: def forward(
      ctx: Any,
      inputs: Any,
      index: int,
      model_slices: Any,
      model_instance: Any,
  ) -> Any:
      drop_index = index
      load_index = index + 1
      max_slices = len(model_slices)
  
      if drop_index >= 0:
          # Move shard from device to offload device.
          model_slices[drop_index].forward_drop()
  
      if load_index < max_slices:
          # Load shard from offload device to device.
          model_slices[load_index].forward_load()
  
      ctx.index = index
      ctx.model_slices = model_slices
      ctx.model_instance = model_instance
  
      return inputs if isinstance(inputs, tuple) else (inputs,)
  
imports: from builtins import isinstance
  import functools
  import logging
  from typing import Any, List, Tuple
  import torch
  from torch import nn
label_desc: Use image node_id 1 for calling the ShardSyncLayer obj's underlying member method code with example usage: obj.forward(ctx, inputs, index, model_slices, model_instance) and returns: unknown