aws_lambda/pytorch/source/torch/utils/checkpoint.py
def checkpoint_sequential(functions, segments, *inputs):
    r"""A helper function for checkpointing sequential models.

    Sequential models execute a list of modules/functions in order
    (sequentially). Therefore, we can divide such a model in various segments
    and checkpoint each segment. All segments except the last will run in
    :func:`torch.no_grad` manner, i.e., not storing the intermediate
    activations. The inputs of each checkpointed segment will be saved for
    re-running the segment in the backward pass.

    See :func:`~torch.utils.checkpoint.checkpoint` on how checkpointing works.

    .. warning::
        Checkpointing doesn't work with :func:`torch.autograd.grad`, but only
        with :func:`torch.autograd.backward`.

    .. warning:
        At least one of the inputs needs to have :code:`requires_grad=True` if
        grads are needed for model inputs, otherwise the checkpointed part of the
        model won't have gradients.

    Args:
        functions: A :class:`torch.nn.Sequential` or the list of modules or
            functions (comprising the model) to run sequentially.
        segments: Number of chunks to create in the model
        inputs: tuple of Tensors that are inputs to :attr:`functions`

    Returns:
        Output of running :attr:`functions` sequentially on :attr:`*inputs`

    Example:
        >>> model = nn.Sequential(...)
        >>> input_var = checkpoint_sequential(model, chunks, input_var)
    """

    def run_function(start, end, functions):
        def forward(*inputs):
            for j in range(start, end + 1):
                if isinstance(inputs, tuple):
                    inputs = functions[j](*inputs)
                else:
                    inputs = functions[j](inputs)
            return inputs
        return forward

    if isinstance(functions, torch.nn.Sequential):
        functions = list(functions.children())

    segment_size = len(functions) // segments
    # the last chunk has to be non-volatile
    end = -1
    for start in range(0, segment_size * (segments - 1), segment_size):
        end = start + segment_size - 1
        inputs = checkpoint(run_function(start, end, functions), *inputs)
        if not isinstance(inputs, tuple):
            inputs = (inputs,)
    return run_function(end + 1, len(functions) - 1, functions)(*inputs)
r"""A helper function for checkpointing sequential models.

    Sequential models execute a list of modules/functions in order
    (sequentially). Therefore, we can divide such a model in various segments
    and checkpoint each segment. All segments except the last will run in
    :func:`torch.no_grad` manner, i.e., not storing the intermediate
    activations. The inputs of each checkpointed segment will be saved for
    re-running the segment in the backward pass.

    See :func:`~torch.utils.checkpoint.checkpoint` on how checkpointing works.

    .. warning::
        Checkpointing doesn't work with :func:`torch.autograd.grad`, but only
        with :func:`torch.autograd.backward`.

    .. warning:
        At least one of the inputs needs to have :code:`requires_grad=True` if
        grads are needed for model inputs, otherwise the checkpointed part of the
        model won't have gradients.

    Args:
        functions: A :class:`torch.nn.Sequential` or the list of modules or
            functions (comprising the model) to run sequentially.
        segments: Number of chunks to create in the model
        inputs: tuple of Tensors that are inputs to :attr:`functions`

    Returns:
        Output of running :attr:`functions` sequentially on :attr:`*inputs`

    Example:
        >>> model = nn.Sequential(...)
        >>> input_var = checkpoint_sequential(model, chunks, input_var)
    
r"""A helper function for checkpointing sequential models.
Sequential models execute a list of modules/functions in order
(sequentially). Therefore, we can divide such a model in various segments
and checkpoint each segment. All segments except the last will run in
:func:`torch.no_grad` manner, i.e., not storing the intermediate
activations. The inputs of each checkpointed segment will be saved for
re-running the segment in the backward pass.

See :func:`~torch.utils.checkpoint.checkpoint` on how checkpointing works.



At least one of the inputs needs to have :code:`requires_grad=True` if
grads are needed for model inputs, otherwise the checkpointed part of the
model won't have gradients.
r " " " A helper function for checkpointing sequential models . Sequential models execute a list of modules / functions in order ( sequentially ) . Therefore we can divide such a model in various segments and checkpoint each segment . All segments except the last will run in : func : ` torch . no_grad ` manner i . e . not storing the intermediate activations . The inputs of each checkpointed segment will be saved for re - running the segment in the backward pass . See : func : ` ~torch . utils . checkpoint . checkpoint ` on how checkpointing works . At least one of the inputs needs to have : code : ` requires_grad = True ` if grads are needed for model inputs otherwise the checkpointed part of the model won ' t have gradients .
def checkpoint_sequential(functions, segments, *inputs):
    def run_function(start, end, functions):
        def forward(*inputs):
            for j in range(start, end + 1):
                if isinstance(inputs, tuple):
                    inputs = functions[j](*inputs)
                else:
                    inputs = functions[j](inputs)
            return inputs
        return forward
    if isinstance(functions, torch.nn.Sequential):
        functions = list(functions.children())
    segment_size = len(functions) // segments
    end = -1
    for start in range(0, segment_size * (segments - 1), segment_size):
        end = start + segment_size - 1
        inputs = checkpoint(run_function(start, end, functions), *inputs)
        if not isinstance(inputs, tuple):
            inputs = (inputs,)
    return run_function(end + 1, len(functions) - 1, functions)(*inputs)
def checkpoint_sequential ( functions , segments , * inputs ) : def run_function ( start , end , functions ) : def forward ( * inputs ) : for j in range ( start , end + 1 ) : if isinstance ( inputs , tuple ) : inputs = functions [ j ] ( * inputs ) else : inputs = functions [ j ] ( inputs ) return inputs return forward if isinstance ( functions , torch . nn . Sequential ) : functions = list ( functions . children ( ) ) segment_size = len ( functions ) // segments end = - 1 for start in range ( 0 , segment_size * ( segments - 1 ) , segment_size ) : end = start + segment_size - 1 inputs = checkpoint ( run_function ( start , end , functions ) , * inputs ) if not isinstance ( inputs , tuple ) : inputs = ( inputs , ) return run_function ( end + 1 , len ( functions ) - 1 , functions ) ( * inputs )
r"""A helper function for checkpointing sequential models.
r " " " A helper function for checkpointing sequential models .
r"""A helper function for checkpointing sequential models.

    Sequential models execute a list of modules/functions in order
    (sequentially). Therefore, we can divide such a model in various segments
    and checkpoint each segment. All segments except the last will run in
    :func:`torch.no_grad` manner, i.e., not storing the intermediate
    activations. The inputs of each checkpointed segment will be saved for
    re-running the segment in the backward pass.

    See :func:`~torch.utils.checkpoint.checkpoint` on how checkpointing works.

    .. warning::
        Checkpointing doesn't work with :func:`torch.autograd.grad`, but only
        with :func:`torch.autograd.backward`.

    .. warning:
        At least one of the inputs needs to have :code:`requires_grad=True` if
        grads are needed for model inputs, otherwise the checkpointed part of the
        model won't have gradients.

    Args:
        functions: A :class:`torch.nn.Sequential` or the list of modules or
            functions (comprising the model) to run sequentially.
        segments: Number of chunks to create in the model
        inputs: tuple of Tensors that are inputs to :attr:`functions`

    Returns:
        Output of running :attr:`functions` sequentially on :attr:`*inputs`

    Example:
        >>> model = nn.Sequential(...)
        >>> input_var = checkpoint_sequential(model, chunks, input_var)
    """ # the last chunk has to be non-volatile
Output of running :attr:`functions` sequentially on :attr:`*inputs`
Output of running : attr : ` functions ` sequentially on : attr : ` * inputs `
functions
A :class:`torch.nn.Sequential` or the list of modules or
functions (comprising the model) to run sequentially.
A : class : ` torch . nn . Sequential ` or the list of modules or functions ( comprising the model ) to run sequentially .
segments
Number of chunks to create in the model
Number of chunks to create in the model
inputs
tuple of Tensors that are inputs to :attr:`functions`
tuple of Tensors that are inputs to : attr : ` functions `
examples
>>> model = nn.Sequential(...)
>>> input_var = checkpoint_sequential(model, chunks, input_var)
>>> model = nn . Sequential ( ... ) >>> input_var = checkpoint_sequential ( model chunks input_var )