paleo/third_party/libcudnn.py
def cudnnSoftmaxBackward(handle, algorithm, mode, alpha, srcDesc, srcData,
                         srcDiffDesc, srcDiffData, beta, destDiffDesc,
                         destDiffData):
    """"
    This routine computes the gradient of the softmax function.

    Parameters
    ----------
    handle : cudnnHandle
        Handle to a previously created cuDNN context.
    algorithm : cudnnSoftmaxAlgorithm
        Enumerant to specify the softmax algorithm.
    mode : cudnnSoftmaxMode
        Enumerant to specify the softmax mode.
    alpha: float
        Scaling factor with which every element of the input tensors is multiplied.
    srcDesc : cudnnTensorDescriptor
        Handle to the previously initialized input tensor descriptor.
    srcData : void_p
        Data pointer to GPU memory associated with the tensor descriptor
        srcDesc.
    srcDiffDesc : cudnnTensorDescriptor
        Handle to the previously initialized input differential tensor descriptor.
    srcDiffData : void_p
        Data pointer to GPU memory associated with the tensor descriptor
        srcDiffData.
    beta: float
        Scaling factor which is applied on every element of the output tensor prior
        to adding the result of the activation Note that if beta is zero, the output
        is not read and can contain any uninitialized data (including Nan numbers).
    destDiffDesc : cudnnTensorDescriptor
        Handle to the previously initialized output differential tensor descriptor.
    destDiffData : void_p
        Data pointer to GPU memory associated with the output tensor descriptor
        destDiffDesc.
    """

    dataType = cudnnGetTensor4dDescriptor(destDiffDesc)[0]
    if dataType == cudnnDataType['CUDNN_DATA_DOUBLE']:
        alphaRef = ctypes.byref(ctypes.c_double(alpha))
        betaRef = ctypes.byref(ctypes.c_double(beta))
    else:
        alphaRef = ctypes.byref(ctypes.c_float(alpha))
        betaRef = ctypes.byref(ctypes.c_float(beta))

    status = _libcudnn.cudnnSoftmaxBackward(
        handle, algorithm, mode, alphaRef, srcDesc, srcData, srcDiffDesc,
        srcDiffData, betaRef, destDiffDesc, destDiffData)
    cudnnCheckStatus(status)

    This routine computes the gradient of the softmax function.

    Parameters
    ----------
    handle : cudnnHandle
        Handle to a previously created cuDNN context.
    algorithm : cudnnSoftmaxAlgorithm
        Enumerant to specify the softmax algorithm.
    mode : cudnnSoftmaxMode
        Enumerant to specify the softmax mode.
    alpha: float
        Scaling factor with which every element of the input tensors is multiplied.
    srcDesc : cudnnTensorDescriptor
        Handle to the previously initialized input tensor descriptor.
    srcData : void_p
        Data pointer to GPU memory associated with the tensor descriptor
        srcDesc.
    srcDiffDesc : cudnnTensorDescriptor
        Handle to the previously initialized input differential tensor descriptor.
    srcDiffData : void_p
        Data pointer to GPU memory associated with the tensor descriptor
        srcDiffData.
    beta: float
        Scaling factor which is applied on every element of the output tensor prior
        to adding the result of the activation Note that if beta is zero, the output
        is not read and can contain any uninitialized data (including Nan numbers).
    destDiffDesc : cudnnTensorDescriptor
        Handle to the previously initialized output differential tensor descriptor.
    destDiffData : void_p
        Data pointer to GPU memory associated with the output tensor descriptor
        destDiffDesc.
    
This routine computes the gradient of the softmax function.
Parameters

handle : cudnnHandle
Handle to a previously created cuDNN context.
algorithm : cudnnSoftmaxAlgorithm
Enumerant to specify the softmax algorithm.
mode : cudnnSoftmaxMode
Enumerant to specify the softmax mode.
alpha: float
Scaling factor with which every element of the input tensors is multiplied.
srcDesc : cudnnTensorDescriptor
Handle to the previously initialized input tensor descriptor.
srcData : void_p
Data pointer to GPU memory associated with the tensor descriptor
srcDesc.
srcDiffDesc : cudnnTensorDescriptor
Handle to the previously initialized input differential tensor descriptor.
srcDiffData : void_p
Data pointer to GPU memory associated with the tensor descriptor
srcDiffData.
beta: float
Scaling factor which is applied on every element of the output tensor prior
to adding the result of the activation Note that if beta is zero, the output
is not read and can contain any uninitialized data (including Nan numbers).
destDiffDesc : cudnnTensorDescriptor
Handle to the previously initialized output differential tensor descriptor.
destDiffData : void_p
Data pointer to GPU memory associated with the output tensor descriptor
destDiffDesc.
This routine computes the gradient of the softmax function . Parameters handle : cudnnHandle Handle to a previously created cuDNN context . algorithm : cudnnSoftmaxAlgorithm Enumerant to specify the softmax algorithm . mode : cudnnSoftmaxMode Enumerant to specify the softmax mode . alpha : float Scaling factor with which every element of the input tensors is multiplied . srcDesc : cudnnTensorDescriptor Handle to the previously initialized input tensor descriptor . srcData : void_p Data pointer to GPU memory associated with the tensor descriptor srcDesc . srcDiffDesc : cudnnTensorDescriptor Handle to the previously initialized input differential tensor descriptor . srcDiffData : void_p Data pointer to GPU memory associated with the tensor descriptor srcDiffData . beta : float Scaling factor which is applied on every element of the output tensor prior to adding the result of the activation Note that if beta is zero the output is not read and can contain any uninitialized data ( including Nan numbers ) . destDiffDesc : cudnnTensorDescriptor Handle to the previously initialized output differential tensor descriptor . destDiffData : void_p Data pointer to GPU memory associated with the output tensor descriptor destDiffDesc .
def cudnnSoftmaxBackward(handle, algorithm, mode, alpha, srcDesc, srcData,
                         srcDiffDesc, srcDiffData, beta, destDiffDesc,
                         destDiffData):
    dataType = cudnnGetTensor4dDescriptor(destDiffDesc)[0]
    if dataType == cudnnDataType['CUDNN_DATA_DOUBLE']:
        alphaRef = ctypes.byref(ctypes.c_double(alpha))
        betaRef = ctypes.byref(ctypes.c_double(beta))
    else:
        alphaRef = ctypes.byref(ctypes.c_float(alpha))
        betaRef = ctypes.byref(ctypes.c_float(beta))
    status = _libcudnn.cudnnSoftmaxBackward(
        handle, algorithm, mode, alphaRef, srcDesc, srcData, srcDiffDesc,
        srcDiffData, betaRef, destDiffDesc, destDiffData)
    cudnnCheckStatus(status)
def cudnnSoftmaxBackward ( handle , algorithm , mode , alpha , srcDesc , srcData , srcDiffDesc , srcDiffData , beta , destDiffDesc , destDiffData ) : dataType = cudnnGetTensor4dDescriptor ( destDiffDesc ) [ 0 ] if dataType == cudnnDataType [ 'CUDNN_DATA_DOUBLE' ] : alphaRef = ctypes . byref ( ctypes . c_double ( alpha ) ) betaRef = ctypes . byref ( ctypes . c_double ( beta ) ) else : alphaRef = ctypes . byref ( ctypes . c_float ( alpha ) ) betaRef = ctypes . byref ( ctypes . c_float ( beta ) ) status = _libcudnn . cudnnSoftmaxBackward ( handle , algorithm , mode , alphaRef , srcDesc , srcData , srcDiffDesc , srcDiffData , betaRef , destDiffDesc , destDiffData ) cudnnCheckStatus ( status )
This routine computes the gradient of the softmax function.
This routine computes the gradient of the softmax function .
""""
    This routine computes the gradient of the softmax function.

    Parameters
    ----------
    handle : cudnnHandle
        Handle to a previously created cuDNN context.
    algorithm : cudnnSoftmaxAlgorithm
        Enumerant to specify the softmax algorithm.
    mode : cudnnSoftmaxMode
        Enumerant to specify the softmax mode.
    alpha: float
        Scaling factor with which every element of the input tensors is multiplied.
    srcDesc : cudnnTensorDescriptor
        Handle to the previously initialized input tensor descriptor.
    srcData : void_p
        Data pointer to GPU memory associated with the tensor descriptor
        srcDesc.
    srcDiffDesc : cudnnTensorDescriptor
        Handle to the previously initialized input differential tensor descriptor.
    srcDiffData : void_p
        Data pointer to GPU memory associated with the tensor descriptor
        srcDiffData.
    beta: float
        Scaling factor which is applied on every element of the output tensor prior
        to adding the result of the activation Note that if beta is zero, the output
        is not read and can contain any uninitialized data (including Nan numbers).
    destDiffDesc : cudnnTensorDescriptor
        Handle to the previously initialized output differential tensor descriptor.
    destDiffData : void_p
        Data pointer to GPU memory associated with the output tensor descriptor
        destDiffDesc.
    """
handle
algorithm
mode
alpha
srcDesc
srcData
srcDiffDesc
srcDiffData
beta
destDiffDesc
destDiffData