Convert custom Convolution from PyTorch to Tensorflow (2.2.0)
I am currently attempting to convert a custom convolution from PyTorch to Tensorflow (V. 2.2.0).

The convolution is defined in PyTorch as:

    self.quantizer = q = nn.Conv1d(1, 2*nq, kernel_size=1, bias=True)
    a = (nq-1) / gap
    #1st half = lines passing to (min+x,1) and (min+x+1/a,0) with x = {nq-1..0}*gap/(nq-1)
    q.weight.data[:nq] = -a
    q.bias.data[:nq] = torch.from_numpy(a*min + np.arange(nq, 0, -1)) # b = 1 + a*(min+x)
    #2nd half = lines passing to (min+x,1) and (min+x-1/a,0) with x = {nq-1..0}*gap/(nq-1)
    q.weight.data[nq:] = a
    q.bias.data[nq:] = torch.from_numpy(np.arange(2-nq, 2, 1) - a*min) # b = 1 - a*(min+x)
    # first and last one are special: just horizontal straight line
    q.weight.data[0] = q.weight.data[-1] = 0
    q.bias.data[0] = q.bias.data[-1] = 1


where nq = 20, min = 0 and max = 1.

My reimplementation looks like this:

my_weight = my_init_weight((1,1,nq*2))
q = tf.nn.convolution(input_q, my_weight)
q = tf.nn.bias_add(q, my_init_bias((40,1), tf.float32))


with these these functions as weight and bias initialization:

def my_init_weight(shape, dtype=None):

    weights = np.zeros(shape, dtype=np.float32)
    weights[:, :, :nq] = -a
    weights[:, :, nq:] = a
    weights[:, :, 0] = weights[:, :, -1] = 0
    return tf.convert_to_tensor(weights, dtype=tf.float32)
def my_init_bias(shape, dtype=None):
    weights = np.zeros(shape[0], dtype=np.float32)
    weights[:nq] = a*min + np.arange(nq, 0, -1)
    weights[nq:] = np.arange(2-nq, 2, 1) - a*min
    weights[0] = weights[-1] = 1
    return weights


The input is a matrix with shape 1681, 1, 1600 for PyTorch (as it uses channels first) and 1681, 1600, 1 for Tensorflow (as it uses channels last) and the out put is 1681, 40, 1600 or 1681, 1600, 40. So it should be correct, however, the output of both convolutions is different. 

Input, Output: Tensorflow on a random 100, 100 image:

my_weight = my_init_weight((1,1,nq*2))
my_weight = tf.nn.bias_add(my_weight, my_init_bias((40,1), tf.float32))
q = tf.nn.convolution(test_conv, my_weight)

q_left, q_right = tf.split(q, 2, axis=2)
q = tf.math.minimum(q_left, q_right)
nbs = tf.reduce_sum(q, axis=0)




Input, Output: PyTorch on a random 100, 100 image:

output = q(input_t_t)
output = torch.min(output[:,:nq], output[:,nq:]).clamp(min=0)    
nbs = output.sum(dim=-1)




Okay, I found the solution:

I forgot to add the .clamp(min=0).

Adding q = tf.clip_by_value(q, 0, tf.keras.backend.max(q)) to

my_weight = my_init_weight((1,1,nq*2))
my_weight = tf.nn.bias_add(my_weight, my_init_bias((40,1), tf.float32))
q = tf.nn.convolution(test_conv, my_weight)

q_left, q_right = tf.split(q, 2, axis=2)
q = tf.math.minimum(q_left, q_right)
q = tf.clip_by_value(q, 0, tf.keras.backend.max(q)) &lt;-----------
nbs = tf.reduce_sum(q, axis=0)


fixed the problem.
