How could a considerable increase in loss leads to an improvement in accuracy?
Check whether padded values get accounted while computing accuracy, which shouldn't be the case. You should create mask for padded values and use that while computing accuracy. For instance, while the y_true value for padded region is 0, and 0 as well for y_pred, then y_true == y_pred counts as a correct prediction and impacts the overall accuracy. This is wrong, and what you should instead do is define a custom accuracy metric that does the following:
correct = y_pred == y_true
mask = tf.cast(tf.cast(y_true, tf.bool), tf.int16)
accuracy = tf.reduce_sum(correct * mask)/tf.reduce_sum(mask)

Hopefully this should give you a correct measure.