Help making a custom categorical loss function in Keras
The issue is very easy to solve assumming you still want to use crossentropy as your loss. 


Tell your model to use temporal sample weights. You can do it like this by model.compile(sample_weight_mode='temporal', **other_params)
Generate your sample weights, I think you are smart enough to write your own implementation. The idea is as you said apply weights of 1 if it is supposed to be counted and apply 0 if it is not supposed to be counted. For example you have a sequence [3,5,1,-3,2,0,0,0,0] then your sample weights will be [1,1,1,1,1,0,0,0,0].
Supply the sample weights during fitting. Simply use model.fit(X,y,sample_weights=sample_weights, **other_fit_params).


Done. Now loss is only counted on non-zero entries of your output.