How to decide if gradients are vanishing?
Vanishing gradients are the updates to the weights (not the weight themselves) over the layers after each training batch.
You should check the update signal between the top-most layer and lowest layer after a single batch.