Why is the "dying ReLU" problem not present in most modern deep learning architectures?
In practice, dead ReLUs connections are not a major issue. Most deep learning networks can still learn an adequate representations with only sub-selection of possible connections. This is possible because deep learning networks are highly over-parameterized.
Even with the possible drawbacks backs of the dying ReLUs problem, the computational effectiveness and efficiency of ReLUs still make them one of the best options currently available.