Why is having high p-value makes dependent variable significant in backward elimination?
Actually, the Null Hypothesis is that the predictor is not significant. Taken from the book Introduction to Statistical Learning:


Null Hypothesis: There is no relationship between X and Y
Alternative Hypothesis: There is some relationship between X and Y


If we have a high p-value, we have an expressive result showing that the null hypothesis is correct and therefore the estimated coefficient come from a normal distribution around zero and can be discarded. I hope this answer your question, please let any comment if you need more help.