Convergence of semi-gradient TD(0) with non-linear function approximation
Apparently there is an example of non-convergence for semi-gradient sarsa, according to Rich Sutton (check slide 35). I guess TD(0) is not so different. So, probably your approximator will need to satisfy certain conditions to proof convergence. 

Maybe this paper will be useful for you. It seems that they show that constraining your network to have relu activation functions allow you to show some convergence properties.