Is there a neural network method for time-varying directed graphs?
I'm seeing recent trend of combining RNN/CNN with GNN(graph neural networks) so that both time dependency and topology are captured. I would suggest you to start by looking at DCRNN (Yaguang Li et al.), it's a strong baseline that everyone uses nowadays. Other good resources: 


Graph wavenet for deep spatial-temporal graph modeling (Zonghan Wu et al.)
Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting (Bing Yu et al.)
STANN: A Spatio–Temporal Attentive Neural Network for Traffic Prediction (Zhixiang He et al.)