In the next post we will dive inside PyTorch code to see how this graph is constructed and where are the relevant pieces should you want to experiment with it!
References

https://pytorch.org/tutorials/beginner/blitz/autograd_tutorial.html
https://web.stanford.edu/class/cs224n/readings/gradient-notes.pdf
https://www.cs.toronto.edu/~rgrosse/courses/csc321_2018/slides/lec10.pdf
https://mustafaghali11.medium.com/how-pytorch-backward-function-works-55669b3b7c62