For every Bayesian Network, is there a Neural Network that gives the same output? After thinking about a question comparing Bayesian Network to Neural Network, I am now wondering if they may not be one and the same thing!

At this point, my maths (unused for over 20 years fails me!).   But....


If infinite random test data was used
The output of the Bayesian Network was found for each data item
And the Neural Network was then trained on it


Would the Neural Network not tent towards being the same as the Bayesian Network?
We don't have to do anything crazy to prove this, we can simply leverage the UAT. 

By the universal approximation theorem, yes, you could concieve of a neural network that precisely matched your Bayesian network.