For every Bayesian Network, is there a Neural Network that gives the same output?
We don't have to do anything crazy to prove this, we can simply leverage the UAT. 

By the universal approximation theorem, yes, you could concieve of a neural network that precisely matched your Bayesian network.