How to create a tensorflow network of two saved tensorflow networks?
I am not sure if I got your point correctly, but Block Based Neural Networks might be what you are searching for. In BBNN each node can be a neural network and w.r.t what you describe one layer BBNN is what you need.
Let's say I've trained and saved 6 different networks where all of the values for hidden layer counts, neuron counts, and learn rates differ.
For example:

1 with 8 hidden layers with 16 neurons in each trained at .1 learn rate.
1 with 4 hidden layers with 4 neurons in each trained at .01 learn rate.
1 with 4 hidden layers with 4 neurons in each trained at .03 learn rate.
1 with 4 hidden layers with 8 neurons in each trained at .01 learn rate.
1 with 4 hidden layers with 8 neurons in each trained at .001 learn rate.
1 with 6 hidden layers with 4 neurons in each trained at .01 learn rate.

How could I create a new network where each of these saved networks acts basically as a neuron? While training this combined network, I don't want to affect the saved network weights and biases, but rather would like to essentially determine which one is more accurate for a given input.
I've achieved this in practice by loading each network, running the data through each network, and then storing all of the outputs which then later feed into the new network, but I feel like there must be a simpler and most importantly, a faster way of doing this.
An example might be two networks: image detection at night and image detection during the day. Each trained and saved separately. I would want another network which essentially takes an image and says "oh we're somewhere in the middle here so let's use 50/50, or oh it's closer to night, but not night completely, use 90% night data and 10% day." So I would want to feed loads of images where it tries each model, but then weights out how valued the data was based on night vs day to create a network which works for either night or day.
Any help highly appreciated. In reality the network I'm shooting for is far bigger and more complicated, but I'm looking for a strategy.