Using best practice with complex iteration I'm new to Rust so I'm still learning how to use best practice in complex examples.

Below is some code I've used in one of my programs, a simple implementation of a standard neural network.

Here is a the file I'm running: https://pastebin.com/p9ntgWPK

Here is the code snippet I'm particularly interested in (lines 60-72 in the pastebin):

    for x in 0..self.neuron_layers.len() - 1 {
        for y in 0..self.neuron_layers[x+1].len() {
            let mut sum:f64 = 0f64;
            for z in 0..self.neuron_layers[x].len() {
                sum += self.neuron_layers[x][z] * self.connection_layers[x][y][z];
            }
            self.neuron_layers[x+1][y] = sigmoid(sum);
        }
    }


To me this code stinks of inefficiency and bad practice, I just have a gut feeling.

So I wanted to ask 2 things.


How can I improve?
What elements within it are generally considered bad practice?
Honestly, that really doesn't look that bad to me.  It's probably what I would come out with after a first go - not that I'm some sort of Rust guru.  That being said, I'll discuss below a couple of things that you might like better.  It'd also be worth running Clippy over the program and having a look at what it says.  Not everyone likes everything Clippy says, but it can be quite informative.

As earlier commenters have said, it would be worth looking into using a linear algebra library, since this appears to be a matrix multiplication and that would probably end up being more efficient, and easier to use or manage.  All that being said, let's examine your program.

The first thing that jumps out at me is that you are doing a floating-point multiplication followed by an addition for every instance of the inner loop.  It so happens that Rust comes with something for that built-in these days.  You could update your inner-most loop to use this:

for x in 0..self.neuron_layers.len() - 1 {
        for y in 0..self.neuron_layers[x+1].len() {
            let mut sum:f64 = 0f64;
            for z in 0..self.neuron_layers[x].len() {
                sum = self.neuron_layers[x][z].mul_add(self.connection_layers[x][y][z], sum);
            }
            self.neuron_layers[x+1][y] = sigmoid(sum);
        }
    }


The two main advantages of this are that it should be less susceptible to floating-point imprecisions, meaning that you actually come out with a 'more accurate' result at the end, and that on certain hardware (actually most processors these days, I think) this should be a faster operation as it uses fused-multiply-add instructions built into said hardware to do both parts in one go.

Secondly, since you seem to be using the z index just to iterate over every entry in neuron_layers[x], you could actually swap over to using map, as per something like the below:

for x in 0..self.neuron_layers.len() - 1 {
    for y in 0..self.neuron_layers[x+1].len() {
        let sum = self.neuron_layers[x].iter().enumerate()
            .map(|(z, nl)| nl * self.connection_layers[x][y][z]).sum();
        self.neuron_layers[x+1][y] = sigmoid(sum);
    }
}


(I didn't have a chance to test this out, so it might need some tweaking to make the compiler totally happy).  In fact, I think you could expand this approach for every element in the first dimension of neuron_layers if you wanted to.  A potential benefit of this would be that Rayon should be quite good for targeting that level of the loop for parallelism.  Perhaps you could use this approach at the top level of the loop, and the mul_add approach in the inner loop.  You'd have to do benchmarks to figure out for sure what is fastest/most memory-efficient though.

Finally, having a quick look at your Pastebin link, I could see a couple of things worth mentioning.  Firstly, it looks like in at least one instance you create a Vec with length 1, but immediately push a known number of other elements onto it.  You should consider using Vec::with_capacity() to create this, as it should mean that you get the exact right amount of memory allocated, to begin with, which should be both faster and more efficient.

Secondly, I think with

for x in 0..input.len() {
            self.neuron_layers[0][x] = input[x];
        }


you could look at using extend_from_slice or copy_from_slice.  I must admit that I have never used either of those though, so I'm not 100% sure how to use them correctly with two Vecs.

I know this is nearly two months after you posted, but I hope this is helpful and/or informative nonetheless :)



EDIT: I realised that I had overlooked something an alternative to using map earlier, which I personally feel is a bit cleaner.  You could use fold instead, which should have the same net effect, but be (imho) a bit nicer to look at.

for x in 0..self.neuron_layers.len() - 1 {
    for y in 0..self.neuron_layers[x+1].len() {
        let sum = self.neuron_layers[x].iter().enumerate()
           .fold(0.0, |acc, (z, nl)| nl.mul_add(
           self.connection_layers[x][y][z], acc));
        self.neuron_layers[x+1][y] = sigmoid(sum);
    }
}


It might be the case that the compiler will complain about capturing things in closures, or something along those lines though, I must admit.  You could look at using zip first, e.g. 

for x in 0..self.neuron_layers.len() - 1 {
    for y in 0..self.neuron_layers[x+1].len() {
        let sum = self.neuron_layers[x].iter().zip(self.connection_layers[x][y].iter())
           .fold(0.0, |acc, (nl, cl)| nl.mul_add(cl, acc));
        self.neuron_layers[x+1][y] = sigmoid(sum);
    }
}


but I'm not sure that would actually help.

The potential downside to this approach, however, is that you are effectively telling the compiler exactly how to do the summation, while with sum you leave that as an implementation detail of Vec - and there's a pretty good chance that Vec's approach will end up working as well as whatever we could write ourselves in 90%+ of cases.

As things currently stand, I think specifying it yourself is probably for the best, but if in the future someone works out a better way to sum over a Vec, that might end up being incorporated into the Vec struct.  If you are simply using sum, you get that 'for free' when you compile the program, whereas if you are using fold you continue using the exact same approach no matter what (which may or may not be what you want).  Again, you would probably have to test the different approaches out and decide which one works best for you.