List of lists of arrays, zipping, modifying in place I came back to some old code today and freaked out.  I was like "How does this even work, and if it works it can't do what I intended."  I ran a few quick tests and confirmed it did actually do what was intended.  I know the long term solution is to avoid this situation by having good unit tests -- then I don't need to freak out, I know the code is doing what I expect (New years resolution right there.)

But it wasn't that the code didn't do what it was supposed to, it did.  What it is doing is kind of complex, and it doesn't make it clear.

orig is a List of list of numpy arrays.  All the inner lists have the same lengths.

The information in orig is known as knowledge.  In particular each matrix inner list has 1-3 piece of knowledge, these knowledge are in have names the first is weight, the second is upwards-bias and the third downwards-bias

Each inner list comes is the knowledge of some particular object called a layer.  When those objects are collected to make a more knowledgeable object, then that object also has knowledge with is the list of the knowledge of its component parts.  So the outer list is a List of knowledge of layers. And is the overall knowledge

Sometimes I would like to destroy some of that knowledge by replacing all the matrices in a layer with a randomly valued matrices of the same size and I would like to specify that with a parameter called layer_nums_to_reset.

This parameter is a list of Booleans; it is equal in length to the outer list.

def rand_mat(rows,cols=None):
    if cols==None:
        ret =np.random.normal(0,0.01,(rows,))
    else:
        ret = np.random.normal(0,0.01,(rows,cols))
    return ret

def reset_layers(layer_nums_to_reset,*orig):
    assert(len(layer_nums_to_reset)  == len (orig[0]))
    def random_like(mat):
        return rand_mat(*mat.shape)

    news = [ [] for _ in orig]

    for layer_info in zip(layer_nums_to_reset, *orig):
        reset = layer_info[0]
        knows = layer_info[1:]
        for new_know_list, old_know in zip (news,knows):
            new_know = random_like(old_know) if reset else old_know
            new_know_list.append(new_know)

    return news


Since this code scares me mere months later, it needs to be improved.  It is a complicated idea, so it is not surprising the implementation is complicated.
1. Code review


There's no documentation! To make your code maintainable, it's vital to explain the purpose of each function. What does it do, what arguments does it take, and what does it return?
The function rand_mat seems useless. It's a very thin wrapper around numpy.random.normal so instead of writing:

random_like(old_know)


write:

np.random.normal(0, 0.01, old_know.shape)


This would allow you to get rid of the rand_mat and random_like functions, which would be two fewer things to understand and maintain.
The numbers 0 and 0.01 should have names. They should perhaps also be parameters to the reset_layers function.
The name layer_nums_to_reset is misleading as it is list of Booleans, not a list of layer numbers. A name like layers_to_reset or maybe just reset would be better.
The assert only checks the length of layer_nums_to_reset against orig[0], but you wrote, "All the inner lists have the same lengths." It would be clearer to assert the stronger condition.
The reset_layers function does not really need to know what is in the layers (whether knowledge or whatever does not matter). In fact, it is confusing to bring this irrelevant detail into the function. Separation of concerns is an important technique for making maintainable code.
The arguments to reset_layers do not correspond to the return value. The caller must pass the layers as individual arguments, but gets back a list of layers. It would be clearer to pass a list of layers and return a list of layers.
You go to some effort to process the layers in an unnatural order: that is, to process the first element of each layer, then the second element of each layer, and so on. I guess you do this so that you only have to iterate over layer_nums_to_reset once. But this seems like a false economy if it makes the function more complex and harder to understand. Keep things simple!


2. Revised code

import numpy as np

def reset_layers(reset, layers, mean=0, std=0.01):
    """Return a list of lists of arrays that contains the same arrays as
    layers but with some replaced by arrays of the same shape
    containing normally distributed random numbers.

    layers -- a list of lists of arrays. The inner lists all have the
              same length, n.
    reset -- an array of Booleans of length n. The replaced arrays are
             at the positions given by the True values in this array.
    mean, std -- the mean and standard deviation of the random numbers.

    """
    assert(all(len(reset) == len(layer) for layer in layers))

    def reset_layer(layer):
        """Reset arrays in a single layer."""
        for res, old in zip(reset, layer):
            if res:
                yield np.random.normal(mean, std, old.shape)
            else:
                yield old

    return [list(reset_layer(layer)) for layer in layers]