Is it possible to expand nodes of a frozen neural network model by width?
I want to expand the number of nodes of a frozen neural network model by width in pytorch. I want to do something like what shown in the below image where grey are frozen weights and green are newly added trainable weights.
 .
I have an initial model which takes 3 inputs and gives one output back, this model also has two hidden layers with nodes h1=5 and h2=3 respectively. I created the model in pytorch and frozen the weights.
import torch
import torch.nn as nn
import torch.nn.functional as F    

class Net(nn.Module):
    def __init__(self):
        super(Net,self).__init__()
        self.fc1 = nn.Linear(3, 5)  
        self.fc2 = nn.Linear(5, 3)
        self.fc3 = nn.Linear(3, 1)
    def forward(self,x):
        x = self.fc1(x)
        x = self.fc2(x)
        x = self.fc3(x)
        return x       

print(Net())

model = Net()
X = torch.rand(5,3)
y = model(X)
print(y)

# Freeze layers
for param in model.parameters():
    param.requires_grad = False

Now I want to expand this model by adding trainable nodes to h1=5+2, h2=3+1 and output= 1+1. Only the newly added nodes should be trainable and all other weights should be frozen, and those frozen weights should have the same weight as the parent model. Can this be done in pytorch or in tensorflow?

There are 2 things need to be done
1. Expand the layers
You really should use ModuleList or ModuleDict to create layers because that would mean you can use loop. I know eval or setattr also work but they tend to break something else so I don't want to use them.
There's 2 ways I can think of. One is directly replace the weight with something bigger and the other one is create a bigger layer and replace the whole layer.
# Replace the weight with randomly generated tensor
fc1_newweight = torch.rand(7, 3)
fc1_newbias = torch.rand(7)
fc1_shape = model.fc1.weight.shape
fc1_newweight[:fc1_shape[0], :fc1_shape[1]] = model.fc1.weight.clone()
fc1_newbias[:fc1_shape[0]] = model.fc1.bias.clone()
model.fc1.weight = torch.nn.Parameter(fc1_newweight)
model.fc1.bias = torch.nn.Parameter(fc1_newbias)

# Replace the weight with the random generated weights from the new layer
fc2_shape = model.fc2.weight.shape
fc2 = nn.Linear(7, 4)
fc2_weight = fc2.state_dict()
fc2_weight['weight'][:fc2_shape[0], :fc2_shape[1]] = model.fc2.weight.clone()
fc2_weight['bias'][:fc2_shape[0]] = model.fc2.bias.clone()
fc2.load_state_dict(fc2_weight)
model.fc2.weight = torch.nn.Parameter(fc2_weight['weight'])
model.fc2.bias = torch.nn.Parameter(fc2_weight['bias'])

# Replace the whole layer
fc3_shape = model.fc3.weight.shape
fc3 = nn.Linear(4, 2)
fc3_weight = fc3.state_dict()
fc3_weight['weight'][:fc3_shape[0], :fc3_shape[1]] = model.fc3.weight.clone()
fc3_weight['bias'][:fc3_shape[0]] = model.fc3.bias.clone()
fc3.load_state_dict(fc3_weight)
model.fc3 = fc3

I'd prefer the 2. or 3. over 1. because the weights will be generated using nn.init.kaiming_uniform instead of just uniform.
2. Select what to be trainable
This is tricky because you can't just set require_grad on only some elements of the weights because you'll get RuntimeError: you can only change requires_grad flags of leaf variables.
But something like this should be a good enough substitute. Again, using ModuleList will make the code here look a lot better too.
y = model(x)
loss = criterion(y, target)
loss.backward()

model.fc1.weight.grad[:fc1_shape[0], :fc1_shape[1]] = 0
model.fc1.bias.grad[:fc1_shape[0]] = 0
model.fc2.weight.grad[:fc2_shape[0], :fc2_shape[1]] = 0
model.fc2.bias.grad[:fc2_shape[0]] = 0
model.fc3.weight.grad[:fc3_shape[0], :fc3_shape[1]] = 0
model.fc3.bias.grad[:fc3_shape[0]] = 0

optimizer.step()

