Adding a feature to a neuron
Say I have the following model:
import torch
import torch.nn as nn
import torch.optim as optim

class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        self.fc1 = nn.Linear(1, 5)
        self.fc2 = nn.Linear(5, 10)
        self.fc3 = nn.Linear(10, 1)

    def forward(self, x):
        x = self.fc1(x)
        print(x) 
        x = torch.relu(x)        
        x = torch.relu(self.fc2(x))
        x = self.fc3(x)
        return x

net = Model()

opt = optim.Adam(net.parameters())
features = torch.rand((3,1)) #3 inputs, each of 1D

I can print the value of my neurons (only the first layer here) with print(x):
net(features)
&gt;&gt;&gt;tensor([[ 0.6703,  0.4484, -0.8529,  1.3119,  0.6741],
        [ 0.9112,  0.6496, -1.2960,  1.8264,  0.4547],
        [ 0.7483,  0.5135, -0.9963,  1.4785,  0.6031]],
       grad_fn=&lt;AddmmBackward&gt;)
tensor([[0.0144],
        [0.0575],
        [0.0284]], grad_fn=&lt;AddmmBackward&gt;)

How can I add a &quot;feature&quot; to each neuron that is a string with a name?
e.g.
print(x)
&gt;&gt;&gt; tensor([[ [0.6703, 'neuron_1'],  [0.4484, 'neuron_2'], [-0.8529, 'neuron_3'],  1.3119,  0.6741],... etc.

I'm not sure if I'll need to change the neuron class. I believe in the forward method I will then need to only take the first element of each neurons tensor: neuron_tensor = [neuron_value, neuron_name]
Update 1:
from @Aditya Singh Rathore comment it sounds like it might not be possible to have a string and a value in the same tensor. Is it possible then to have a value instead of a string to represent the neurons?
From before neuron_tensor = [neuron_value, neuron_name] where neuron_name is a string.
Is this possible instead? : neuron_tensor = [neuron_value, neuron_name] where neuron_name is just a value (e.g 1 for neuron 1, 2 for neuron 2)

What you want is possible, although I'm not sure what you exactly want to do with this. Basically you seem to want to include the &quot;index&quot; in the intermediate tensor for some purpose, and again discard it when passing to the next layer.
class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        self.fc1 = nn.Linear(1, 5)
        self.fc2 = nn.Linear(5, 10)
        self.fc3 = nn.Linear(10, 1)

    def forward(self, x):
        x = self.fc1(x)
        idx_tensor = (torch.arange(x.shape[1]) + 1).unsqueeze(0).repeat_interleave(repeats=x.shape[0], dim=0)
        x = torch.cat([x.unsqueeze(2), idx_tensor.unsqueeze(2)], dim=2)
        print(x)
        
        x = torch.relu(x[:, :, 0])    
        x = torch.relu(self.fc2(x))
        x = self.fc3(x)
        return x

Running this gives:
net = Model()

opt = optim.Adam(net.parameters())
features = torch.rand((3,1))
net(features)

&gt;&gt;&gt;tensor([[[ 0.0817,  1.0000],
         [ 0.8084,  2.0000],
         [ 1.6118,  3.0000],
         [ 0.8658,  4.0000],
         [-0.1583,  5.0000]],

        [[ 0.2881,  1.0000],
         [ 0.6946,  2.0000],
         [ 1.3760,  3.0000],
         [ 0.6098,  4.0000],
         [-0.1240,  5.0000]],

        [[ 0.1919,  1.0000],
         [ 0.7476,  2.0000],
         [ 1.4859,  3.0000],
         [ 0.7291,  4.0000],
         [-0.1400,  5.0000]]], grad_fn=&lt;CatBackward&gt;)

tensor([[-0.2841],
        [-0.2191],
        [-0.2495]], grad_fn=&lt;AddmmBackward&gt;)

Note that a typical tensor cannot be both integer and float at the same time, so the 1, 2, 3 will be stored as float 1.000.., 2.000... etc.
I suggest if your purpose is something like a fancy printing, then maybe look into torch's hook functions? For example, you can do something like:
import pandas as pd

def fc_hook_fn(module, input, output):
    print(&quot;\n&quot; + &quot;#&quot; * 60)
    print(f&quot;In layer {module}&quot;)
    print(&quot;#&quot; * 60 + &quot;\n&quot;)
    cols = [f&quot;Neuron-{i + 1}&quot; for i in range(output.shape[1])]
    idx = [f&quot;Input-{i + 1}&quot; for i in range(output.shape[0])]
    neuron_activations = pd.DataFrame(output.detach().numpy(), columns=cols, index=idx)
    print(neuron_activations)

net.fc.register_forward_hook(fc_hook_fn)

Now each time something passes through fc1, the function above will be triggered. You don't need to put your print(x) in the forward method.

############################################################
In layer Linear(in_features=1, out_features=5, bias=True)
############################################################

         Neuron-1  Neuron-2  Neuron-3  Neuron-4  Neuron-5
Input-1 -0.948735 -0.901034 -0.290353 -0.082616 -0.405337
Input-2 -0.725904 -0.801648 -0.302922 -0.045514 -0.580485
Input-3 -0.829738 -0.847960 -0.297065 -0.062802 -0.498870

