Understanding the usage of nn.Linear for Forward Propagation in PyTorch
The purpose of this study is to build a simplified forward propagation model that reproduces the code structure in PyTorch, yet does not use any of the PyTorch libraries. The idea is to do a matrix multiplication while emulating the code structure, including class definitions as in pyTorch.
PyTorch code for Forward Propagation
from torch import nn
#DEFINE THE REQUIRED CLASS

class Network(nn.Module):
    def __init__(self):
        super().__init__()
        
        # Inputs to hidden layer linear transformation
        self.hidden = nn.Linear(784, 256)
        # Output layer, 10 units - one for each digit
        self.output = nn.Linear(256, 10)
        
        # Define sigmoid activation and softmax output 
        self.sigmoid = nn.Sigmoid()
        self.softmax = nn.Softmax(dim=1)
        
    def forward(self, x):
        # Pass the input tensor through each of our operations
        x = self.hidden(x)
        x = self.sigmoid(x)
        x = self.output(x)
        x = self.softmax(x)
        
        return x
#use
model = Network()
...
ps = model.forward(some_tensor)

here is my reproducer code:
class my_mul:
        def __init__(self, h, w):
                self.dim1 = h
                self.dim2 = w
                self.layer = LAYER(self.dim1, self.dim2)
        def forward(self, X):
                X =  self.layer.doit( X)
                return X

class LAYER:
        def __init__(self, h, w):
                self.dim1 = h
                self.dim2 = w
        def __call__(self, Z):
                self.matrix2 = Z
        def doit(self, X):
                self.matrix1 = np.random.rand(self.dim1, self.dim2)
                print('matrix 1 in class LAYER ',self.matrix1)
                X =  np.matmul(self.matrix1, self.matrix2)

import numpy as np
#
np.random.seed(0)
# initialize matrix2 which emulates the tensor being passed down to the forward 
# propagation step within a deep network training

matrix2 = np.random.rand(2,2)
print('matrix2 ',matrix2)
#
# use the __call__ method of LAYER to pass matrix2 to LL
LL = LAYER(2,2)
LL(matrix2)

MM = my_mul(2,2)
P = MM.forward(matrix2)
print('the product of the 2 matrices is ', P)


The above 'reproducer code' fails as follows. The problem is that the data is not passed correctly and I don't know how to make it work.
matrix2  [[0.5488135  0.71518937]
 [0.60276338 0.54488318]]
matrix 1 in class LAYER  [[0.4236548  0.64589411]
 [0.43758721 0.891773  ]]
Traceback (most recent call last):
  File &quot;all_s.py&quot;, line 30, in &lt;module&gt;
    P = MM.forward(matrix2)
  File &quot;all_s.py&quot;, line 7, in forward
    X =  self.layer.doit( X)
  File &quot;all_s.py&quot;, line 19, in doit
    X =  np.matmul(self.matrix1, self.matrix2)
AttributeError: 'LAYER' object has no attribute 'matrix2'



The issue is, when you are calling the forward function on MM your initialized model, self.matrix2 hasn't been defined as the error suggests:

AttributeError: 'LAYER' object has no attribute 'matrix2'

Just to be clear: MM() is equivalent to calling MM.__call__(). You haven't called it, hence the error.
I'm not sure why you would have both an implementation in forward and __call__. In PyTorch the high level API call is made through __call__ which is what you might expect. And, __call__ will call forward as well as trigger registered hooks on the module.
A quick fix would be to define your matrix (the underlying component of your linear layer) in the initialization (i.e. inside __init__). Then, when called the matrix multiplication between the input and that matrix is performed.
class LAYER:
    def __init__(self, h, w):
        self.dim1 = h
        self.dim2 = w
        self.matrix1 = np.random.rand(self.dim1, self.dim2)

    def __call__(self, Z):
        return self.doit(Z)

    def doit(self, X):
        return np.matmul(X, self.matrix1)


Although, something like this would be clearer:
class Model:
    def __init__(self, h, w):
        self.layer = Linear(h, w)

    def __call__(self, x):
        return self.forward(x)

    def forward(self, x):
        x =  self.layer(x)
        return x

class Linear:
    def __init__(self, h, w):
        self.weights = np.random.rand(h, w)

    def __call__(self, x):
        return self.forward(x)

    def forward(self, x):
        return np.matmul(x, self.weights)


x = np.random.rand(2, 2)
model = Model(2, 2)
model(x)

Now you can use additional layers in your Model class. And a backward pass on Linear!
