slow Inference time for Neural Net
I have written a simple Fully connected Neural Network in Pytorch. I saved the model and loaded it in C++ using LibTorch but my inference time is pretty slow for my application field. Inference time right now is about 10 ms. Is it normal or am I doing something wrong?

I measured the inference time on python only first. then to maybe make it faster I loaded the network on C++ but it didn't help.

Here is the code for network 

class network(nn.Module):
    def __init__(self):
        super(network,self).__init__()
        input_nodes = 362
        hidden_nodes1 = 50
        hidden_nodes2 = 30
        output_nodes = 1

    self.fc1 = nn.Linear(input_nodes,hidden_nodes1)
    nn.init.xavier_uniform_(self.fc1.weight)
    self.bn1 = nn.BatchNorm1d(num_features=hidden_nodes1)

    self.fc2 = nn.Linear(hidden_nodes1,hidden_nodes2)
    nn.init.xavier_uniform_(self.fc2.weight)
    self.bn2 = nn.BatchNorm1d(num_features = hidden_nodes2)

    self.fc3 = nn.Linear(hidden_nodes2,output_nodes)
    nn.init.xavier_uniform_(self.fc3.weight)

    self.out_act = nn.Sigmoid();

def forward(self,X):
    X = F.relu(self.bn1(self.fc1(X)))
    X = self.fc2(X)
    X = F.dropout2d(X,p=0.3)
    X = F.relu(X)
    X = self.fc3(X)
    out = self.out_act(X)
    return out


I want inference to somewhat take around 0.01 milliseconds.

How much data did you use for the inference? If it is only a few data points, I think there will be no much difference in execution time between python and C++. Maybe try with much more data? 

Also, the architecture you are using is straightforward; it can probably run in CPU very well for inference. Don't forget to give feedback with your tests! I also want to know what is happening. :)
