How to estimate the computational cost in a neural network? Given a neural network(assuming no regularisation/dropout), I want to determine the computational cost of doing a forward and a backward pass of a datapoint. I want the measure to be of independent of the programming language/operating system involved. I can think of it as a graph model, with some activations applied on nodes, then I think it can be estimated by hand.
Also is there any online tool/package which would do the estimate if I input the neural network ?
I am new to computer science, so probably it is a very naive question to most of the users, I am offering my apologies it is so.
It takes $O(n)$ time, where $n$ is the size of the neural network (e.g., number of nodes plus number of connections; in a graph model, number of vertices plus number of edges).