How to deal with big output values after classification layer during training? In AI libraries such as, Tensorflow, Keras etc., how the big output numbers are dealt during training process? For example, for classification layer, it will have some outputs such as [23.4, -21254.3, 32123.4]. In next step, these numbers will go into softmax function which will take power of base e with each output numbers. However, this may result in extreme big numbers in case of extremely large positive number or extremely small negative number. So, how are extreme cases dealt during training process?
As you stated in your question, those numbers go into a softmax function. Another name for softmax is normalized exponential function. Softmax normalizes numbers where the sum is constrained to be 1 and each value becomes the probability of categorical membership.
In the specific case of [23.4, -21254.3, 32123.4], applying the softmax transforms them to [0, 0, 1]. The numbers become either 0 or 1 because of numerical underflow.