How to deal with ternary Output neurons in the Output classification layer of a simple feedforward Neural Net?
The typical approach for this kind of scenario is to handle the output as a discrete element and have each output be a probability distribution over the discrete output space, that is, for each output you generate 3 numbers between 0 and 1 where the sum of them adds up to 1. You normally obtain this by generating for each position a vector of 3 elements in $\mathbb{R}$ instead of a single element and then passing it through a softmax function.
The typical loss function for that is the categorical cross-entropy, which receives the probability vector (although depending on the implementation and the parameters, it may receive the vector before the softmax, which are called the &quot;logits&quot;), and the index of the expected discrete output.
This is the usual approach for multi-class classification that you will find in most tutorials, like this one.