How is it possible to teach a neural network to perform addition?
Welcome to AI.SE @bilanush.

Here's an example approach that might make things clearer. There are other ways to train a neural network to do this however.

In your earlier example with an image, you probably noticed that the network receives the image as a series of values, representing each pixel in the image. The network then learns which of a series of output neurons should be active in response to a given set of pixel values. Those output neurons, when read in an appropriate way, correspond to the correct label for the image. The difference between the set of outputs that should have been active, and the set that were active, forms the basis of the error signal that allows the network to learn.

You've probably heard that computers represent numbers with binary digits. So you could think of the number 16 as being: 00010000 in "8-bit binary". In 16-bit binary, this number would be 0000000000010000, and so on.

So one way of viewing your problem is a function mapping binary inputs to binary outputs (very similar to labelling a black-and-white image). For instance, the input 00010000 (16) should produce the output 00010001 (17). The input 00100011 (35) should produce the output 00100100 (36), and so on.

As before, you will have a set of output neurons. In this case, it should be wide as the set of input neurons. As before, the error signal is the difference between the expected inputs and outputs.

As to the question of how they can learn this function "without plus", in fact the individual neurons in a network perform just two operations: addition of their inputs, and a non-linear transformation of the sum. It has been proven that these are sufficient to learn any function from inputs to outputs, as long as the network contains 3 layers or more, and the as long as the middle layer is wide enough, but here it should be easy to see how addition might emerge.