How does a recurrent connection in a neural network work?
When a recurrent network is calculated you can imagine the network is 'unrolled' out through time.



When visualized this way, you can see activation and loss can be calculated using the same method as a typical network.  

So to clarify using the example you provided. Yes, the input to node #5 is the output from node #1, node #2, and the previous time steps node #4.

It is important to note; Typically the input and output to RNN's are expected to be time dependent. That is, for each time step t there is input t and output t. This makes RNN especially good at things like evaluating sequences. Although it is possible to keep the input and output equal throughout time.