What role do circular network structures play in neural networks? I'm developing small virtual neural networks and for the most part, the networks have remained feed-forward. When I create circular structures, it becomes a challenge to find a balance with the neural refactory period, stopping neurons from firing again for a period of time, giving them high action potential thresholds, then reducing that back down to fire again -it's tough to find a balance that stops infinite feedback loops but allows circular structures to play a role in processing. 

I've done some reading on this and my vague idea so far is that circular structures perhaps allow the network to react to information based on its pattern, react to the information over time, so the C in an input series of A, B, C can be handled differently than the C in input series A, A, C. 

Is that accurate, basically? What's the role of circular connection structures in a  neural network? I'm asking not only about multi-neuron circular structures like N1 => N2 => N3 => N1 but also about neurons that connect to themselves (N1 => N1) (which from my reading is something that happens if I understand correctly, but I'm unsure whether it achieves a similar affect as a larger scale circular structure).

 (=> represents a neural connection)
In contrast to artificial neural networks, which are almost all feed-forward architectures, networks in biological brains are highly recurrent. In the networks of cortex, the majority of synaptic inputs a neuron receives come from nearby neurons in the same area of cortex (Binzegger et al. 2004). These are the "circular" connections that you are referring to.

So recurrent connectivity is the rule, and simple feed-forward (or feed-back) projections between cohorts are the exception. The networks that result are therefore highly dynamic, and respond in complex ways to inputs changing over time. In general there is no one "reason" why recurrent connections exist. But there are guidelines about how connections between excitatory and inhibitory neurons are made.

The patterns of connections which define a network determine the computational properties of that network. At a first level, the strength and connectivity between excitatory and inhibitory neurons control the overall stability of the network (e.g. Wilson &amp; Cowan 1973; Tsodyks et al. 1997). It's certainly possible to have self-connections and high recurrence, and still keep the network stable.

Secondly, changing the rules by which neurons connect to each other (Ko et al. 2011; Cossell et al. 2015; Lee et al. 2016; Wilson et al. 2017) changes the computational properties of the resulting network (Muir &amp; Mrsic-Flogel 2015).

In general, biological neural networks are highly recurrent dynamical systems which bear only passing relationship to the acyclic networks that we use to think about feed-forward and feed-back connections in the brain.