What role do circular network structures play in neural networks?
In contrast to artificial neural networks, which are almost all feed-forward architectures, networks in biological brains are highly recurrent. In the networks of cortex, the majority of synaptic inputs a neuron receives come from nearby neurons in the same area of cortex (Binzegger et al. 2004). These are the "circular" connections that you are referring to.

So recurrent connectivity is the rule, and simple feed-forward (or feed-back) projections between cohorts are the exception. The networks that result are therefore highly dynamic, and respond in complex ways to inputs changing over time. In general there is no one "reason" why recurrent connections exist. But there are guidelines about how connections between excitatory and inhibitory neurons are made.

The patterns of connections which define a network determine the computational properties of that network. At a first level, the strength and connectivity between excitatory and inhibitory neurons control the overall stability of the network (e.g. Wilson &amp; Cowan 1973; Tsodyks et al. 1997). It's certainly possible to have self-connections and high recurrence, and still keep the network stable.

Secondly, changing the rules by which neurons connect to each other (Ko et al. 2011; Cossell et al. 2015; Lee et al. 2016; Wilson et al. 2017) changes the computational properties of the resulting network (Muir &amp; Mrsic-Flogel 2015).

In general, biological neural networks are highly recurrent dynamical systems which bear only passing relationship to the acyclic networks that we use to think about feed-forward and feed-back connections in the brain.