What roles do circular connection structures play in the brain?
'Circular connections', as you refer to them, are super common in the nervous system. This motif usually referred to as feedback connections, and serves a variety of roles. 

Connectivity studies can be broadly grouped in to those that look at brain areas, and those that look at individual neurons (usually within a single brain area). 

My background is more the latter, so that's what I'll focus on. 

Firstly, your question misses an important distinction: inhibitory neurons. So if neuron C was inhibitory, we would have a negative feedback loop, a classical circuit that can be used to produce a homeostatic like signal, keeping a system within certain bounds; a delayed feedback loop can also act as a 'signal terminator', allowing an action to be stopped once it reaches a certain threshold. Lastly, under some circumstance, feedback loops can produce oscillations, which are applicable to a number of tasks.

But more generally, recurrent connectivity is hugely important in the cortex. The majority of inputs to neurons come from neighbouring excitatory neurons (see https://www.ncbi.nlm.nih.gov/pubmed/7638624), and if A connects to B then B is more likely to connect to A than by chance (see http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1054880&amp;tool=pmcentrez&amp;rendertype=abstract for a really nice study on this).

So why? For a variety of reasons. Networks with feedback mechanisms can do more, is the basic answer; from negative feedback for homeostasis, to persistent network states. My personal feeling is that too much of computational neuroscience models a network as an operator - it takes an image, say, and outputs what it thinks the image is. And whilst that is important, it's not what the brain is. We need to maintain a constant internal model of the world around us, rather than going 'that thing is a cat. That thing is a burger'. Perhaps recurrent activity maintains the cortex in a certain state - knowing that the cat is there - and inputs serve to move the state so that we can update our internal models when the sensory environment changes.