DCS

01 March 2018

New Ideas for Brain Modelling 3
Kieran Greer, Distributed Computing Systems, Belfast, UK.
http://distributedcomputingsystems.co.uk
Version 1.6

Abstract: This paper considers a process for the creation and subsequent firing of sequences
of neuronal patterns, as might be found in the human brain. The scale is one of larger
patterns emerging from an ensemble mass, possibly through some type of energy equation
and a reduction procedure. The links between the patterns can be formed naturally, as a
residual effect of the pattern creation itself. This paper follows-on closely from the earlier
research, including two earlier papers in the series and uses the ideas of entropy and
cohesion. With a small addition, it is possible to show how the inter-pattern links can be
determined. A new compact Grid form of an earlier Counting Mechanism is also
demonstrated. It is possible to explain how a very basic repeating structure can form the
arbitrary patterns and activation sequences between them, and a key question of how
nodes synchronise may even be answerable. The paper finishes with an implementation
architecture, for the realisation and storage of knowledge and memory, as part of a general
design, based on distributed neural components.

Index Terms: pattern creation, synchronisation, neural, grid view, link, reduction.

1

Introduction

This paper considers a process for the creation and subsequent firing of sequences of
neuronal patterns, as might be found in the human brain. The scale is one of larger patterns
emerging from an ensemble mass, possibly through some type of energy equation such as
entropy and a reduction procedure. The links between the patterns can be formed naturally,
as a residual effect of the pattern creation itself. If the process is valid, then the pattern
creation can be relatively simplistic and automatic, where the neuron does not have to do
anything particularly intelligent. The pattern interfaces become slightly abstract without

1

firm boundaries and exact structure is determined more by averages or ratios. If the process
is based on an energy equation that reduces to a more stable state, it may require less
intentional behaviour than, for example, creating clusters in the constructive way of
deliberately linking between nodes. For a large-scale structure this might be preferable. This
paper follows-on closely from the earlier research, including two earlier papers in the series
and uses the ideas of entropy and cohesion. An example of the reactive-proactive
comparison can be carried out using dynamic links [13] or an earlier Counting Mechanism
[14]; but a new compact Grid form is then suggested that may be another general
mechanism. The paper finishes with an implementation architecture, for the realisation and
storage of knowledge and memory, as part of a general design, based on distributed neural
components

The rest of the paper is organised as follows: Section 2 describes the recent and new theory
about the linking mechanisms. Section 3 compares the cognitive architecture with other
research. Section 4 traces though some general classifiers, to show differences in how they
would interpret data. Section 5 suggests a repeating process that can create and activate
patterns as part of the architecture, including pattern resonance. Section 6 introduces
another new structure that tries to match earlier research under the idea of neuron pairing.
Section 7 then describes a specific implementation plan for the lower knowledge-based half
of the architecture, while section 8 gives some conclusions and discussion on the work.

2

Linking Patterns

A key point to the theory is that a lot of the neurons inside a pattern link with each other. It
is not the case that there is a single path from one neuron to the next, but there are lots of
links between all of the neurons. The test theory of [2] also used this principle and it is also
assumed in the tests of [10]. While not every node needs to connect with every other node;
if most nodes connect with each other, it is quite a good way of defining the pattern shape.
It could be imagined that the nodes still have connections to other places, but when excited,
the pattern group will reinforce itself sufficiently to make it significant, while other more
sparsely connected sets would possibly die out. It is a well-known phenomenon, as

2

described in the related work section. It would also be the case that closer ‘patterns’ would
prefer to connect with each other. This would then also refer to more similar concepts,
when the whole process can be more consistent with state transitions and such things. The
papers [10][7] have already considered the general mechanisms and used energy with
entropy as the measures to describe how the patterns might form. This paper extends those
equations slightly to include the inter-pattern connections.

2.1

Earlier Research

Earlier papers have also discussed pattern creation and linking, under different levels of
complexity. The very first cognitive model [12] proposed a 3-level hierarchy of increasing
complexity and functionality. The top level would cluster similar concepts (or neuron
patterns) and then allow one group to trigger another group through a reinforcement link,
created from time-based events. In Figure 1, for example, the top level of the original
cognitive model is shown. A trigger probably means something firing in sequence instead of
in parallel and represents a state change. This top level consists of higher level concepts,
drawn as the octagonal shapes. They are higher level because they group together lower
level patterns. The idea of a concept is abstract and so the term could ultimately reduce to a
single neuron, which might not be very helpful.

Figure 1. The Original Cognitive Model [12] top level clustering.

The model has been tested a certain amount, but it became clear that the trigger link
between the clusters was not much different to a single node link. It would be created in
essentially the same way, through reinforcement. This top level was expanded in Figure 4 of
[10] so that the other models of concept trees [9] and the symbolic neural network [13]
3

could be incorporated. The concept trees can be considered as more knowledge-based, or
even static memory structures. The symbolic neural network again represents symbols or
concepts, but performs the same function as Figure 1, and would use time-based events to
dynamically link lower level concepts into high-level ones. See the paper for more details
about this. These two structures have been joined at a time-based events layer, where
Figure 1 probably sits at the time-based layer and above. This is indicated in the figure with
blue regions that circle the concepts that fire together. One can imagine a stimulus
activating a number of these trees that then fire in parallel. This might then activate other
patterns in sequence.

Figure 2. Concept Trees with the Neural Network Structure [10].

As the mechanisms are general, they can be repeated, depending on the scale for example,
where a single concept is still a small pattern, and so on. A hierarchical structure is also
assumed in most models [16], as is a difference between the physical space and the logical
space. A hierarchy is useful because it can be a more economic structure and it can also
perform a different function at each level, but for this paper, it is mostly about the
structure. So Figure 2 combines dynamic experience-based at the top with static knowledgebased at the bottom. The very top GC global nodes in the neural network represent the
accumulated branches and can trigger other global nodes. For state changes or firing
4

differences to be recognised, a time factor has to be included. Let a source stimulus activate
groups of neurons that then fire in parallel, but a state change might require the activation
of patterns not initially connected with the stimulus. This is then also a sequential process as
the energy source would come from the activated patterns and not the original stimulus,
and could change the stimulus result. In Figure 3, for example, there are two connections
between the patterns, which might happen at different levels. This figure is used again in
following sections.

Figure 3. Two pattern ensembles with residual inter-pattern links between them.

2.2

Reactive Linking

Reactive linking may have more relevance where patterns sequentially and laterally activate
each other, rather than try to form a hierarchy. Assume for example, that nodes in a region
fire in some arbitrary manner. This would produce a larger mass or ensemble out of which
distinct patterns can emerge and it is more of a reactive than a proactive process. These
patterns are defined in physical space by the fact that their nodes have more links between
each other than with other regions. A simplistic view of the reduction process is shown in
Figure 3, where the patterns themselves are defined because they have more intra-pattern
links and the inter-pattern links are the dotted lines between them.

If a pattern is mostly all connected, then one scenario is a small but persistent energy input
that eventually stimulates all of the other nodes. Another scenario is a much larger stimulus
that is able to trigger the pattern over a much shorter period of time. Therefore, depending

5

on what patterns fire and the firing strength, this can cause other patterns to be more or
less important, and be immediately realisable or realisable only through time.

2.3

Circuit Recognition

One thing missing from the model of Figure 2 is a clear description of the closing cortex
node, as described in Hawkins and Blakeslee [16] , chapter 6. They describe the cortex as
having 6 layers and a visual process, for example, uses 4 layers where 3 layers form a
hierarchy with the fourth (IT) layer joining this up and recognising the lower-level patterns
as a significant feature. It completes a circuit which also helps the signal to say active. The
author’s current model is at a slightly coarser scale than this, but it is possible to see this
type of hierarchy in it. Concept trees look like a basic diverging structure in 2-D, but could
overlap and be more complex in 3-D. Strangely, as shown in Figure 2, the concept trees may
be the indexes to a shallow Concept Base [13] hierarchy, represented by the blue timebased layer. That layer could also compare with the top hierarchy layer in the visual cortex,
for example. The closing (IT) nodes of the visual cortex would then be the first level of the
symbolic neural network, as is shown in the figure. This is of course, not intentional, but at
least the different ideas can then have some overlap.

3

Related Work

Physical systems are included in the related work, because this research is interested in how
global structures might change, not individual events. Other topics are neural networks, biorelated and self-organising. A review paper presents a philosophy that has lots of
comparisons with the author's own work. The main problems that the paper notes are the
knowledge homogeneity problem (what heuristic to choose) and the integration of
common-sense contextual knowledge, which are both required to allow more generality in
the model.

6

3.1

Global Properties

Weisbuch [29] describes the properties of a chaotic system inspired by statistical physics.
One theory written about states that even if there is randomness or disorder at the local
level, there may still be order at the global level. The disorder itself can repeat in a
consistent way. The Physics model of percolation describes this phenomenon and allows for
insulators or conductors between local nodes, but does not require a regular order at that
level. He writes that:

‘The simple models used by physicists are based on periodic networks, or grids, and
simplified components of two different types are placed on the nodes, such as for
example conductors or insulators in the problem known as percolation. These
components are randomly distributed, and the interactions are limited to pairs of
neighboring nodes. For large enough networks, we perceive that certain interesting
properties do not depend on the particular sample created by a random selection, but
of the parameters of this selection. In the case of the aforementioned
insulator/conductor mixture, the conductivity between the two edges of the sample
depends only on the ratio of the number of conductive sites to the number of
insulating sites’.

Therefore, the interpretation of the result is as much about ratios or percentages than fixed
boundaries. Each single interaction becomes only a small percentage of the desired result
and therefore, if ‘more’ of the interactions are correct than incorrect at a global level, the
desired result can be achieved. It may also be about the type of neuron behaviour. The idea
of ‘neurons that fire together wire together’ is the well-known doctrine of Hebb (mentioned
in [17]) and is central to how the patterns might form. Weisbuch notes that using Hebb’s
rule in a neural network will result in the network attractors representing the patterns. The
system reduces to the state of the attractors, which are the sequences of repeating states.
Memories can also be created this way.

Even if this research considers structure and not function, in biology, Interneurons are
created for just that purpose [18] – for statistical connectivity and not functionality. Their

7

results indicate that statistical connectivity can account for much of the specific synaptic
patterning between neurons. Chemospecific steering is still essential and can tweak things,
but it is non-specific – not aimed at any particular pair of neurons. They show that the exact
positioning of neurons within their layer is not critical but that some intentional direction is
possible. They note that the model would predict head-on collisions between neurons,
where a chemspecific signal can steer axons and dendrites around them. The paper [23] is
more computer-based and describes tests that show how varying the refractory (neuron
dynamics) time with relation to link time delays (signal), can vary the transition states. They
note that it is required to only change the properties of a small number of driver nodes,
which have more input connections than others and these nodes can control
synchronisation locally.

3.2

Earlier Research

Earlier research ideas that are relevant include [6] -[14] and are mostly concerned with how
the system as a whole would work. The ReN [11] stands for Refined Neuron. It is a new idea
for re-balancing the network, by converting excess energy into new intermediary neurons.
They activate first and then activate the ‘main’ neuron, before it can send its signal. This can
refine the neuron values, because they become fractional parts of the total input signal.
With the ReN however, it is a proactive process, where surplus energy forces new
connections to re-balance the system. It also suggests a backwards flow down the channels
that increases pressure and stimulates new growth. The theory of an internal force is
controversial, but even if only the growth process is used, it is interesting that the different
parts are consistent mathematically. The ReN starts with the idea that there is an excess of
energy. This excess energy is created when more than one input neuron fires at the same
time. This overloads a main neuron that still receives energy when it is recovering from its
firing event. Neurons firing together includes the idea of resonance among the activated
synapses (axons/dendrites), which then induces new paths to grow. The new neuron
therefore represents some type of (sub)concept created by the firing group. And the
process is necessary to re-balance the system and reduce entropy, or improve efficiency. If
looking at the definition of entropy as part of thermodynamics, it is stated in terms of how
much energy in a system is no longer available for conversion into mechanical work.
8

The new theory of this paper, described in Figure 3, considers a more reactive process over
a larger domain. The residual effect of a persistent general stimulus keeps certain patterns
alive, but these would form in centres of attraction. A section of [7] is quite interesting and
considers how the structure might be determined: ‘It is easy to understand tree structures
getting automatically narrower, but to broaden out requires the deliberate addition of new
nodes and links to them. Re-balancing is always an option, where excess signal might
encourage new neurons to grow, as in a ReN. Or many neuron clusters can interact and link
with each other, but still provide specific paths into their own individual set of nodes. An
idea of nested patterns might also help. Smaller or less important patterns at the periphery
can be linked to by a more common mass in the centre, for example, leading to a kind of
tree structure. In which case, it can be less of a deliberate act and more the residual result
of a region being stimulated in a particular way.’

3.3

Biology-Related

The work of Tkačik et.al. [25] could be relevant. It uses statistical mechanics to try to explain
some of the mechanisms that occur in the biological brain. They note that while the brain is
not a thermodynamic system, it exhibits some of those properties and so it can be
measured that way. They also note that the brain is a nonequilibrium system, but the theory
of the ReN helps to adjust the system into more equilibrium. If dealing with a critical state E,
they suggest that you can count the number of potential states that have energy close to or
less than E and consider those as the degrees of freedom or entropy. They then show how
their results suggest a thermodynamic limit to the neural activity, but have no definite
explanation of why. The limit suggests a boundary and so possibly, inhibitors switch off
activity to prevent saturation ([27], figure 4), or if the activity is mostly inwards, then a finite
outer boundary will exist, which can produce a more closed system. Note that firing would
also be timed, where an initial energy surge might eventually be switched off as inner
patterns fire inhibitors back into the surrounding area. Their results then show that energy
and entropy are linear with respect to each other. The model of this paper is consistent with
that energy-entropy relation. If base patterns fire before nested ones, or as in a tree
structure, patterns further along a path, then the base patterns necessarily require more
9

energy as they fire first and fuel the other sequences. Also, if they are at the base, then
there are many more potential states to move into. After a sequence is activated, patterns
further along a path would then require less global energy to be activated and would
provide fewer new potential states to move to. Size is not a problem, because the nested
regions can scale with the larger base regions and the whole process is self-repeating, as
described in section 5. They measured spiking events of individual neurons and so it does
not mean that these structures were present, but if the relation is valid, then it would be
applicable in a general sense.

Also interesting are the theories of Neural Oscillations and Neural Binding1. These brain
theories are used to help to explain how the brain is able to comprehend, what it sees, for
example and therefore overlaps with thought and consciousness. With one visual system
theory (Gray, Singer and others [24]), synchronous oscillations in neuronal ensembles bind
neurons representing different features of an object. Gestalt psychology is also used, where
objects are seen independently of their separate pieces. They have an ‘other’ interpretation
of the sub-features and not just a summed whole of them. Chen uses the idea that
perception depends on ‘topological invariants that describe the geometrical potentiality of
the entire stimulus configuration’. The idea here is that looking at a certain object creates a
stimulus which in turn fires a constellation of that object, and then the object becomes
recognized. The rebuttal is the fact that independent constellations would require too many
neurons and pathways [21]. Singer and Gray write that:

‘Representation of a feature by a population of cells raises binding problems when
nearby contours evoke graded responses in overlapping groups of neurons. Of the
many simultaneous responses, those evoked by the same contour need to be
distinguished and evaluated together to avoid interference with the responses elicited
by neighboring contours. A similar need for response selection and binding arises in
the context of perceptual grouping.
…

1

See Wikipedia, for example.
10

Essentially, one cell would be required for every distinguishable feature, for each
higher-order feature combination, and ultimately for every distinguishable perceptual
object. Moreover, because of its inherent lack of flexibility, such a mechanism cannot
easily cope with the representation of new or modified patterns.’

They also note that shared patterns would have many advantages, especially with neuron
numbers, but a flexible process is required to clearly recognise when parts of a pattern fire.
They then go on to write:

Simulation studies by Softky & Koch (1993) suggest that the interval for effective
summation of converging inputs is only a few milliseconds in cortical neurons. Thus, if
synchronization of discharges can be achieved with a precision in the millisecond
range, it can define relationships among neurons with very high precision. Moreover, if
synchrony is established rapidly and maintained only over brief intervals, different
assemblies can be organized in rapid temporal succession.

This information is interesting with respect to a new structure suggested in section 6.

3.4

Neural Models

Neural networks are the most obvious type of distributed model. The paper [4] is interesting
and may even be trying to implement something similar in hardware. They note that
electrical synapses are bi-directional, unlike the chemical ones, but this means an
intentional signal in both directions and not an excess one. The term small-world effect [28]
has similarities with pattern groups connected by residual links and is described as part of
their new Hopfield network [19][1] design:

‘It is known that brain neural network is comprised of millions of neurons and their
connections. Major connections are of short distance and the number of long-distance
connections is less, which is acclaimed as the so-called small-world effect. Recently,
the small-world neural network model was thus inspired and proposed.
…
11

More specifically, it has greater local interconnectivity than a random network, but the
average path length between any pair of nodes is smaller than that of a regular
network. The combination of large clustering and short path length makes it an
attractive model capable of specialized processing in local neighborhoods and
distributed processing over the entire network.’

The Hopfield neural network, and its stochastic equivalent are auto-associative or memory
networks. With the memory networks, information is sent between the input and the
output until a stable state is reached, when the information does not change. These are
resonance networks, such as bidirectional associative memory (BAM), or others. But they
can only provide a memory recall of the data that was input and are also constrained in size
to that amount of data. If some of the input pattern is missing however, they can still
provide an accurate recall of the whole pattern. They also prefer the data vectors to be
orthogonal without overlap. After the different stable states have been learned, they
represent a type of energy function. A new pattern may be associated with a set of the state
vectors, but one state will be closer to it, which will produce the best match.

Hopfield networks are also attractive because the units can operate asynchronously of each
other. Each unit can compute its excitation at random times and change its state
independently of the others. Also described in [22], chapter 13:

Hopfield’s approach illustrates the way theoretical physicists like to think about
ensembles of computing units. No synchronization is required, each unit behaving as a
kind of elementary system in complex interaction with the rest of the ensemble. An
energy function must be introduced to harness the theoretical complexities posed by
such an approach.

But while the learning can be asynchronous, the weights and links are still very directed,
although self-organisation and an energy function can maybe take precedence over a
supervised learning approach. Watts and Strogatz [28] then took the bi-directional links and
replaced it with a circular architecture to produce the small-world networks. More
importantly, the nodes are not all linked with each other, or completely randomly linked,
12

but somewhere in-between. The linking pattern would still provide more group support
than a random setup, for example. Considering Figure 2 and Figure 3 of this paper, is it
possible to see small-world networks at the time-based events layer? If not, then it is still
another look-alike structure to relate to, as possibly is [23].

Other types of model include Cognitive Architectures. The paper [20] is very interesting and
uses a more knowledge-based vernacular to describe other cognitive systems. For example,
where the current research has used the terms knowledge or experience-based, the paper
[20] may use the terms declarative or procedural knowledge, to mean probably the same
thing. The paper also describes that solutions to the problem have to implement
prototypes, exemplars and/or theory-theories structures. It shows that a number of other
cognitive systems have implemented these artefacts and that a general theory needs to
accommodate them. It will be shown in section 7 that the author’s model can also be
described using the more knowledge-based sets of terms and that the current work can
accommodate the required transformations, to allow the information to flow seamlessly,
from pattern-based data into knowledge.

4

Cluster Interpretations – Global Pattern vs New Instances

This example shows how even very basic reinforcement mechanisms can produce very
different interpretations of pattern sequences, as described using the author’s own
clustering algorithms. Consider the following example: There is a set of nodes A, B, C, D, E, F
and G. Input patterns activate the nodes as follows:

1.
2.
3.
4.
5.
6.
7.

A fires with B, C, D and E.
B fires with A, C and D.
C fires with A, B and D.
D fires with A, B and C.
E fires with A, F and G.
F fires with E and G.
G fires with E and F.

13

4.1

Counting Mechanism with New Instances

A counting mechanism [13][14] has been suggested previously. With basic reinforcement, a
weight is incremented when a variable is present and decremented when it is not. With the
counting mechanism, a weight is incremented when a variable is present and a grouprelated weight is incremented when any variable in the group is present. If considering the
counting mechanism, then the first set of counts, shown in Table 1 would occur if the whole
group A-G was updated as a single entity. The global count (G) is updated for every pattern
presentation and the local count (I) is updated only when the node itself is presented. This
would suggest that the best cluster group is B, C, D and E, but that is not correct.

A

B

C

D

E

F

G

I

5

4

4

4

4

3

3

G

7

7

7

7

7

7

7

Table 1. Counting Mechanism when updating all variables together.

If each pattern instance is presented separately and updated separately however, then a
different result is shown in Table 2. Using some basic rules, it is possible to calculate what
the correct groups are, as in: A, B, C, D and E, F, G, by comparing the local and global counts.
The possibility of a joining link between the two groups is not very clear however, unless
you compare variable names and so the next section shows a more compact form of the
counting mechanism, where it is converted into a grid format.

A, B, C, D, E

A, B, C, D

E, F, G, A

E, F, G

I

1

3

1

2

G

7

4

3

2

Table 2. Counting Mechanism for individual group instances.

14

4.2

Grid-Based Entropy Equation

It is clear then that A, B, C and D all reinforce each other (pattern 1), as does E, F and G
(pattern 2), but there is still an inter-pattern link between A and E (in both pattern 1 and 2).
With the grid format, the structure is a single pattern group again, but the count updates
are different and for each individual instance, as shown in Table 3. The grid format lists the
variables both as rows and columns. Each time a pattern is presented, the related cell value
for both the row and the column is incremented by 1. The grid result can then be read as
follows: Each row represents the count values for a single variable. The update process
updates every variable in the presented pattern each time, which means all crossreferencing cell values for every row in the pattern. To determine the best clusters then, for
a variable (row) scan across and select the other variables with the largest count values and
their rows can then be checked for consistency – they should have similar count values for
each variable. If one has a larger or smaller count value, then it probably belongs to a
different cluster. The variables with different counts might then be considered for linking
between patterns, where they have some association but belong to a different pattern.
Note that each entity (grid entry) is still individual and is represented by its occurrence in
distinct time-based events.

A

F

C

D

E

B

G

A

x

0

4

4

2

4

0

F

0

x

0

0

3

0

3

C

4

0

x

4

0

4

0

D

4

0

5

x

0

4

0

E

2

3

0

0

x

0

3

B

4

0

4

4

0

x

0

G

0

3

0

0

3

0

x

Table 3. Display of the reinforcement between pattern presentations grouping A, B, C and D,
plus E, F and G, with a single inter-pattern link A-E.

15

With the example set of events, the nodes would also be beside each other in the grid, so
the variables have been mixed up in the grid placings to show that the correct clusters can
still be determined. It may not be necessary to update a self-reference, but for each row,
scan across and find the other variables that have the largest count values. In row A, for
example, the counts suggest that it should be clustered with B, C and D, even though they
are not placed beside variable A in the grid. This grid format is actually a transposition of the
counting mechanism. In the Counting Mechanism [14], the global count represents the
global pattern, or the global pattern bit for each node, while with the grid, it is the whole
grid. The Counting Mechanism updates unique instances, which suggests an attempt to
create new patterns. The grid splits an existing pattern, although it might be possible to add
variables to the grid dynamically, which the counting mechanism accommodates.

A different test tried to add keyed entries for each new pattern instance, but also allow for
nested patterns inside of an existing instance. That did not prove successful, suggesting that
the original counting mechanism is correct to create a new instance for each unique pattern
that is presented. As the grid does not reply on prior classifications of the categories, it is
really a self-organising mechanism for categorical data.

4.3

Test Examples

If considering the new grid structure of section 4.2, then collaborative filtering, such as the
Entree Chicago Recommendation Data Set [3][5], may be relevant. The action or request of
a user of the system was coded and could be interpreted as a category. Then each user
session could be a group of related categories. A basic test program can then count the
frequencies that the categories occur together, as in the grid example, and create clusters of
categories that more commonly occur together. These can even be related back to the
restaurant or user, but there are no clear results to compare with. Another test that did
produce results was the Plants dataset [26]. It was used in [15] to mine for association rules,
which are similar as the data can be a set of binary events to associate together. With the
dataset, different species in each state of USA and Canada are listed and the states were
clustered on similar species. The results, given in Appendix A, do appear to show clustering
based on geography or climate. One problem was highlighted. If a category ended up by
16

itself, not clustered with any other category, then with the current algorithm, it is not
obvious how to cluster it with another category or group. A more detailed examination
would be required to give a good suggestion for that. However, the clustering process is a
one-pass process and took only a few seconds to complete, on a standard laptop.

5

Repeating Structures and Sequences

It is now clear that a general process can be described for how patterns may form and link
with each other. It uses a structure that can repeat at different levels of granularity and can
form in an automatic and arbitrary way. It is also completely mechanical in nature, requiring
no sense of real intelligence. Throughout the research, similar processes have repeated,
giving some confidence to their relevance. At the finest level of granularity, there can be
single neurons with links between them. These can group together to form a pattern, which
is what a concept is more likely to be. The formation process can leave weaker or residual
sets of links between patterns, as described in section 2, that actually act as controllers of
state transitions, to activate the patterns in sequence. If considering Figure 1, for example, it
would be possible to replace the octagonal nodes with a pattern of neurons and the single
links with the residual set. That whole figure can then be a higher-level concept made of
lower-level ones, or a sub-process, or anything that is a sub of something else. This still
leaves the trigger link between the larger concept patterns. So why not make that a residual
set of links as well, but between two even larger concept groups. The nesting and time
element in the structure is also obvious and a repeated process can occur to activate the
required transitions.

For example, if an area of the brain is activated, then one pattern may fill with signal before
another pattern. After a while, the pattern can activate an associated one through the
residual set of links, but as that is a weaker connection, it would occur after the pattern
itself is activated. Statistically, signal travels through the pattern itself more quickly than to
another one. The pattern may also throw inhibitors out into its environment, to switch off
neighbouring patterns and make it the dominant one. So, a larger region is activated and
can switch off competing regions. It may also contain nested structures [10][7] that would

17

represent sub-concepts and as the region fires it activates some of the sub-regions. These
sub-regions may also be linked in a similar way and repeat the process. To be completely
successful however, they would also need longer lateral connections with other regions.
Therefore, lateral links between pattern groups and regions can come from any of the
activated neurons.

5.1

Cohesion and Resonance

The equations and method of paper [7] are also relevant. The values in Table 3 really only
come from the reinforcement equation 1 in that paper, but in the grid view, the
reinforcement value can be split-up for each separate variable. Note that the equations of
that paper do work with the table values. The entropy equation 8 in [7] is used to determine
pattern membership through a similarity across the node counts. It uses the idea of a
variance value scaled by the number of node events. This is important because different
node sets might be presented each time and then averaged to give each node the same
overall count, but each node might not be presented each time. An even count, or a 0
variance, is then preferred.

An earlier paper [10] noted that the pattern nodes might like to be evenly spaced between
each other with regard to distance, as described in section 5.2 of that paper. This also has
some relation with the scaled variance idea. For example, if we have 4 nodes spaced along a
single path, then normalised distance values of 3, 3 and 3 would be preferred to something
like 3, 1 and 5. The argument considers the actual firing sequences of each node and is
therefore multiplicative. Neuron 1 must fire x times before it activates neuron 2, that must
fire x times before it activates neuron 3, and so on. This would also magnify any small
differences between distance setups. For the example therefore, if the distance numbers
are multiplied together, then the first set gives the value 27, while the second set gives the
value of 15. So, an equal set of normalised distances will give the largest value and an
optimisation equation could make use of that fact. There is no known reason why a real
neural setup would have to be normalised however and so why not have distances of 3, 3
and 5 instead? One reason will be the synchronisation argument given next, but another
would be to use the variance equation 8 as the actual scaling factor. Therefore, if the mean
18

is 3.6, then the standard deviation is 2.6 / 3 = 0.8666 and variance is 0.1333. Even for a
computer simulation, it might be an interesting possibility.

5.2

Synchronising Nodes

An even variance also makes a lot of sense when considering node synchronisation inside
patterns. If a root node activates a number of linked nodes, then if they are at an equal
distance from the root node, they will all fire at the same time. If they subsequently do the
same, then the nodes that they link to will also fire at the same time. Therefore, it is not just
the fact that closer nodes cluster together, but rather that nodes of an equal distance from
each other cluster into patterns. It is also helpful if the linked nodes feed back into the
pattern again, so that they can activate the other pattern nodes with the same time interval
and the whole firing sequence will be synchronised. Consider a basic example of a single line
of nodes, where the node threshold is 5 and a node can fire every 1 second. Node 1 fires at
time t1, then node 2 can fire only at time t5 and node 3 can fire only at time t25. So if these
nodes were to fire at the same time, they would require extra input, to trigger them in a
single time event. Therefore, the feedback from each node to other ones in the same
pattern would be required. The feedback links would not only define the pattern shape but
also determine the firing rates, which would help the pattern to resonate. With so many
connections however, saturation would be likely and so that would need to be managed.

Another set of nodes may also be evenly spaced, but at a different distance, when they
would synchronise differently and fire at a different rate. If the distance between the nodes
is compared with a wavelength or maybe a frequency, then with different pattern groups in
the same region, the brain can receive variable signals and the combinations could help with
ideas about thought or consciousness.

6

Neuron Binding

The author would like to introduce another new structure here, shown in Figure 4. It is not
necessarily part of the human brain, but it fits in with the theory of these papers and is
again able to match-up some of the equations. In particular, the mechanism described in [8]
19

could be relevant. Biology has already suggested theories about neural oscillations and
binding that include neural pairing. The pairing helps to group the neurons into specific
patterns that the brain can understand, when different features can become synchronised
and oscillate together. This new structure also relies on pairing neurons and may be loosely
related to the earlier biological research. With this, two neurons would be used to represent
an entity. One neuron exists in a base ensemble that is a flat structure and the other would
exist in a hierarchical structure. Neural duplication is well-known about, but this structure
would have duplication at a very close range. To find this structure therefore, some of the
base neurons would be activated and probably realise an ensemble activation. That gives a
much larger area to stimulate initially. The hierarchical structure then provides more
meaning to the patterns that may fire together.

Therefore, the base ensemble fires and activates neurons. If these are paired with neurons
in the hierarchical structure, then the tree structure can also be activated. The hierarchy
would likely send its signal somewhere else as well, possibly to the top level neural network.
Negative feedback can switch neurons off again, when they will not fire during the next
iteration and subsequently switch some of the tree off, but that can then be re-activated
through a sustained base signal, see the next section. The more structured upper layer still
gives more definition and further details about this simple structure are given in [6]. The
base ensemble could therefore be compared with the bottom level of the original 3-layer
architecture [12] that also tries to optimise its links, while the tree could fit in the middle
aggregating layer. As described in section 2.3, this more accurate structure then acts as an
index layer for the base neurons in the more intelligent cortex region. The middle level is
more deliberate and knowledge-based, while the upper level is more reactionary and
experience-based.

20

Figure 4. Neuron Pairing can be used in lower half of Figure 2 as Concept Tree, or possibly
inverted, in upper half as Neural Network.

6.1

Enclosing Reactivation

The paper [8] considered what was thought to be a new mechanism, where if a neuron was
inactive, it would be activated as determined by its enclosing ensemble, but if it received its
own signal – positive or negative – that would determine the behaviour. Therefore, even if
negative feedback flows from above, if some neurons are inactive, they can still switch back
on again as part of the larger ensemble. So, there can be a temporal and controlled element
to the signalling process. If there is a question about stabilising a signal, then this
mechanism can repeat an activation signal, where the repetition would make it significant. It
also makes the supposed new mechanism of [8] look a bit more obvious, but the exact
mechanics of it were not thought about previously. So, to summarise: there are a number of
local circuits that can be managed through the neuron pairs, as well as the larger enclosing
patterns that can consistently reinforce and sustain a signal and the hierarchy provides
some structural meaning, while the base ensemble makes the region easier to find.

21

7

Implementation Details

This section describes how it is possible to go from the neural ensembles to the structured
trees using the computer program components of the current research. Each
transformation stage is also seamless and automatic, where a model for the lower, static
half of Figure 2, has emerged. While it is not fully implemented or complete yet, it is
essentially the design described in Figure 4. So, we start with the neural ensembles or
ensemble mass, where patterns of nodes periodically fire together. Each firing sequence can
be vectorised and treated as a single event [20]. The author has originally used a
reinforcement linking mechanism [13] to represent the strength of these firing sequences. It
was then shown that a counting mechanism could represent the same thing [13][14] and in
this paper, the counting mechanism has been transformed into a grid-like entropy heuristic.
Entropy is a good idea for this type of clustering and the grid can probably be represented
by a distributed architecture, so it has not changed the cognitive nature of the system
particularly. The grid is much more efficient for a computer program, for example. So the
point is that the ensemble mass can be automatically transformed into a vector-style of
structure, with weighted sets of concepts or features. What appears to be missing from
other designs is contextual information. The linking mechanism could provide this as the
path information, for example. Each word term in the vector could therefore have a
contextual set of words associated with it. The context can be any type of input – the
sensorised environment, for example. The other problem of homogeneity or selecting the
correct heuristic is still unsolved, because that requires a higher level of intelligence and for
this model, it would require the same process to manifest itself in different ways using the
same underlying structures. Although, if the brain itself is modularised and each part is only
slightly different, the problem becomes one of merging the resulting signal sets, rather than
making an intelligent decision. If using the counting mechanisms, then the descriptive
information needs to be added, but that should be possible if it is available.

So, we now have vectors of weighted terms with related metadata. These could be stored
and then added to the concept base as tree structures. A process that probably also occurs
in biology could be useful here. If a neuron is firing more often, then it is more likely to

22

create a new neuron. This would map directly to the frequency count or weight in the term
vector. Therefore, the terms with the largest values should be realised first in the concept
trees as the base nodes. The next largest values should form the next level and so on, where
it only remains to link up the different levels with branches. So, this process is clearly
seamless and has transformed a neural mass into a structured tree. With contextual
information, depending on the dynamic nature of it, the more static tree can be traversed in
different ways and therefore produce different exemplars. The final solution is still unclear
however, where different value sets on the same structure is also a problem.

The design just described relates to exemplars and prototypes as well. Possibly the theories
should be placed with the symbolic neural network at the top and so they do not have to be
considered. What would be the prototype and what would be the exemplar? The vectorial
structure is a set of concepts/features, or whatever, with related frequency values. This is
therefore a prototype structure of what a larger concept requires. An exemplar is an
example of this to compare with, which can in fact be the tree structure. If stored as a
memory, then it can be retrieved and quickly mapped to. It may then require smaller
adjustments to map exact values, where a basic tree template has been provided. A higher
frequency in the ensemble node could trigger lower level nodes in the tree, that can
successfully activate the branches and realise the base concept of the tree.

8

Conclusions and Discussion

This paper has described how patterns may form and more importantly link with each other
through a general reduction process of a larger ensemble mass. The process can make use
basic statistics and entropy equations [7] and here, the process considers inter-pattern
sequences. The signal strength inside of each pattern is stronger than between patterns,
which helps to define it. However, time can be used to build up a signal, or more or fewer of
the connections can be firing at the same time and so ratios or percentages are key. The
influence of the input stimulus or the desired result are also factors. To show how the
patterns and inter-pattern links might be realised, a Grid view of a global ensemble, updated
by a series of pattern presentations has been demonstrated. This is in fact an alternative

23

view of the Counting Mechanism [14], where possibly, one method is for separate instances
and one is for the global view. The Grid format works well with event-based or binary input
and successfully clustered some plant data.

The pattern-forming reduction process is much less organised than the concept trees and
symbolic neural network of Figure 2 and earlier research, but that is OK, as it defines a larger
area that can make use of probability and allow errors on a smaller scale. The trigger that is
indicated in the diagrams is similar to a basic link, except that it is between two sets of
patterns, not nodes. As the structure gets smaller however, it becomes more precise and a
hierarchy can be created by repeating the clustering at different levels. A general process
for the pattern creation, binding and state transitions has therefore been explained that
includes time and nesting as fundamental to the structure. This has also been described in
terms of the current computer programs and components. How this interacts with the more
dynamic, symbolic neural network at the top, is yet more research. It has also been
suggested that distance between the nodes could be a vital part of synchronization and as
with time, could help patterns to resonate their signals. The grid process of matching similar
counts would also help here, as the linking structure between the nodes is then more
uniform. As this is an automatic process, it then asks some questions. If pattern resonance is
based simply on node distance, then why would a search process find it or not find it? If the
construction process is automatic and some thought processes return the patterns, why
would all thought processes not return the related patterns? Why is some information more
difficult to find?

The whole design is based strongly on the distributed neural architecture of the human
brain and as such, the author would argue that it is one of the more cognitive designs. It is
noted that heterogeneity in the brain is still an unknown quantity [20] and real intelligence
is currently missing, but a modular design that works the same inside of each module, could
be an option. As with cellular automata, the same input would then give different results.
This would suggest that a summed and weighted total for a node is insufficient and it should
have some other defining characteristic, as with phenotype/genotype in genetics. The brain
itself contains different types of neurons in the different regions, for example. The author
does not know if the brain can ‘intelligently’ choose particular heuristics, so while it has
24

flexibility, decisions probably have to be automatic and in-built. But combinations from a
number of different modules can still produce any variety of result. The task then changes
from intelligent selection to interpreting the combined output correctly, which is still an
unanswerable question.

References
[1] Ackley, D.H., Hinton, G.E. and Sejnowski, T.J. (1985). A Learning Algorithm for Boltzmann
Machines, Cognitive Science, Vol. 9, pp. 147-169.
[2] Anderson, J.A., Silverstein, J.W., Ritz, S.A. and Jones, R.A. (1977) Distinctive Features, Categorical
Perception, and Probability Learning: Some Applications of a Neural Model, Psychological
Review, Vol. 84, No. 5.
[3] Burke, R. The Wasabi Personal Shopper: A Case-Based Recommender System. In Proceedings of
the 11th National Conference on Innovative Applications of Artificial Intelligence, pages 844-849.
AAAI, 1999.
[4] Duan, S., Dong, Z., Hu, X., Wang, L. and Li, H. (2016). Small-world Hopfield neural networks with
weight salience priority and memristor synapses for digit recognition, Neural Computing and
Applications, Vol. 27, Vol. 4, pp. 837 - 844.
[5] Entree Chicago Recommendation Data Data Set, http://archive.ics.uci.edu/ml/datasets/
Entrée+Chicago+Recommendation+Data. (last accessed 14/2/17)
[6] Greer,

K.

(2017).

New

Ideas

for

Brain

Modelling

4,

available

on

arXiv

at

https://arxiv.org/abs/1708.04806.
[7] Greer, K. (2017). A Brain-like Cognitive Process with Shared Methods, Int. J. Advanced
Intelligence Paradigms (IJAIP), Inderscience, accepted for publication, available on arXiv at
http://arxiv.org/abs/1507.04928.
[8] Greer, K. (2016). A Repeated Signal Difference for Recognising Patterns, BRAIN. Broad Research
in Artificial Intelligence and Neuroscience, Vol. 7, No. 3, pp. 139 - 147, ISSN 2067-3957 (online),
ISSN 2068 - 0473 (print). Published on arXiv at http://arxiv.org/abs/1604.05170.
[9] Greer, K. (2014). Concept Trees: Building Dynamic Concepts from Semi-Structured Data using
Nature-Inspired Methods, in: Q. Zhu, A.T Azar (eds.), Complex system modelling and control
through intelligent soft computations, Studies in Fuzziness and Soft Computing, Springer-Verlag,
Germany, Vol. 319, pp. 221 – 252, 2014. Published on arVix at http://arxiv.org/abs/1403.3515.
[10]Greer, K. (2014). New Ideas for Brain Modelling 2, in: K. Arai et al. (eds.), Intelligent Systems in
Science and Information 2014, Studies in Computational Intelligence, Vol. 591, pp. 23 – 39,
25

Springer International Publishing Switzerland, 2015, DOI 10.1007/978-3-319-14654-6_2.
Published on arXiv at http://arxiv.org/abs/1408.5490. Extended version of the SAI'14 paper,
Arguments for Nested Patterns in Neural Ensembles.
[11]Greer, K. (2016). New Ideas for Brain Modelling, IOSR Journal of Engineering (IOSRJEN), Vol. 6,
Issue 10, October, pp. 33 - 51, ISSN (e): 2250-3021, ISSN (p): 2278-8719. Also available on arXiv
at http://arxiv.org/abs/1403.1080.
[12]Greer, K. (2012). Turing: Then, Now and Still Key, in: X-S. Yang (eds.), Artificial Intelligence,
Evolutionary Computation and Metaheuristics (AIECM) - Turing 2012, Studies in Computational
Intelligence, 2013, Vol. 427/2013, pp. 43-62, DOI: 10.1007/978-3-642-29694-9_3, SpringerVerlag Berlin Heidelberg. Published on arXiv at http://arxiv.org/abs/1403.2541.
[13]Greer, K. (2011). Symbolic Neural Networks for Clustering Higher-Level Concepts, NAUN
International Journal of Computers, Issue 3, Vol. 5, pp. 378 – 386, extended version of the
WSEAS/EUROPMENT International Conference on Computers and Computing (ICCC’11).
[14]Greer, K. (2011). Clustering Concept Chains from Ordered Data without Path Descriptions,
Distributed Computing Systems, available on arXiv at http://arxiv.org/abs/1403.0764.
[15]Hämäläinen, W. and Nykänen, M. (2008). Efficient discovery of statistically significant association
rules. Proceedings of the 8th IEEE International Conference on Data Mining (ICDM 2008), pp.
203-212. IEEE Computer Society.
[16]Hawkins, J. and Blakeslee, S. On Intelligence. Times Books, 2004.
[17]Hebb, D.O. (1949). The Organisation of Behaviour.
[18]Hill, S.L., Wang, Y., Riachi, I., Schürmann, F. and Markram, H. (2012). Statistical connectivity
provides a sufficient foundation for specific functional connectivity in neocortical neural
microcircuits, Proceedings of the National Academy of Sciences.
[19]Hopfield, J.J. (1982). Neural networks and physical systems with emergent collective
computational abilities, Proceedings of the National Academy of Sciences of the USA, vol. 79,
No. 8, pp. 2554 – 2558.
[20]Lieto, A., Lebiere, C. and Oltramari, A. (2017). The knowledge level in cognitive architectures:
Current limitations and possible developments, Cognitive Systems Research.
[21]Muller, H.J., Elliot, M.A., Herrmann, C.S and Mecklinger, A. (2001). Neural Binding of Space and
Time: An Introduction, Visual Cognition, Vol. 8, pp. 273 - 285.
[22]Rojas, R. Neural Networks: A Systematic Introduction, Springer-Verlag, Berlin and online at
books.google.com, 1996.
[23]Rosin, D.P., Rontani, D., Gauthier, D.J. and Scholl, E. (2013). Control of synchronization patterns
in neural-like Boolean networks, arXiv preprint repository, http://arxiv.org.

26

[24]Singer, W. and Gray, C.M. (1995). Visual feature integration and the temporal correlation
hypothesis, Annu Rev Neurosci. Vol. 18, pp. 555 - 586.
[25]Tkačik, G., Mora, T., Marre, O., Amodei, D., Palmer, S.E., Berry, M.J. and Bialek, W. (2015).
Thermodynamics and signatures of criticality in a network of neurons, Proceedings of the
National Academy of Sciences, Vol. 112, No. 37, pp. 11508 - 11513.
[26]USDA, NRCS. 2008. The PLANTS Database (https://plants.usda.gov/java/, 31 December 2008).
National

Plant

Data

Center,

Baton

Rouge,

LA

70874-4490

USA.

http://archive.ics.uci.edu/ml/datasets/Plants (last accessed June 2017).
[27]Vogels, T.P., Kanaka Rajan, K. and Abbott, L.F. (2005). Neural Network Dynamics, Annu. Rev.
Neurosci., Vol. 28, pp. 357 - 376.
[28]Watts DJ and Strogatz SH. (1998). Collective dynamics of ‘small world’ networks, Nature, Vol.
393, pp. 440 – 442.
[29]Weisbuch, G. (1999). The Complex Adaptive Systems Approach to Biology, Evolution and
Cognition, Vol. 5, No. 1, pp. 1 - 11.

27

Appendix A
The results of clustering the plants dataset [26] are listed here. The clusters are listed first,
followed by the abbreviation codes for each USA or Canada state or province.

Clusters
1. fl, hi, pr.
2. nc, va.
3. il, in, ia, mo.
4. ky, tn.
5. la, tx.
6. md, de.
7. mi, wi.
8. ak, yt.
9. az, nm.
10. ca, nv, or.
11. co, ut, wy.

12. ga, al.
13. id, mt, wa.
14. ny, pa, ri, vt, ns, on, nj.
15. oh, wv, qc.
16. ab, bc, sk.
17. mb, nb, nt.
18. nf, nu, pe, fraspm.
19. ks, ne.
20. nd, sd, dengl.
21. ok, ar, gl.

22. ct.
23. dc.
24. ms.
25. sc.
26. vi.
27. me.
28. ma.
29. mn.
30. nh.
31. lb.

States
U.S. States:
ab Alabama
ak Alaska
ar Arkansas
az Arizona
ca California
co Colorado
ct Connecticut
de Delaware
dc District of Columbia
fl Florida
ga Georgia
hi Hawaii
id Idaho
il Illinois
in Indiana
ia Iowa
ks Kansas
ky Kentucky
la Louisiana

me Maine
md Maryland
ma Massachusetts
mi Michigan
mn Minnesota
ms Mississippi
mo Missouri
mt Montana
ne Nebraska
nv Nevada
nh New Hampshire
nj New Jersey
nm New Mexico
ny New York
nc North Carolina
nd North Dakota
oh Ohio
ok Oklahoma
or Oregon
pa Pennsylvania

pr Puerto Rico
ri Rhode Island
sc South Carolina
sd South Dakota
tn Tennessee
tx Texas
ut Utah
vt Vermont
va Virginia
vi Virgin Islands
wa Washington
wv West Virginia
wi Wisconsin
wy Wyoming

Canada:
al Alberta
bc British Columbia
mb Manitoba
nb New Brunswick
lb Labrador
nf Newfoundland
nt Northwest Territories
ns Nova Scotia
nu Nunavut
on Ontario
pe Prince Edward Island
qc Québec
sk Saskatchewan
yt Yukon
dengl
Greenland
(Denmark)
fraspm St. Pierre and
Miquelon (France)

28

