abstract: The purpose of this paper is to show a contributor the required style for a paper submitted to the ECAI conference. Authors should realize that once a paper is accepted, the final manuscript submitted by the author will be almost identical to the final, published version that appears in the book, except for pagination and the insertion of running headlines. Author of accepted paper should submit one final zip/rar file containing only one, final, version of the paper including all files necessary to compile the paper as well as a compiled pdf output. Proofreading as regards technical content and English usage is the responsibility of the author. ECAI does not accept submissions in MsWord. If you have any questions regarding the instructions, please contact the IOS Press Book Department through iospress.com/contact. The abstract should contain no more than 200 words.
address: Short Affiliation of First Author; Short Affiliation of Second Author and Third Author
author:  [^1];  ;
bibliography: ecai.bib
title: Guidelines for Preparing a Paper for the
  European Conference on Artificial Intelligence

# Page limit

The page limit for ECAI scientific papers is **7** pages, plus one (**1**) additional page for references only. Scientific papers should report on substantial novel results. The reference list may start earlier than page 8, but only references are allowed on this additional eighth page. The page limit for ECAI highlights is **2** pages. They are intended for disseminating recent technical work (published elsewhere), position, or open problems with clear and concise formulations of current challenges.

Please consult the most recent Call For Papers (CFP) for the most up-to-date detailed instructions.

Page limits are *strict*. Overlength submissions will be rejected without review.

# General specifications

The following details should allow contributors to set up the general page description for their paper:

1.  The paper is set in two columns each 20.5 picas (86 mm) wide with a column separator of 1.5 picas (6 mm).

2.  The typeface is Times Modern Roman.

3.  The body text size is 9 point (3.15 mm) on a body of 11 point (3.85 mm) (i.e., 61 lines of text).

4.  The effective text height for each page is 56 picas (237 mm). The first page has less text height. It requires an additional footer space of 3.5 picas (14.8 mm) for the copyright inserted by the publisher and 1.5 picas (6 mm) of space before the title. The effective text height of the first page is 51 picas (216 mm).

5.  There are no running feet for the final camera-ready version of the paper. The submission paper should have page numbers in the running feet.

# Title, author, affiliation, copyright and running feet

## Title

The title is set in 20 point (7 mm) bold with leading of 22 point (7.7 mm), centered over the full text measure, with 1.5 picas (6 mm) of space before and after.

## Author

The author's name is set in 11 point (3.85 mm) bold with leading of 12 point (4.2 mm), centered over full text measure, with 1.5 picas (6 mm) of space below. A footnote indicator is set in 11 point (3.85 mm) medium and positioned as a superscript character.

## Affiliation

The affiliation is set as a footnote to the first column. This is set in 8 point (2.8 mm) medium with leading of 8.6 point (3.1 mm), with a 1 point (0.35 mm) footnote rule to column width.

## Copyright

The copyright details will be inserted by the publisher.

## Running feet

The running feet are inserted by the publisher. For submission you may insert page numbers in the middle of the running feet. Do not, however, insert page numbers for the camera-ready version of the paper.

# Abstract

The abstract for the paper is set in 9 point (3.15 mm) medium, on a body of 10 point (3.5 mm). The word Abstract is set in bold, followed by a full point and a 0.5 pica space.

# Headings

Three heading levels have been specified:

1.  A level headings

    -   The first level of heading is set is 11 point (3.85 mm) bold, on a body of 12 point (4.2 mm), 1.5 lines of space above and 0.5 lines of space below.

    -   The heading is numbered to one digit with a 1 pica space separating it from the text.

    -   The text is keyed in capitals and is unjustified.

2.  B level headings

    -   The second level of heading is set is 11 point (3.85 mm) bold, on a body of 12 point (4.2 mm), 1.5 lines of space above and 0.5 lines of space below.

    -   The heading is numbered to two digits separated with a full point, with a 1 pica space separating it from the text.

    -   The text is keyed in upper and lower case with an initial capital for first word only, and is unjustified.

3.  C level headings

    -   The third level of heading is set is 10 point (3.5 mm) italic, on a body of 11 point (3.85 mm), 1.5 lines of space above and 0.5 lines of space below.

    -   The heading is numbered to three digits separated with a full point, with a 1 pica space separating it from the text.

    -   The text is keyed in upper and lower case with an initial capital for first word only, and is unjustified.

4.  Acknowledgements

    This heading is the same style as an A level heading but is not numbered.

# Text

The first paragraph of text following any heading is set to the complete measure (i.e., do not indent the first line).

Subsequent paragraphs are set with the first line indented by 1 pica (3.85 mm).

There isn't any inter-paragraph spacing.

# Lists

The list identifier may be an arabic number, a bullet, an em rule or a roman numeral.

The items in a list are set in text size and indented by 1 pica (4.2 mm) from the left margin. Half a line of space is set above and below the list to separate it from surrounding text.

See layout of Section on headings to see the results of the list macros.

# Tables

Tables are set in 8 point (2.8 mm) on a body of 10 point (3.5 mm). The table caption is set centered at the start of the table, with the word Table and the number in bold. The caption is set in medium with a 1 pica (4.2 mm) space separating it from the table number.

A one line space separates the table from surrounding text.

The table caption is centered on the table measure. If it extends to two lines each is centered.

Processors

1
2
4

2-8

Window

◇
◇
□
△
◇
□
△

1273
110
21.79
89%
6717
22.42
61%

2145
116
10.99
50%
5386
10.77
19%

3014
117
41.77
89%
7783
42.31
58%

4753
151
71.55
77%
7477
61.97
49%

5576
148
61.60
80%
7551
91.80
45%

◇ execution time in ticks  □ speed-up values  △ efficiency values

# Figures

A figure caption is set centered in 8 point (2.8 mm) medium on a leading of 10 point (3.5 mm). It is set under the figure, with the word Figure and the number in bold and with a 1 pica (4.2 mm) space separating the caption text from the figure number.

One line of space separates the figure from the caption. A one line space separates the figure from surrounding text.

# Equations

A display equation is numbered, using arabic numbers in parentheses. It is centered and set with one line of space above and below to separate it from surrounding text. The following example is a simple single line equation:

$$Ax=b
\label{thesystem}$$

The next example is a multi-line equation:

$$\begin{aligned}
(x+y)(x-y) & = & x^2-xy+xy-y^2\\
(x+y)^2    & = & x^2+2xy+y^2
\end{aligned}$$

The equal signs are aligned in a multi-line equation.

# Program listings

Program listings are set in 9 point (3.15 mm) Courier on a leading of 11 point (3.85 mm). That is to say, a non-proportional font is used to ensure the correct alignment.

A one line space separates the program listing from surrounding text.

    void inc(x)
    int* x;
    {
        *x++;
    }

    int y = 1;
    inc(&y);
    printf("%d\n",y);

# Theorems

The text of a theorem is set in 9 point (3.15 mm) italic on a leading of 11 point (3.85 mm). The word Theorem and its number are set in 9 point (3.15 mm) bold.

A one line space separates the theorem from surrounding text.

# Footnotes

Footnotes are set in 8 point (2.8 mm) medium with leading of 8.6 point (3.1 mm), with a 1 point (0.35 mm) footnote rule to column width[^2] .

# References

The reference identifier in the text is set as a sequential number in square brackets. The reference entry itself is set in 8 point (2.8 mm) with a leading of 10 point (3.5 mm), and the list of references is sorted alphabetically.

# Sample coding

The remainder of this paper contains examples of the specifications detailed above and can be used for reference if required.

# Programming model

Our algorithms were implemented using the *single program, multiple data* model (SPMD). SPMD involves writing a single code that will run on all the processors co-operating on a task. The data are partitioned among the processors which know what portions of the data they will work on .

## Structure of processes and processors

The grid has $P=P_{\rm{r}}\times P_{\rm{c}}$ processors, where $P_{\rm{r}}$ is the number of rows of processors and $P_{\rm{c}}$ is the number of columns of processors.

### Routing information on the grid

A message may be either *broadcast* or *specific*. A broadcast message originates on a processor and is relayed through the network until it reaches all other processors. A specific message is one that is directed to a particular target processor.

Broadcast messages originate from a processor called *central* which is situated in the 'middle' of the grid. This processor has co-ordinates $(\lfloor P_{\rm{r}}/2 \rfloor,\lfloor P_{\rm{c}}/2
\rfloor)$. Messages are broadcast using the *row–column broadcast* algorithm (RCB), which uses the following strategy. The number of steps required to complete the RCB algorithm (i.e., until all processors have received the broadcast value) is given by $\lfloor
P_{\rm{r}}/2\rfloor+\lfloor P_{\rm{c}}/2\rfloor$.

A specific message is routed through the processors using the *find-row–find-column* algorithm (FRFC) detailed in . The message is sent from the *originator* processor vertically until it reaches a processor sitting in the same row as the *target* processor. The message is then moved horizontally across the processors in that row until it reaches the target processor. An accumulation based on the recursive doubling technique , would require the same number of steps as the RCB requires. If either the row or column of the originator and target processors are the same then the message will travel only in a horizontal or vertical direction, respectively (see ).

# Data partitioning

We use *data partitioning by contiguity*, defined in the following way. To partition the data (i.e., vectors and matrices) among the processors, we divide the set of variables into $P$ subsets $\{\,W_{p}\,\}_{p=1}^{P}$ of $s=N/P$ elements each. We assume without loss of generality that $N$ is an integer multiple of $P$. We define each subset as $W_{p}=\{(p-1)s+j\}_{j=1}^{s}$ (see , and for details).

Each processor $p$ is responsible for performing the computations over the variables contained in $W_{p}$. In the case of vector operations, each processor will hold segments of $s$ variables. The data partitioning for operations involving matrices is discussed in Section .

# Linear algebra operations

## Saxpy

The saxpy $w=u+\alpha v$ operation, where $u$, $v$ and $w$ are vectors and $\alpha$ is a scalar value, has the characteristic that its computation is *disjoint elementwise* with respect to $u, v$ and $w$. This means that we can compute a saxpy without any communication between processors; the resulting vector $w$ does not need to be distributed among the processors. Parallelism is exploited in the saxpy by the fact that $P$ processors will compute the same operation with a substantially smaller amount of data. The saxpy is computed as $$w_{i}=u_{i}+\alpha v_{i},\quad \forall i\in\{W_{p}\}_{p=1}^{P}
\label{distsaxpy}$$

## Inner-product and vector 2-norm

The inner-product $\alpha=u^{T}v=\sum_{i=1}^{N}{u_{i}v_{i}}$ is an operation that involves accumulation of data, implying a high level of communication between all processors. The mesh topology and the processes architecture used allowed a more efficient use of the processors than, for instance, a ring topology, reducing the time that processors are idle waiting for the computed inner-product value to arrive, but the problem still remains. The use of the SPMD paradigm also implies the global broadcast of the final computed value to all processors.

The inner-product is computed in three distinct phases. Phase 1 is the computation of partial sums of the form $$\alpha_{p}=\sum_{\forall i\in\{W_{p}\}}{u_{i}\times v_{i}},\quad p=1,\ldots,P
\label{distutv}$$

The accumulation phase of the inner-product using the RCA algorithm is completed in the same number of steps as the RCB algorithm (Section ). This is because of the need to relay partial values between processors without any accumulation taking place, owing to the connectivity of the grid topology.

The vector 2-norm $\alpha=||\,u\,||_{2}=\sqrt{u^{T}u}$ is computed using the inner-product algorithm described above. Once the inner-product value is received by a processor during the final broadcast phase, it computes the square root of that value giving the required 2-norm value.

## Matrix–vector product

For the matrix–vector product $v=Au$, we use a *column partitioning* of $A$. Each processor holds a set $W_{p}$ (see Section ) of $s$ columns each of $N$ elements of $A$ and $s$ elements of $u$. The $s$ elements of $u$ stored locally have a one-to-one correspondence to the $s$ columns of $A$ (e.g. a processor holding element $u_{j}$ also holds the $j$-th column of $A$). Note that whereas we have $A$ partitioned by columns among the processors, the matrix–vector product is to be computed by *rows*.

The algorithm for computing the matrix–vector product using column partitioning is a generalization of the inner-product algorithm described in Section (without the need for a final broadcast phase). At a given time during the execution of the algorithm, each one of $P-1$ processors is computing a vector $w$ of $s$ elements containing partial sums required for the segment of the vector $v$ in the remaining 'target' processor. After this computation is complete, each of the $P$ processors stores a vector $w$. The resulting segment of the matrix–vector product vector which is to be stored in the target processor is obtained by summing together the $P$ vectors $w$, as described below.

Each processor other than the target processor sends its $w$ vector to one of its neighboring processors. A processor decides whether to send the vector in either the row or column direction to reach the target processor based on the FRFC algorithm (see Section ). If a vector passes through further processors in its route to the target processor the $w$ vectors are accumulated. Thus the target processor will receive at most four $w$ vectors which, when summed to its own $w$ vector, yield the desired set of $s$ elements of $v$.

## Matrix–vector product—finite-difference approximation

We now consider a preconditioned version of the conjugate-gradients method . Note that we do not need to form $A$ explicitly. This implies a very low degree of information exchange between the processors which can be effectively exploited with transputers, since the required values of $u$ can be exchanged independently through each link.

The preconditioning used in our implementations is the polynomial preconditioning (see , , and ), which can be implemented very efficiently in a parallel architecture since it is expressed as a sequence of saxpys and matrix–vector products.

We have $l$ rows and columns in the discretization grid, which we want to partition among a $P_{\rm{r}}\times P_{\rm{c}}$ mesh of processors. Each processor will then carry out the computations associated with a block of $\lfloor
l/P_{\rm{r}}\rfloor+\hbox{sign}\left( l\bmod P_{\rm{r}}\right)$ rows and $\lfloor l/P_{\rm{c}}\rfloor+\hbox{sign}\left( l\bmod
  P_{\rm{c}}\right)$ columns of the interior points of the grid.

The matrix–vector product using the column partitioning is highly parallel. Since there is no broadcast operation involved, as soon as a processor on the boundary of the grid (either rows or columns) has computed and sent a $w_{p}$ vector destined to a processor 'A', it can compute and (possibly) send a $w_{p}$ vector to processor 'B', at which time its neighboring processors may also have started computing and sending their own $w$ vectors to processor 'B'.

At a given point in the matrix–vector product computation, the processors are computing $w$ vectors destined to processor A. When these vectors have been accumulated in the row of that processor (step 1), the processors in the top and bottom rows compute and send the $w$ vectors for processor B, while the processors on the left and right columns of the row of processor A send the accumulated $r$ vectors to processor A (step 2). Processor A now stores its set of the resulting $v$ vector (which is the accumulation of the $w$ vectors). In step 3, the processors in the bottom row compute and send the $w$ vectors for processor C while the processor at the left-hand end of the row of processor B sends the accumulated $w$ vectors of that column towards processor B. The next steps are similar to the above.

In our implementation, we exploit the geometry associated with the regular grid of points used to approximate the PDE. A geometric partitioning is used to match the topology and connectivity present in the grid of transputers (Section ).

The discretization of the PDE is obtained by specifying a grid size $l$ defining an associated grid of $N=l^2$ interior points (note that this is the order of the linear system to be solved). With each interior point, we associate a set of values, namely the coefficients $C, N, S, E\,$ and $W$.

# Conclusion

We have shown that an iterative method such as the preconditioned conjugate-gradients method may be successfully parallelized by using highly efficient parallel implementations of the linear algebra operations involved. We have used the same approach to parallelize other iterative methods with similar degrees of efficiency (see and ).

We would like to thank the referees for their comments, which helped improve this paper considerably

[^1]: Corresponding Author. Email: email@example.com.

[^2]: This is an example of a footnote that occurs in the text. If the text runs to two lines the second line aligns with the start of text in the first line.


author: A.L. Wilmot-Smith, D.I. Pontin and G. Hornig.
Division of Mathematics, University of Dundee, Dundee, DD1 4HN, UK
title:  Dynamics of braided coronal loops – I. Onset of magnetic reconnection

The response of the solar coronal magnetic field to small-scale photospheric boundary motions including the possible formation of current sheets via the Parker scenario is one of open questions of solar physics. Here we address the problem via a numerical simulation. The three-dimensional evolution of a braided magnetic field which is initially close to a force-free state is followed using a resistive MHD code. A long-wavelength instability takes place and leads to the formation of two thin current layers. Magnetic reconnection occurs across the current sheets with three-dimensional features shown, including an elliptic magnetic field structure about the reconnection site, and results in an untwisting of the global field structure.

**Keywords:** Magnetohydrodynamics (MHD) – plasmas – Sun: corona – Sun: magnetic fields

#  Introduction

Parker's notion of the 'topological dissipation' of coronal magnetic fields (Parker, 1972) continues to generate much debate. Simply put, Parker's suggestion is that following boundary motions of sufficient complexity the magnetic field of a coronal loop will be unable to ideally relax to a smooth force-free equilibrium and instead tangential discontinuities in the field, corresponding to current sheets, will develop. In general terms the possible outcomes of relaxation are a development of singular current sheets (e.g. Ng & Bhattacharjee, 1998; Janse & Low, 2009), development of thin but non-singular current layers (e.g. Longcope & Strauss, 1994; Galsgaard *et al.* 2003), and a smooth equilibrium with large-scale current features (e.g. van Ballegooijen 1985; Craig & Sneyd, 2005, Wilmot-Smith *et al.* 2009a). It should also be noted that the distinction between the first two cases may be difficult to determine numerically.

In a previous a paper (Wilmot-Smith *et al.* 2009a) we considered the ideal relaxation of a braided magnetic field towards a force-free equilibrium. The magnetic field configuration was based on the pigtail braid and imposed as an initial condition (rather than being built up through boundary motions). This particular braid was chosen since it is the simplest non-trivial braid with no net twist and, accordingly, is the most conservative realistic case to examine. More complex braided fields will, in general, contain components of the pigtail-type. There is ample motivation for modelling loops as having braided components. Photospheric turbulence subjects loop footpoints to a random walk, with motions of the fragments about each other acting to braid (or unbraid) the overlying loop. However, while there is some evidence for the existence of braided loop configurations (see the left-hand image of Figure  for example), most coronal loops appear to be close to a potential field (e.g. Figure , right-hand image). This may be an effect of the large aspect ratios typical to loops, since a winding of a field line around another field line is almost undetectable when smoothed out over the length of the loop (see also Berger & Asgari-Targhi, 2009). Another reason could be that reconnection is very efficient in maintaining a low degree of topological complexity in loops. The present work is designed in part to test whether this is the case.

In Wilmot-Smith *et al.* (2009a) an ideal Lagrangian relaxation scheme (Craig & Sneyd 1986; Pontin *et al.* 2009) was used to ideally evolve the braided field towards a force-free state. A smooth near force-free equilibrium was attained with large-scale current features i.e. without any tangential discontinuities or even strong current concentrations. (This situation may be compared with Parker (1994) where the same pigtail braid situation is considered as a thought experiment and concluded to inevitably lead to tangential discontinuities. It appears that Parker's assumption of $\alpha = 0$ in the domain fails; we indeed have $\int \alpha \ dS=0$, where $S$ is a cross-sectional surface through the braid, but $\alpha$ varies significantly between field lines).

Although the local current density in the ideal equilibrium is of large-scale, a global quantity, the integrated parallel current, $\int J_{\|} \ dl$, has small scales (here the parallel indicates parallel to the magnetic field and the integral is taken along magnetic field lines). By small scales we mean that $\int J_{\|} \ dl$ varies significantly between neighbouring field lines as the field lines pass though a particular plane, such as the lower boundary. In Wilmot-Smith *et al.* 2009a it was suggested that these small scales in $\int J_{\|} \ dl$ could lead to the development of a resistive instability and so a loss of equilibrium of the field. This mechanism would be distinct to that of Parker's topological dissipation but have the same consequence.

The consideration that for sufficiently small scales in the integrated parallel current then a force-free magnetic field will be resistively unstable (the instability being dependent, for a given scale in the integrated parallel current, on the value of the resistivity) motivated us to put the end, near force-free, state of the Lagrangian relaxation into a resistive MHD code and test whether or not the state is stable (for various values of resistivity allowed by numerical limitations). It turns out that the braided field as implemented in the resistive code is not stable. This finding, together with the subsequent evolution of the field, allows us to address a number of key questions related to MHD and the behaviour of coronal magnetic fields. The results are described in this series of papers. Here we describe the details relating to the numerical setup of the problem and the early evolution of the system. A subsequent paper addresses the long term evolution of the system.

# Numerical Scheme and Simulation Setup

## Numerical Scheme

The numerical scheme employed in the simulations that follow is described briefly below (further details may be found in Nordlund & Galsgaard (1997) and at http://www.astro.ku.dk/$\sim$kg). We solve the three-dimensional resistive MHD equations in the form $$\begin{aligned}
\frac{\partial {\bf B}}{\partial t} & = & - \nabla \times {\bf E},
\label{numeq1}\\
{\bf E} & = & -\left( {\bf v} \times {\bf B} \right)
\: + \: \eta {\bf J}, \label{numeq2}\\
{\bf J} & = & \nabla \times
{\bf B}, \label{numeq3}\\
\frac{\partial \rho}{\partial t} & = & -
\nabla \cdot \left( \rho {\bf v} \right), \label{numeq4}\\
\frac{\partial}{\partial t}\left( \rho {\bf v} \right) & = & - \nabla
\cdot \left( \rho {\bf v} {\bf v} \: + \: {\underline {\underline
\tau}} \right) \: - \: \nabla P \: + \: {\bf J} \times {\bf B},
\label{numeq5}\\
\frac{\partial e}{\partial t} & = & -\nabla \cdot
\left( e {\bf v} \right) \: - \: P \: \nabla \cdot {\bf v} \: + \:
Q_{visc} \: + \: Q_{J} \label{numeq6},
\end{aligned}$$ where ${\bf B}$ is the magnetic field, ${\bf E}$ the electric field, ${\bf v}$ the plasma velocity, $\eta$ the resistivity, ${\bf J}$ the electric current, ${\rho}$ the density, ${\underline {\underline
\tau}}$ the viscous stress tensor, $P$ the pressure, $e$ the internal energy, $Q_{visc}$ the viscous dissipation and $Q_{J}$ the Joule dissipation. An ideal gas is assumed, and hence $P \: = \: \left(
\gamma -1 \right) \: e \: = \: {\textstyle \frac{2}{3}}e$. These equations have been non-dimensionalised by setting the magnetic permeability $\mu_0 = 1$, and the gas constant ($\mathcal{R}$) equal to the mean molecular weight ($M$). The result is that for a volume in which $| \rho |=| {\bf B} | = 1$, time units are such that an Alfvén wave would travel one space unit in one unit of time.

The equations (1-6) are solved on staggered meshes; with respect to a mesh on which $\rho$ and $e$ are defined at the body centre of the cell, $\mathbf{B}$ and $P$ are defined at face centres and $\mathbf{E}$ and $\mathbf{J}$ at edge centres. In this way the required MHD conservation laws are automatically satisfied. Derivatives are calculated using sixth-order finite differences that return a value which is shifted half a grid-point up or down with respect to the input values. When the staggered mesh means that some quantity must be interpolated, data values are calculated using a fifth-order interpolation method at the relevant position. A third-order predictor-corrector method is employed for time-stepping.

In all simulation runs we employ a spatially uniform resistivity model. Viscosity is calculated using a combined second-order and fourth-order method (sometimes termed 'hyper-viscosity'), which is capable of providing sufficient localised dissipation where necessary to handle the development of numerical instabilities (Nordlund & Galsgaard 1997). The effect is to 'switch on' the viscosity where very short length scales develop, while maintaining a minimal amount of viscous dissipation where the velocity field is smooth.

## Creating the Initial Condition

As discussed in Sec. , the initial state for the resistive MHD simulations is drawn from the final state of the Lagrangian relaxation experiment of Wilmot-Smith *et al.* 2009a. The quantities previously known from the Lagrangian code (see Craig & Sneyd 1986) are the magnetic field ${\mathbf{B}}$ and the current ${\mathbf{J}}$ and in the near force-free relaxed state these are known on a highly distorted mesh. We describe below the process of constructing the field on the rectangular grid required for the resistive MHD simulations.

In order to ensure that the initial magnetic field is divergence-free, we work first with the vector potential $\mathbf{A}$ for $\mathbf{B}$. In the relaxation scheme, the calculation of $\mathbf{A}$ requires only that we know the initial vector potential before relaxation and the mesh deformation. In terms of the initial mesh $\mathbf{X}$ and the final 'relaxed mesh' $\mathbf{x}$, the $i$th component of the final vector potential $\mathbf{A}^f$ is given (see Appendix) in terms of the initial vector potential $\mathbf{A}^0$ by $$\label{eq:Atoexplain}
A^f_i = \sum_{j=1}^3 A^0_j \frac{{\partial X_j}}{\partial x_i}.$$

To create the input magnetic field for the MHD simulations we then interpolate this vector potential onto a rectangular grid. Since the magnetic field components are face-centred on the staggered grid, the vector potential components are interpolated onto locations corresponding to edge centres of the desired grid. We then obtain the magnetic field by taking the $curl$ of $\mathbf{A}$ using the sixth-order finite differences described above, which yields magnetic field components at face-centres as required. An interpolation scheme using biharmonic spline radial basis functions was applied to $\mathbf{A}$, the particular scheme chosen to maximize the smoothness of the corresponding current density $\mathbf{J}$, which involves second derivatives of $\mathbf{A}$. To further improve this smoothness a simple five-point smoothing algorithm was finally applied to $\mathbf{A}$, before taking the $curl$.

The result of the above is that the initial braided magnetic field for our MHD simulations is divergence-free to accuracies on the order of truncation errors of the sixth-order finite differences (with typical maximum $|\nabla \cdot {\bf B}| \sim 10^{-6}$ within the domain). The topology of the magnetic field turns out to be well conserved by the process, another important consideration for the experiment. However, a drawback is that the quality of the force-free approximation is not perfectly maintained; the initial state is further from force-free than the relaxed field of the Lagrangian experiment. Details and implications are discussed in Secs.  and .

## Initial State

The initial magnetic field is given on a domain of size $[-24,24]$ in the vertical ($z$) direction and $[-6,6]$ in both the horizontal ($x,y$) directions. Covering this domain we take a uniform mesh of $320^{3}$ cells and employ closed boundary conditions in all three directions. The magnetic field is line-tied, and can be very closely approximated by $\mathbf{B}=[0,0,1]$ at the boundaries, i.e. it is directed perpendicular to the $z$ boundaries and parallel to the $x$ and $y$ boundaries. To achieve the perpendicular condition the Lagrangian relaxation experiment was re-run on the larger horizontal domain (now $[-6,6]^{2}$ compared with $[-4,4]^{2}$ in Wilmot-Smith *et al.* 2009a). The braided field is centered in the middle of the domain and the field is close to uniform in the region external to the braid. Accordingly the results presented here are shown for only the subsection of the full domain in which the important dynamics occur, specifically $[-3,3]^2 \times [-24,24]$.

An isosurface of current in the initial state is given the left-hand image of Fig. . The current has large-scales (see also the upper-left hand image of Fig.  where contours of current in a horizontal cross-section are shown) with two fingers of current extending vertically through the domain. Some sample field lines further illustrating the nature of the field are given in the right-hand image of the same figure. Although non-trivial, the initial state has little magnetic energy in excess of the homogeneous field ($0.96\%$ in $[-3,3]^2 \times [-24,24]$). The aspect ratio employed in the model is 1:8. Although this is larger than that of many previous simulations it is smaller than a realistic ratio for a coronal loop (1:50, say). In the configuration the poloidal field components are small compared with the toroidal components so that the field lines look almost straight. This level of braiding would be observationally difficult to distinguish from a potential field.

To initialise the simulation the dimensionless plasma density (Sec. ) is set at $\rho=1$ throughout the domain and the internal energy as $e=0.1$. The result is a plasma-$\beta$ at $t=0$ that lies in the range $\beta \in [0.1,0.14]$. For the results described in the main section of this paper (Sec. ) we consider the early evolution of the system (up to $t=14$) with time measured in units of the Alfvén time. A uniform resistivity of $\eta = 0.001$ has been taken and the effect of changing the resistivity is discussed at various points in the following text.

# Results

## Basic Properties

Figure  shows the maximum absolute values of the current, velocity and Lorentz force, and the kinetic energy ($\int \frac{1}{2} v^{2} dV$) for the time interval ($t \in [0,14]$) under consideration. The domain taken in all cases is the central section, $[-3,3]^{2} \times [-24,24]$, of the full box. The variation in quantities is shown for a sequence of uniform resistivities decreasing by over an order of magnitude, specifically $\eta = 0.005, 0.001$ and $\eta = 0.0002$.

The initially high maximum Lorentz force, $\vert {\bf j} \times {\bf B} \vert_{max} =0.501$ decreases rapidly over the first few time units. Both the high value and the decrease are artifacts of the method used to create the initial state. The interpolation required to transfer the state to the Eulerian grid (as described in Sec. ) results in some noise in the initial magnetic field and current density. Some noise persists even with the application of a smoothing algorithm to the vector potential for the magnetic field and this is particularly noticeable in the Lorentz force rather than the magnetic field and current density alone. Thus whilst the final state of the Lagrangian relaxation experiment had a very small maximum Lorentz force (specifically $\vert {\bf j} \times {\bf B} \vert_{max} \approx 2 \times 10^{-2}$), the initial state here is further from force-free. The decrease over $t \in [0,2]$ then arises though a smoothing of the noise in the initial state.

Turning now to the remaining quantities shown in Fig. , two primary features are found. Firstly a growth in the kinetic energy and maximum velocity for $t \in [0,4]$ occurs, and this growth is independent of resistivity, $\eta$. For the remaining time considered there is no significant change in kinetic energy.

Secondly, a linear growth in the current density occurs for $t \in [8,12]$. The rate at which the maximum current density increases is higher for lower resistivity, $\eta$. At $t=12$ the maximum current density is achieved; this maximum is higher for lower resistivity but for all three resistivities the growth phase ends at the same time.

The lack of dependence of kinetic energy on resistivity may suggest an ideal instability is present. The subsequent linear growth of current would then be a non-linear consequence of this instability rather than its initial appearance. This growth is clearly dependent on resistivity, being slower for higher values of $\eta$. Little is known about the non-linear phase of instabilities and such a dependence may still be consistent with an ideal instability with a non-linear phase damped by $\eta$. Strong conclusions are clearly difficult to draw at this stage. An additional consideration is that the implementation of the field on the new grid has resulted in significant Lorentz forces in the initial state. We return to these questions in Section  but now proceed to consider the the nature of the currents within the domain, now fixing $\eta = 0.001$.

## Formation of Current Layers

Figure  shows isosurfaces of current at 50% of the domain maximum ($\vert \mathbf{J} \vert =\vert \mathbf{J} \vert_{max}/2$) for a sequence of increasing times (for the initial state see Fig. ). In the early stages ($t \in [0,4]$) the current diffuses slightly while maintaining its large scales in the perpendicular direction. A symmetric evolution follows and after the phase of current growth two current concentrations are present, centered at $z=3.4$ and $z=-3.6$. We call these the two 'initial current layers'.

The formation of these two initial current layers is best illustrated by considering a horizontal cross section through the central plane ($z=0$). Figure  shows contours of the vertical component of current ($\vert {J}_{z} \vert$) in that plane at $t=0, 6, 12$. The $z$-component is taken since it significantly dominates over the two horizontal components, as evident in the shape of the current sheets (Fig. ). Note that in order to incorporate both sheets the cross-sectional plane chosen, $z=0$, does not intersect the centre of either current sheet and so the magnitude of current in this plane is somewhat low in comparison to the domain maximum. The collapse of the two oppositely signed large-scale fingers of current present in the initial state into two thin current sheets of correspondingly the same sign is clearly shown. Also evident is the formation of a weaker current envelope around the braided flux, separating it from the uniform background field. Cross-sections of $\vert {J}_{z} \vert$ in the horizontal planes through the centres of the two current sheets are shown in the final two images of Fig. .

## Predictors of Current Layers

The field lines along which these two current sheets form turns out to be well predicted by the regions of high integrated parallel current in the initial state. In resistive MHD the integrated parallel current is related to the integrated parallel electric field via the relation $\int J_{\|} dl = \eta \int E_{\|} dl$ (in the case of a uniform resistivity $\eta$, as considered here). The integrated parallel electric field is a key quantity for 3D magnetic reconnection; for a localised reconnection region the maximum value of $\int E_{\|} dl$ determines the rate of reconnection.

Shown in the lower right-hand panel of Fig.  are contours of the integrated parallel current, $\int J_{\|} dl$, in the initial state, $t=0$, where the path of the integral is taken over magnetic field lines. Again the contour is shown in the central plane ($z=0$). To obtain this contour map, field lines have been integrated through $160^{2}$ grid of points covering the domain $x,y \in [-3,3]^{2}$, $z=0$. Two peaks in the quantity are present and the structure is quite different to that of the current itself in the initial state. We now identify those field lines in this tracing procedure for which the value of $\vert \int J_{\|} dl \vert$ is greater than or equal to 75% of the domain maximum, noting the locations where they intersect the lower boundary (where the locations of the field lines are held fixed). For the sequence of times of Fig.  we trace field lines starting from those locations on the lower boundary up through the domain and mark with a cross in that same figure their point of intersection with the $z=0$ plane. Here the difference in colours indicates field lines with positive (black crosses) and negative (white crosses) integrated parallel current, although this distinction is made only to facilitate identification of the locations. It is found that these field lines, traced from the initial locations of high integrated parallel current, are good indicators for the locations of formation of the two current layers. Since the flux on the lower boundary is held fixed these may be identified as the same field lines for as long as the evolution remains ideal. Whilst the evolution will be ideal only during the early stages of the simulation it is clear that the tracers do, nevertheless, provide a useful predictor for the locations of current sheet formation.

Locations of high integrated parallel current are not a commonly used indicator for current sheet formation. Indeed it is quasi-separatrix layers (QSLs), regions where the field-line connectivity varies strongly (Priest & Démoulin, 1995) that are widely thought of as likely sites of current sheet formation. To identify QSLs (as well as their intersections, hyperbolic flux tubes or HFTs) the squashing factor (Titov *et al.* 2002) is used. Usually denoted by $Q$, the squashing factor is an indictor of field line connectivity and takes on high values in regions where the field line mapping is strongly distorted. Regions of high $Q$ outline QSLs. As discussed in Wilmot-Smith *et al.* (2009b), the braided magnetic field taken as the initial state here contains several QSLs. Contours of the squashing factor, $Q$, in the central plane ($z=0$) are shown in Fig.  at $t=0$ (left) and $t=12$ (right). For the calculation again $160^{2}$ points over the region $x,y \in [-3,3]^{2}$ have been used, a number comparable to the grid resolution.

Several regions of high $Q$ are present in both snapshots. The two regions of highest $Q$ in the initial state are associated with the two initial current layers at the later time, that is, the current sheets have formed along QSLs of the field. At the same time, several other regions of high $Q$ are present (both at $t=0$ and $t=12$) that are not associated with any particular current features. For example, in the initial state there are eight distinct regions where $\log_{10}\left(Q\right)$ is greater than 85% of its maximum value but only two of these regions correspond to particular current features at $t=12$. These results suggest that the integrated parallel electric current and the squashing factor could be used in conjunction for predicting current sheet formation more accurately than by using $Q$ alone.

## Plasma Flows and Reconnection

We move now to consider the nature of the plasma flows within the domain. First recall that the low values of the plasma beta (Sec. ) imply the dynamics will be dominated by the Lorentz force rather than the gas pressure. With both the magnetic field and the current having stronger vertical than horizontal components we have that the Lorentz force is primarily in the horizontal direction and so, similarly, are the plasma flows. The Lorentz force drives a flow with a dipolar structure in the six regions of initially strongest current with direction dependent on the sign of the twist in that region. An illustration of such a flow is shown for the plane $z=3.4$ (a negative twist region corresponding to one of the sites of current sheet formation) in the first (upper left) image of Fig. .

The next four images in that figure show the development of the flow for a sequence of increasing times up to the point of maximum current. In the sequence the length of the arrows indicating flow direction have been normalized to each image. The background contours show the vertical component of current in that plane with the colour scale normalised to the current at $t=12$ (lower–left-hand image), given by the first colour-bar. The sequence clearly shows the association of the stagnation part of the dipolar flow with the current intensification. The out-flowing plasma from the location of stagnation sets up a counter-flow to the initial dipolar structure leading to oppositely directed flows on either side of the weak enclosing current sheath. The final image shows plasma flows and current in the plane $z=-3.6$ at $t=12$. This cross-section is across the second current sheet and shows the naturally expected inversion of the flow direction. The result is that the global flow structure is dominated by rotational components the direction of which varies both vertically along the structure and on either side ($y>0$, $y<0$) of the braided field.

As already indicated by Fig. , at $t=12$ the maximum magnitude of the plasma flows ($\vert \mathbf{V} \vert_{max} \sim 0.24$) is a significant fraction of the Alfvén speed (${V}_{A} \sim  1.2$). These strongest flows are associated with the global rotation and not with outflows from the two initial current layers (which have an associated $\vert \mathbf{V} \vert_{max} \sim 0.15$). However, we do expect magnetic reconnection to be taking place across these current sheets and so proceed to consider the nature of this reconnection. For this we concentrate again on the initial current layer centered at $z=3.4$ (a similar situation occurs about the other current layer) and focus on the structure of the field and flows in the plane perpendicular to the magnetic field at the location of maximum current magnitude, $\vert J\vert$.

In the left-hand image of Fig.  we indicate in more detail the nature of the stagnation flow about the current sheet.

Superimposed on the background are contours of the out-of plane component of vorticity, i.e. the component of vorticity in the direction of the magnetic field at the location of maximum current. The vorticity shows a quadrupolar configuration around the current sheet.

Perhaps more of a surprise is the structure of the magnetic field in the region. The right-hand image of Fig.  shows the components of the magnetic field in the cross-sectional plane under consideration. The magnetic field is shown to have an elliptic configuration about the current sheet, i.e. about the reconnection region. This finding contrasts with the two-dimensional picture of magnetic reconnection

under which the process can only occur at a hyperbolic (X-type) null-point of the magnetic field. In three-dimensions a much wider variety of possible reconnection sites exist. Reconnection may be associated with 3D null-points (e.g. Lau & Finn, 1990; Priest & Pontin, 2009), magnetic separators (which connect two null-points, e.g. Longcope & Cowley, 1996; Pontin & Craig, 2006; Haynes *et al.*, 2007), or may occur in the absence of any such topological features (Schindler *et al.*, 1988), the latter sometimes termed 'non-null reconnection'. In particular, the local magnetic field structure need not be hyperbolic but may be elliptic (Hornig & Priest 2003), as recently found in some 3D numerical simulations of reconnection. For example, Wilmot-Smith & De Moortel (2007) considered reconnection occurring along a quasi-separator and found an elliptic field structure in perpendicular cross-sections. The separator configuration of Parnell *et al.* (2010) showed an elliptic structure along a significant length of the separator under consideration. Parnell *et al.* (2010) also discussed the separator case theoretically, concluding an elliptic configuration would be a generic situation given a sufficiently strong current density along the separator. Our findings demonstrate that an elliptic field configuration may be present about reconnection sites in the non-null case. Additionally, tracking these field lines back to the initial setup, an elliptic perpendicular field configuration is again found indicating that locally hyperbolic structures are not necessary for current intensification. As previously discussed, the squashing factor ($Q$) at the two reconnection sites is very high which demonstrates a further point; regions of highest-$Q$ within a domain may have a locally elliptic field configuration.

# Nature of Instability

Evidently the initial magnetic field configuration is not in a stable equilibrium. Since an exact equilibrium of the ideal relaxation code employed in Wilmot-Smith *et al.* (2009a) is known to be linearly ideally stable, the lack of stability could arise from one of a number of factors:

1.  A resistive instability. The relaxed state of Wilmot-Smith *et al.* (2009a) contains small scales in the integrated parallel current. Small enough scales (for a given resistivity) in this quantity are incompatible with a resistive equilibrium.

2.   A non-linear ideal instability not previously found in the ideal evolution of Wilmot-Smith *et al.* (2009a) since that evolution only guaranteed linear stability.

3.   A lack of equilibrium in the initial state either since:

    -   The path to equilibrium of the ideal relaxation is fictitious and an exact equilibrium had not been reached. Numerical difficulties (described in Pontin *et al.* 2009) result in the final state of the ideal relaxation having $\vert {\bf J} \times {\bf B} \vert_{max} \approx 2 \times  10^{-2}$, i.e. is not perfectly force-free. Thus the final state of Wilmot-Smith *et al.* 2009a need not be stable (or, indeed accessible via a real MHD relaxation dynamics).

    -   The technique used to transfer the initial state between codes has perturbed the magnetic field further from force-free.

At the beginning of Sec.  we gave one suggestion, that the formation of the current layers may be due to an ideal instability as evidenced by the lack of dependence of kinetic energy on resistivity (see Fig. ). Here we seek to determine additional information that may identify the cause of the dynamical evolution. We have noted that the integrated parallel current is a good predictor of the location of the two initial current layers. The integrated parallel current is a global quantity and so the global structure of the field may play a key role. The global structure is also important in, for example, the kink instability where a magnetic field with a set number of turns per unit length becomes unstable as more turns are added by increasing the length of the system. Whilst the kink-instability as it is usually considered applies to a tube with a well-defined single axis a similar kink-like instability, dependent on the total twist of the system, may apply to our braided field.

With these considerations in mind we examine the evolution of only the middle section of the initial state of the braided magnetic field that is, we cut-out the section of the field in the above described experiment that lies in $z \in [-8,8]$, $x,y \in [-6,6]$ at $t=0$. This field is inserted as an initial condition in a new run, now keeping the flux fixed on $z=\pm 8$, the new upper and lower boundaries of the domain. To maintain consistency we use the same resolution in the horizontal direction $320^{2}$ and a similar effective resolution in the vertical direction, $128$ cells over $z \in [-8,8]$.

In the evolution of this new 'middle-third' field we find the system adjusts from its initial condition (with zero plasma velocity) to an approximate equilibrium in which the current structure is qualitatively similar to that of the initial state. To illustrate, contours of current in the central plane ($z=0$) are shown at $t=20$ in Fig.  (right) which may be compared with Fig.  (upper left) where the corresponding contours in the initial state, $t=0$, are shown. The maximum current in the domain is shown as a solid line in the left-hand image of Fig.  (left-hand axis), the kinetic energy integrated over $x,y \in [-3,3]$, $z \in [-8,8]$ as a dashed line (left-hand axis) and the maximum Lorentz force within the domain as a dot-dashed line (right-hand axis). As in the evolution of the full field, the Lorentz force in this initial state is high as a result of the transfer between grids and decreases rapidly over the first few time units. However, the Lorentz force subsequently stabilises at a low value as the system readjusts to equilibrium. The result that the middle section of the braid alone is in an equilibrium state suggests that an instability is present in the full braided field (rather than a lack of equilibrium due to numerical artefacts). Furthermore the instability is of a long-wavelength type with the full structure of the braided field being key.

Returning to the evidence of Figure , during the time $t \in [8,12]$ where the instability is clear in the current growth there is only a very slight dependence of the maximum velocity within the domain on resistivity and no increase in kinetic energy. This effect may be due to the confining nature of the strong background field external to the braided field structure which results in a deflection of the outflowing plasma around the boundary of the braided field (see Fig. ). Prior to $t \approx 8$ no dependence of the flow on resistivity is seen suggesting an ideal dynamics where the system is adjusting to the distance from equilibrium. The current growth in $t \in [8,12]$ does have a clear separation according to resistivity $\eta$. However the increase is linear (rather than exponential) suggesting a dominant non-linear phase and the growth is slower for higher resistivity suggesting the non-linear phase is damped by resistive effects.

# Conclusions

A previous paper (Wilmot-Smith *et al.* 2009a) considered the ideal relaxation of a braided magnetic field towards a force-free equilibrium. Here we have taken the final state of that relaxation and used it as an initial condition for a full resistive MHD simulation.

The braided field is not in a stable equilibrium; two thin current sheets form after a short time (around a quarter of the time for an Alfvén wave to cross the numerical box in the vertical direction). The linear rate of increase of current density and the maximum strength of the current is found to increase with decreasing resistivity although the evolution of the total kinetic energy in the domain is independent of resolution. We conclude that the instability is possibly an ideal one although the details of how it occurs have not been determined. The wavelength of the instability was tested by considering the MHD evolution of the middle third section of the braid alone and this new field was found to be stable. Hence a long-wavelength dependence is implied.

The initial configuration contains many regions of high squashing factor, $Q$, corresponding to quasi-separatrix layers. Plasma flow across QSLs is often thought likely to lead to current sheet formation. The two current sheets that form here do align with two of the highest regions of $Q$ although the remaining regions of high $Q$ do not correspond to any particular current features. The locations of the two current layers are well predicted by the peak values of the integrated parallel current in the initial state. In the central plane this quantity shows two clear peaks and it is at these regions that the two current layers later form.

These two current layers correspond to reconnection sites. Perpendicular to the layers the magnetic field has an elliptic structure, an admissible property of three dimensional reconnection that has only recently been found. The flow about the reconnection sites is of an asymmetric stagnation type, although large-scale rotational flows dominate the global structure; the effect of reconnection is not a strong acceleration of the flow but a more subtle untwisting process leading to the change in magnetic field topology.

The longer-term evolution of the system will be considered in a future paper.

**Acknowledgements**
The authors would like to thank Nasser Al-Salti for helpful suggestions and Klaus Galsgaard for the use of his code. A.W.S. and G.H. acknowledge financial support from the UK's STFC. The simulations described were carried out on the STFC and SFC (SRIF) funded linux clusters of the UKMHD consortium at the University of St Andrews.

The Transition Region and Coronal Explorer, TRACE, is a mission of the Stanford-Lockheed Institute for Space Research (a joint program of the Lockheed-Martin Advanced Technology Centre's Solar and Astrophysics Laboratory and Stanford's Solar Observatories Group), and part of the NASA Small Explorer program.

**References**

Abraham, R., Marsden, J.E., Ratiu, T. 1988, Manifolds Tensor Analysis and Applications, Applied Mathematical Sciences 75, 370, Springer-Verlag, New York.

Berger, M.A., Asgari-Targhi, M., 2009, ApJ, 705 347-355.

Craig, I. J. D., Sneyd, A. D. 1986, ApJ, 311 451-459.

Craig, I. J. D., Sneyd, A. D. 2005 Sol. Phys., 232 41-62.

Galsgaard, K., Titov, V.S., Neukirch, T. 2003, ApJ, 595 505.

Haynes, A.L., Parnell, C.E., Galsgaard, K., Priest, E.R. 2007, Proc. Roy. Soc. A, 463 1097-1115.

Hornig, G. 1997, Phys. Plasmas 4, 646-656.

Hornig, G., Priest, E.R. 2003, Phys. Plasmas, 10, 2712-2721.

Janse, Å. M., Low, B.C. 2009, ApJ, 690 1089-1104.

Lau, Y-T., Finn, J.M. 1990, ApJ, 350 672-691.

Longcope, D.W, Cowley, S.C. 1996, Phys. Plasmas, 3 (8) 2885-2897.

Longcope, D., Strauss, H.R. 1994, ApJ, 437, 851-859.

Ng, C.S., Bhattacharjee, A. 1998, Phys. Plasmas, 5, 4028-4040

Nordlund, A., Galsgaard, K. 1997, A 3D MHD code for parallel computers. Technical report, Astronomical Observatory, Copenhagen University.

Parker, E.N. Spontaneous Current Sheets in Magnetic Fields With Applications to Stellar X-Rays. 1994, Oxford University Press, Inc., New York (pp. 23-24).

Parnell, C.E., Haynes, A.L., Galsgaard, K. 2010, JGR, 115 A02102.

Pontin, D.I., Craig, I.J.D. 2006, ApJ, 642 568-578.

Pontin, D.I., Hornig, G., Wilmot-Smith, A.L., Craig, I.J.D. 2009, ApJ, 700 (2) 1449-1455.

Priest, E.R., Demoulin, P. 1995, JGR, 100 (A12) 23443-23464.

Priest, E.R., Pontin, D.I. 2009, Phys. Plasmas, 16 122101.

Schindler, K., Hesse, M., Birn, J. 1988, JGR, 93 5547-5557.

Titov, V.S., Hornig, G., Démoulin, P. 2002, JGR, (A8) SSH 3-1 1164.

van Ballegooijen, A.A. 1985, ApJ, 298, 421.

Wilmot-Smith, A.L., De Moortel, I. 2007, A&A, 473 615-623.

Wilmot-Smith, A.L., Hornig, G., Pontin, D. I. 2009a, ApJ, 696 1339-1347.

Wilmot-Smith, A.L., Hornig, G., Pontin, D.I. 2009, ApJ, 704 (2) 1288-1295.


abstract: Representing meaning in the form of high dimensional vectors is a common and powerful tool in biologically inspired architectures. While the meaning of a set of concepts can be summarized by taking a (possibly weighted) sum of their associated vectors, this has generally been treated as a one-way operation. In this paper we show how a technique built to aid sparse vector decomposition allows in many cases the exact recovery of the inputs and weights to such a sum, allowing a single vector to represent an entire set of vectors from a dictionary. We characterize the number of vectors that can be recovered under various conditions, and explore several ways such a tool can be used for vector-based reasoning.
author: Douglas Summers-Stay; Peter Sutor; Dandan Li
title: Representing Sets as Summed Semantic Vectors

# Introduction

Representing concepts as high-dimensional vectors for reasoning is one of the most powerful ideas to come out of biologically-inspired computing. Unlike purely symbolic approaches, high-dimensional vectors have a structure that is rich enough to model the subtle and complex associations that concepts in memory have with one another, allowing generalization, analogy, and the combination of ideas.

There are various approaches to generate such high-dimensional vectors, including modeling semantic primitives with random vectors and using the weights of artificial neural networks. One of the most successful (in terms of immediate applications) sources of vectors has come out of the statistical linguistics community: creating vectors for words by ensuring that words which share a similar context in text corpora are mapped to similar vectors. Throughout this paper we will work with such distributional semantic vectors, but the ideas here can be applied to any architecture that uses high-dimensional vectors to represent meaning. As such we will usually refer to these vectors as representing words or concepts, but without intending to imply that distributional semantic vectors are themselves complete models of the meaning of words or concepts.

An operation common to many such approaches is vector addition or averaging to represent the meaning of a set of vectors. The meaning of this vector is in some way between the meanings associated with the vectors that make it up. For example, the word vector "astronomer" averaged with the word vector "physicist" will typically result in a vector close (in a suitable metric) to the word vector "astrophysicist."

However, this has generally been treated as a kind of summarization of meaning that inevitably loses track of the vectors that make up the sum. It would be convenient to be able to reverse this process: to take a vector which is a weighted sum of meaningful vectors, and find what the weights and inputs were. To a certain extent this is possible simply by looking for the nearest neighbors of the averaged vector in the dictionary. Because of inherent properties of high-dimensional Euclidean spaces and the small size of the dictionary compared to the size of the space, an averaged vector which is the mean of two word vectors will almost always be closer to these two word vectors than to any other word vector. But for more than a few words averaged together, the ability to recover words by looking at near neighbors fails rapidly. In this paper, we show how sparse vector decomposition techniques can be applied to overcome this limitation of summed vectors. This allows them to be directly useful for many kinds of tasks which would have been impractical in a vector-based representation previously. To our knowledge, this is the first use of such sparse decomposition techniques for recovering exact weights for individual word vectors from a vector sum.

What we show in this paper is the following surprising result: a single summed vector can be used to represent the entire set of words from which it was derived, as long as the set is smaller than the limits outlined in section . Rather than thinking of a summed vector as representing a single meaning, we can think of it as picking out an entire set of meanings. For example, the Wikipedia page "list of fruits" lists 90 different fruits. A single 600-dimensional vector[^1] can be used to recall this entire list, provided each fruit is assigned a vector in the dictionary. Beyond that, each fruit on the list can be assigned a weight that it is possible to recover exactly.

# Related Work

Pentti Kanerva's "hyperdimensional computing" is mainly concerned with large binary vectors, but also can include floating-point valued vectors. Kanerva writes,

> "A set of vectors can be combined by componentwise addition, resulting in a vector of the same dimensionality... \[T\]he arithmetic-sum-vector is normalized, yielding a mean vector. It is this mean-vector that is usually meant when we speak of the sum of a set of vectors. The sum (and the mean) of random vectors has the following important property: it is similar to each of the vectors being added together. The similarity is very pronounced when only a few vectors are added and it plays a major role in artificial neural-net models. *The sum-vector is a possible representation for the set that makes up the sum* \[emphasis added\]."

To recover elements of the set, the paper recommends searching the dictionary for the nearest neighbor, subtracting it off, and searching for the nearest neighbor of the remainder. He notes that only small sets can be decomposed in this way.[^2] These ideas are explored further in the subfield called Vector Symbolic Architectures.

In the Hierarchical Temporal Memory model introduced by Jeff Hawkins, concepts are represented by sparse high-dimensional binary vectors, and the "union" of concepts is formed by applying the logical OR operation to these. If the logical AND operation is applied to a vector and this union and the original vector is returned, it belongs to the set with a probability that is easily calculated. This method gives no way of weighting concepts in the sum, however, and is not easily applied to floating-point valued vectors.

The notion of representing the meaning of a set by the simplex whose vertices are the set members in a semantic vector space was explored by Peter Gardenfors. A practical application of this with vectors derived from the weights of a neural network was discussed in .

Dominic Widdows and Trevor Cohen discuss a logic of projection operators in vector spaces. In their system, "Each projection operator projects onto a (linear) subspace; the conjunction of two operators projects onto the intersection of these subspaces; their disjunction projects onto the linear sum of these subspaces; and the negation is the projection onto the orthogonal complement." When these subspaces have the restriction that weights on semantic vectors must sum to one and be positive, they are cut down to simplices and the intersection and union operations he introduces become those discussed in this paper.

Our previous work discusses the possibility of representing a concept by a summed vector of synonyms of the concept and representing the relation between concepts as a vector which can be decomposed into a chain of reasoning as discussed in section . In that work we used a fixed $\lambda$ parameter for LASSO and didn't attempt an exact recovery of the associated weights.

# Decomposition of Vector Sums

Distributional semantic vector spaces represent words as (typically normalized) high-dimensional Euclidean vectors. A set of word vectors is often represented by a sum or average of each word in the set. We will refer to this as a *summed vector* or an *averaged vector* (if it has been normalized) as opposed to the *word vectors* listed in the *dictionary* (a mapping between the strings for words and their associated vectors). The problem of finding a weighted sum of vectors that adds to another vector is a linear regression problem, equivalent to solving a system of linear equations, where the weights are the solutions to each variable and the word vectors are the columns of coefficients on each respective variable. When the number of word vectors in the dictionary is larger than the dimensionality of the vectors, the solution is not unique. In order to choose from among these solutions, we impose the restriction that the weights on most words will be zero (sparsity).

This is known as a sparse sum problem, and can be solved with tools such as the Least Absolute Shrinkage and Selection Operator (LASSO), which selects a few vectors to have non-zero weight in order to approximate the sum.

LASSO balances sparsity against exactness in finding sparse sums with a parameter, $\lambda$. It can be difficult to choose the correct lambda, so we use a screening method called DPP (Dual Polytope Projection) to efficiently test over the full range of $\lambda$ values from 0 to 1, and gather all the candidate non-zero weighted vectors from this range. With a dictionary size less than or equal to the dimensionality of the vectors, it is possible to simply solve the resulting system of linear equations exactly. That is, if it is possible to narrow down the list of possible non-zero weighted vectors to no more than the dimensions of the embedding space, all the weights and associated word vectors can be exactly recovered. Because of this, we choose the $n$ vectors given the highest weight, where $n$ is the dimensionality of the vectors, and solve exactly with these vectors as the dictionary. As long as the correct vectors are included among these $n$ vectors, the exact weights will be recovered.

There is good reason to believe that the brain makes use of some kind of sparse decomposition in order to make sense of complex inputs. It has been speculated that this could be achieved through competition among neural units in a winner-take-all architecture. Others have found that sparsity can be achieved by appropriate thresholding. Note that in the experiments in this paper, the individual word vectors are *not* sparse. The sparsity is in which words are chosen from the dictionary.

| method | number of word vectors recoverable |
|:---|---:|
| nearest-neighbors | 2 |
| LASSO with $\lambda=.02$ | 5 |
| DPP-screening with $\lambda$ ranging from 0 to 1 | 36 |

Screening methods outperform nearest-neighbor and LASSO for recovery of word vectors from an averaged vector. In this experiment, a dictionary of 100,000 300-dimensional vectors was used.

# Experiments: Recovery of Weights and Word Vectors

When trying to recover sparse solutions, there is a phase transition between problems where the solution can be recovered exactly, and problems where this rarely is possible. Problems whose sparsity (non-zero weights / dimensionality) is low enough and whose undersampling ratio (dimensionality / dictionary size) are high enough can be shown to be exactly solvable.

The theoretical probability of a solution being recoverable has to do with the dual of the LASSO problem. It can be calculated by the following formula (see for a derivation): $$\frac{f_k(AC^N)}{f_K(C^N)}$$ where $f_k$ is the number of k-dimensional faces on a polytope, $C^N$ is the cross-polytope[^3] with $N=$ dictionary size, and $A$ is the dictionary matrix which is $n \times N$. For example, for a dictionary size of 10000 and a dimensionality of 100, 300, and 600, the number of terms that should be recoverable in *most* cases is 11, 41 and 97 respectively. These results match the experimental results in figure fairly well. According to the theory, below 3, 11, and 25, respectively, *all* weights can be recovered perfectly for vectors of these dimensions. The word vectors in the n-dimensional models used in this paper are all well modeled by an n-dimensional normal distribution with a mean of zero and are approximately in general position, which means these theoretical bounds should be applicable.[^4] In these experiments we varied four parameters:

1.  Dimensionality of the word vectors

2.  Number of vectors averaged

3.  Semantic similarity of the vectors being averaged

4.  Amount of noise

We used publicly available word embeddings from the Bar-Ilan University NLP lab.[^5] Word embeddings created with a window size of 5, of dimensions 25, 100, 300, and 600 were tested. This data was normalized before the experiments. The experiments were run ten times and the results averaged to emphasize the trend rather than the variability. In general, figures -  show that a smaller dictionary size and a larger dimensionality of the embedding space lead to a larger number of vectors which can be recovered. This can be thought of in terms of density of terms in the embedding space: when the density is low, the number of terms that can be recovered is high.

However, in most real uses, the vectors grouped in a set will have something in common. To simulate this situation, we chose one word at random and chose half of its nearest neighbors at random. These words are all clustered together in the embedding space, but are mixed together equally with words that are not part of the average. In this situation we were able to recover a much larger number of words: for the same dictionary size and vector dimensions which was able to recover around 20 words in the general case (see Figure ), we could recover more than 90 weights and word vectors (see Figure ).

The decomposition process is also robust to the presence of noise, as seen in Figure . This is especially important in architectures that mimic the noisy environment of the brain. Vectors which do not appear in the dictionary would also have a similar effect to noise, and some of the operations that one might wish to apply to a vector, such as looking for analogous concepts, can be treated as finding the true answer plus an unknown noise vector.

# Applications of Summed and Averaged Vectors

The decomposition process[^6] allows us to map vectors in a consistent way from the original vector space (with 300 or so dimensions, whose basis vectors are uninterpretable) to a new sparse overcomplete basis made up of the words in the dictionary. The canonical example from : $$X = 1.0 \times king - 1.0 \times man + 1.0 \times woman$$ can be thought of as a vector in this new basis. When it is mapped back into the original basis, it can be found to be nearby the vector for $queen$. As this is simply a change in basis, vector addition and scalar multiplication on vectors in one basis still hold in the other basis.

There are several different interpretations we can give such a vector.

1.  **A weighted semantic average**. The most direct interpretation is that an averaged vector represents a word semantically between the words which it is the weighted average of. If the word "nashi" is not in the dictionary, we might represent it by a vector halfway between apple and pear (two fruits it resembles and is related to.) Generally speaking, all the properties of something "semantically between" two other things can be expected to share the properties of one or the other, or have a property which is itself a blend between the properties of the two. A nashi, for example, has a yellow-green color and a flavor similar to a pear but a crunchiness and shape closer to an apple. This isn't always true – some properties of an object semantically between $a$ and $b$ may be very different from either $a$ or $b$ – but the exceptions have to be learned as exceptions, while this can be considered the default. This interpretation requires that all weights are positive and sum to one.

2.  **A multi-set**. The summed vector can represents all of the word vectors that make it up, with integer weights above indicating that there is more than one copy of the concept in the multi-set. To add something to the multi-set, we add its semantic vector to the summed vector, and to remove a copy of it from the multi-set we subtract it. This interpretation requires that all weights are positive integers, although generalizations of multi-sets with negative multiplicities could also be used.

3.  **A set**. To represent a set rather than a multi-set, weights above one must be thresholded to one after anything is added to the set. Intersection of sets A and B can be accomplished by decomposing the summed vectors $\vec{a}$ and $\vec{b}$, finding the intersection of the sets of word vectors, and creating a new summed vector from the set of words $\{A \cap B\}$. To form the set union, one can decompose $\vec{a}$ and $\vec{b}$, and form a new vector as the sum of all the word vectors in either $\vec{a}$ or $\vec{b}$.

    However, the decomposition operation is more computationally expensive than addition of vectors, so it is desirable to minimize its use by performing the decomposition after a series of intersection or union operators have been applied. Suppose we have three vectors $\vec{a}$, $\vec{b}$, and $\vec{c}$, representing the sets $A$, $B$, and $C$. We assign each word vector in $\vec{a}$ equal weight $i$, each word vector in $\vec{b}$ weight $j$, and in $\vec{c}$ weight $k$. We form a new vector $\vec{d}=\vec{a}+\vec{b}+\vec{c}$. Then we decompose $\vec{d}$. The weights of the resulting vectors tell what part of the Venn diagram the vector falls into. Those word vectors with weight $i+j+k$ fall into the intersection of all three sets, those with weight $i+j$ fall into the region $A\cap B \cap \neg C$ (and similarly for the other subsets), and those with any positive weight fall into the union $A \cup B \cup C$. These weights will be incorrect if the bounds in the experiment section are exceeded, but the general principle that the most highly weighted word vectors in the sum belong to the intersection of the most sets being combined will still be approximately true.

    The symbol $\top$ can be used to represent the sum of all the vectors in the dictionary, $v_1+...+v_n$. The set $\neg A$ can then be written $\top - (\vec{a_1} + ... + \vec{a_n})$. The union of any positive term $\vec{v}$ with $\top$ is $\vec{v}\cup\top=\top$, while the intersection is $\vec{v}\cap\top=\vec{v}$. The set $\{B\cap\neg A\}$ can then be calculated as $B-B \cap A$.[^7] If needed, $\top$ could be included as a word in the dictionary.

4.  **A fuzzy set**. When we know that our knowledge in an area is complete, performing intersections by taking the literal set intersection given above is correct, and won't introduce any error. But in most real-world applications, we are less than completely certain. Should strawberry preserves be included in the intersection of the set of fruits with the set of things that are sweet? What if we are talking about nutritional content rather than botany? It would be impossible to quantify all of these issues directly except for very narrow topics. For many applications, though, it is helpful to have a graded notion of set intersection and union. The purpose of embedding in a vector space is to take advantage of this "approximate knowledge of many things" that is implicitly available in the structure of the embedding.

    Fuzzy set theory allows inclusion in a set to be a matter of degrees, which is also known as graded classification. If weights between zero and one are allowed, the set interpretation becomes a fuzzy set interpretation. Prototypical examples of a class would be given the highest weight, while more marginal members would be given lower weight. Such weights could be assigned by taking a survey of usage (perhaps $15/20$ respondents felt that $strawberry\_preserves$ should belong to this set, for example, so it would be assigned a weight of $.75$ in the pre-normalized sum) or, if such information is unavailable, including terms near to the known members with weight that decreases as distance from the known core increases according to some monotonically decreasing function.

    In fuzzy set theory, the notions of intersection and union are generalized to handle fractional set inclusion. To calculate the intersection of fuzzy sets $F$ and $G$, we use the function $$F \cap G= min(w_{F1},w_{G1})\vec{v_1} + ... +min(w_{Fn},w_{Gn})\vec{v_n}.$$ The union is $$F \cup G= max(w_{F1},w_{G1})\vec{v_1} + ... +max(w_{Fn},w_{Gn})\vec{v_n}.$$ These functions can also be used for intersection and union of standard sets when the weights are ones and zeros.

5.  **A probability distribution**. Weights here represent a probability distribution over the averaged word vectors. The weights are interpreted as probabilities, and the distribution of weights must be positive and sum to one. Taken this way, the vector $(.6 * \vec{apple} + .4 * \vec{pear})$ would mean a probability of .6 that the object is an apple, and .4 that it is a pear. If we have some function that maps the vector $\vec{apple}$ to $\vec{round}$ and $\vec{pear}$ to $\vec{pear\_shaped}$, and the function is locally linear, then it would map this vector to $(.6 * \vec{round} + .4 * \vec{pear\_shaped})$, showing how probabilities about entities can be carried over into probabilities about their features.

    If the probability distributions are independent, we can calculate a vector representing the probability of $(A\ or\ B)$ by taking the sum of averaged vectors $\vec{a} + \vec{b}$ and re-normalizing (similar to set union above). The probability of $(A\ and\ B)$ can be calculated by decomposing $\vec{a}$ and $\vec{b}$, taking the element-wise product (that is, multiplying the probabilities of each word in $A$ with the probability of the same word in $B$), re-normalizing, and creating a new averaged vector from the new renormalized weights (which amounts to set intersection in the case of equal weights).

6.  **An information structure**. Because the weights are recovered by decomposition as well as the vectors, some information about additional structure on the set can be carried by the weights themselves. As a simple example, one can assign a weight of 1 to the vector for the first word in a sentence, 2 to the second word, and so on. When the weights are recovered, the order of the words is also recovered.[^8] Another possibility is weighting the object of the sentence more heavily than the subject (or vice-versa) to distinguish them. Perhaps even the entire tree structure of a parsed sentence could be encoded in the weights. These ideas are not yet fully developed, and desirable properties of semantic nearness between sentences will not be preserved by such arbitrary weighting schemes.

One challenge is that these various interpretations interfere with one another: if we have some averaged vectors representing semantic averages and others representing probability, it is impossible to tell them apart, and operations on them will tend to mix these interpretations together.

## A Class Simplex

An averaged vector is decomposed into its component word vectors by solving a system of linear equations, whose rows are the equations and whose columns are made up of the word vectors. This is equivalent to finding the barycentric coordinates of the averaged vector within a simplex whose vertices include the word vectors. In the case of the derived fruit vector discussed above, for example, the vertices of this simplex would be the vectors for the original list of fruits. Instead of 90 individual points, we have a 90-dimensional region that represents the set of fruits. Using the first interpretation (semantic betweenness), each point within this space represents some imaginable fruit. Although we may not be able to name all of these possible fruits, we would still likely recognize each as belonging to the class "fruit" and any properties of this fruit we would expect to be shared by some of the other fruits. Depending on the weights, the fruit represented in this way can be more or less similar to any subset of fruits. Gardenfors argues that all natural concepts are convex regions of conceptual space. Although his ideas for how to create the dimensions of the embedding space are different, the principle is the same: if the concepts at all vertices of a face of a simplex share a particular property, then we can assume that any point within that face also shares that property.

We should not, however, expect to find any word vectors inside this region among the words in the dictionary (except those forming the vertices) because it takes up such a tiny fraction of the overall space. If our knowledge is incomplete, there may be word vectors that should belong to the class but haven't been added yet. We can measure the distance from any point to the nearest surface of the simplex, and points nearby can be considered as likely candidates for belonging to the class. The projection of the point onto this nearest point of the simplex, in barycentric coordinates, tells which words from among the set it is most semantically similar to, and with what weights.

Measuring distance to the nearest point on the surface of the simplex is more accurate for classification than distance to the class mean. We created 3734 classes of size between 30 and 300 by taking triples from conceptnet with the same relation and second term, but differing in their first term. For each class, the mean $L_2$ distance from a randomly selected 300-dimensional vector from this list to the mean of the rest of the vectors in the class was .94 with a standard deviation of .05, while the mean distance from the vector to the simplex formed by the rest of the vectors in the class was .88 with a standard deviation of .08. This indicates that distance to the simplex can discriminate somewhat between vectors belonging to a class and vectors the same distance from the mean which do not belong to the class. This is due to the shape of the class distribution, which is flattened in some dimensions. A similar effect was used in where only the most significant dimensions were used to measure distance to the center of a class.

## Deductive Logic

To find a chain of reasoning in propositional logic, one combines statements about propositions such as $A \rightarrow B$ and $B \rightarrow C$ to conclude $A \rightarrow C$. These propositions can be thought of as picking out a set of terms for which the proposition holds, so a summed vector can be used to represent a proposition. The statement $A \rightarrow B$ can be considered as an instruction to replace A with B, which can also be represented as the summed vector $-\vec{a} + \vec{b}$. (This is closely related to the "logical space" system described in .) We can create a knowledge base of such statements stored as vectors, and use that knowledge base as a new dictionary over which we can perform decomposition. To test whether $A \rightarrow D$, we form the vector $-\vec{a}+\vec{d}$ and see whether it can be decomposed into vectors from this knowledge base, such as

$$(-\vec{a}+\vec{b})+(-\vec{b}+\vec{c})+(-\vec{c}+\vec{d}) = (-\vec{a}+\vec{d})$$

If the sets A,B,C, and D above are all represented by the summed vector of the elements belonging to the set, the quantities $(-\vec{a}+\vec{b})$, $(-\vec{b}+\vec{c})$, and so forth will all be a positive sum of word vectors, since implication must alway take a set to a superset (if Socrates is in the set of all men, and we are given the fact that $\{men\} \rightarrow \{mortals\}$, Socrates must belong to the set of mortals). This is a little disingenuous, since if we already knew precisely which elements belong to each set there would be no need to perform deductive reasoning at all. However, even if the intermediate sets are represented by some other vectors, the chain can still be found because these all intermediate steps cancel out in the end.

When the knowledge base is embedded in a semantic vector space it can go beyond what can be deduced from the premises in purely deductive reasoning into what Charles Pierce called "ampliative reasoning." This includes analogical, inductive, and abductive reasoning. Traditional knowledge bases tend to be brittle because a query posed in a slightly different way than the knowledge was entered will have no results. The vector space embedding allows some generalization of facts that helps overcome this brittleness. If an exact decomposition cannot be found, an approximate decomposition will be returned.

## Analogical Reasoning

Semantic vectors are able to perform analogical reasoning with some simple arithmetic. This is also possible with vectors representing sets. For example, suppose we wanted to capture the meaning of the concept $Detroit$ in the sentence "Detroit won the game on Monday night." The average of the word vectors $\{\vec{Detroit}$, $\vec{Detroit\_Lions}$, $\vec{Detroit\_Pistons}$, $\vec{Detroit\_Tigers}$, $\vec{Michigan}\}$, when $\vec{Cleveland}$ is added to it and $\vec{Detroit}$ subtracted from it, has the following as its highest weighted components: $\{\vec{Cleveland}$, $\vec{Cleveland\_Cavaliers}$, $\vec{Ohio}$, $\vec{Cavaliers}$, $\vec{Browns}$, $\vec{Cleveland\_Indians}\}$. This is a simple example, but illustrates how the notion of analogical reasoning can begin to be extended from individual words (where distributional semantic vectors have shown human levels of performance on multiple choice tests) to analogies between concepts. If we wish to know a more exact correspondence to each part of the analogy, we can use a word-level analogy over the much smaller dictionary created by the set of results of the set-level analogy.

# Learning from Definitions

New terms can be added to the dictionary by defining them by means of the terms already in the dictionary along with the operations of union, intersection, negation, and analogy. For example, here is a definition of $nashi (tree)$:

-   an Asian tree also cultivated in Australia and New Zealand, Pyrus pyrifolia, of the rose family, having apple-shaped, pear-colored, juicy fruit.

We can define a vector $\{trees\}$ representing the set of trees by adding together all known types of tree. If we have knowledge that some of these are $\{asian\_trees\}$, we can define a subset including just those. Otherwise, we need to define a set of $\{asian\_things\}$ and find the fuzzy intersection of that with $\{trees\}$. We can further intersect this with the set $\{rose\_family\}$.

We now have a $\vec{nashi\_tree}$ vector that is a weighted sum of asian things, trees, and plants in the rose family. We can similarly create a $nashi\_fruit$ vector which is a weighted sum of apple-shape things, pear-colored things, juicy things, and fruit. But to represent that the fruit belongs to the plant is probably not best captured by intersection or union. Instead, we can define a relation vector $has\_fruit$ between a tree and its fruit by averaging the differences $(pear\;-\;pear\_tree)$, $(apple\;-\;apple\_tree)$ and so forth. Since we know that the $nashi\_tree$ and $nashi\_fruit$ vector have this same relationship, we can take $((nashi\_tree+has\_fruit) +(nashi\_fruit-has\_fruit))/2$ to get a revised $\vec{nashi\_tree}$ vector. $\vec{nashi\_tree}$ can now be added to the dictionary, and the phrase will be appropriately placed in the vector space to allow it to be used for analogical reasoning and similar operations.

# Conclusion

Decomposing summed vectors is a reliable tool that allows high-dimensional vectors to be used to represent sets, probability distributions, ordered lists, and many other possibilities with exact solutions up to a certain size limit. The relationship between two concepts can also be decomposed into a set of known relations, forming a chain of deductive reasoning connecting the concepts. As opposed to working with symbolic knowledge bases, these vector-based methods degrade gracefully in the case of noise or missing information.

[^1]: A vector that represents a category as an average is often very close to the vector representing the word for the category. In the case of the fruit category vector discussed here, the word 'fruit' is the closest word in the dictionary to the averaged vector. suggests that this is due to the fact that the hypernym 'fruit' is associated with each of its hyponyms, but only some of the hyponyms are associated with each other.

[^2]: Kanerva typically works with 10000-dimensional vectors, which would allow this method to work for somewhat larger sets than the 25-600 dimensional vectors discussed in this paper.

[^3]: The cross-polytope is a generalization of the square and octahedron to higher dimensions.

[^4]: A calculator for the 100% solvable and 50% solvable cutoff in terms of sparsity and undersampling ratio can be found at <a href="people.maths.ox.ac.uk/tanner/polytopes.shtml" class="uri">people.maths.ox.ac.uk/tanner/polytopes.shtml</a>.

[^5]: We also verified that results reported for 300-dimensional vectors were similar for Mikolov's 300-dimensional word2vec embeddings trained on the Google News corpus.

[^6]: In order to perform a decomposition using DPP, a summed vector must be first be normalized. It can be de-normalized after the decompostion.

[^7]: Widdows suggests that $\{B \cap \neg A\}$ be calculated by projecting $\vec{b}$ onto the space perpendicular to $\vec{a}$. This is equivalent to adding $-w*\vec{a}$ to $\vec{b}$, for some particular positive $w$, but this $w$ will be higher than is strictly necessary to reduce the components in $a\cap \neg b$ to zero, if the vector $b \cap \neg{a}$ is not already orthogonal to $a \cap \neg b$. This may be desirable, depending on how the result is to be interpreted. In practice, though, the nearest neighbors of a vector from the dictionary are surprisingly robust to changes in the weight of that vector, so finding the correct value for w may not be critical for many applications.

[^8]: A slightly more sophisticated weight coding scheme would be needed to handle sentences which may contain repeated words, such as using a random floating-point number to represent each position and using a lookup table for potential sums of these.