Question: As the size and complexity of models and datasets grow, so does the need for communication-efficient variants of stochastic gradient descent that can be deployed on clusters to perform model fitting in parallel. describe two variants of data-parallel SGD that quantize and encode gradients to lessen communication costs. For the first variant, QSGD, they provide strong theoretical guarantees. For the second variant, which we call QSGDinf, they demonstrate impressive empirical gains for distributed training of large neural networks. Building on their work, we propose an alternative scheme for quantizing gradients and show that it yields stronger theoretical guarantees than exist for QSGD while matching the empirical performance of QSGDinf. Deep learning is booming thanks to enormous datasets and very large models, leading to the fact that the largest datasets and models can no longer be trained on a single machine. One common solution to this problem is to use distributed systems for training. The most common algorithms underlying deep learning are stochastic gradient descent (SGD) and its variants, which led to a significant amount of research on building and understanding distributed versions of SGD. Implementations of SGD on distributed systems and data-parallel versions of SGD are scalable and take advantage of multi-GPU systems. Data-parallel SGD, in particular, has received significant attention due to its excellent scalability properties (; ; ; ; ; ; ; ; . In data-parallel SGD, a large dataset is partitioned among K processors. These processors work together to minimize an objective function. Each processor has access to the current parameter vector of the model. At each SGD iteration, each processor computes an updated stochastic gradient using its own local data. It then shares the gradient update with its peers. The processors collect and aggregate stochastic gradients to compute the updated parameter vector. Increasing the number of processing machines reduces the computational costs significantly. However, the communication costs to share and synchronize huge gradient vectors and parameters increases dramatically as the size of the distributed systems grows. Communication costs may thwart the anticipated benefits of reducing computational costs. Indeed, in practical scenarios, the communication time required to share stochastic gradients and parameters is the main performance bottleneck (; ; ; . Reducing communication costs in data-parallel SGD is an important problem. One promising solution to the problem of reducing communication costs of data-parallel SGD is gradient compression, e.g., through gradient quantization (; ; ; ; ; ;). (This should not be confused with weight quantization/sparsification, as studied by ; ; ;, which we do not discuss here.) Unlike full-precision data-parallel SGD, where each processor is required to broadcast its local gradient in full-precision, i.e., transmit and receive huge full-precision vectors at each iteration, quantization requires each processor to transmit only a few communication bits per iteration for each component of the stochastic gradient. One popular such proposal for communication-compression is quantized SGD (QSGD), due to. In QSGD, stochastic gradient vectors are normalized to have unit L 2 norm, and then compressed by quantizing each element to a uniform grid of quantization levels using a randomized method. While most lossy compression schemes do not provide convergence guarantees, QSGD's quantization scheme, is designed to be unbiased, which implies that the quantized stochastic gradient is itself a stochastic gradient, only with higher variance determined by the dimension and number of quantization levels. As a , are able to establish a number of theoretical guarantees for QSGD, including that it converges under standard assumptions. By changing the number of quantization levels, QSGD allows the user to trade-off communication bandwidth and convergence time. Despite their theoretical guarantees based on quantizing after L 2 normalization, Alistarh et al. opt to present empirical using L ∞ normalization. We call this variation QSGDinf. While the empirical performance of QSGDinf is strong, their theoretical guarantees on the number of bits transmitted no longer apply. Indeed, in our own empirical evaluation of QSGD, we find the variance induced by quantization is substantial, and the performance is far from that of SGD and QSGDinf. Given the popularity of this scheme, it is natural to ask one can obtain guarantees as strong as those of QSGD while matching the practical performance of the QSGDinf heuristic. In this work, we answer this question in the affirmative by providing a new quantization scheme which fits into QSGD in a way that allows us to establish stronger theoretical guarantees on the variance, bandwidth, and cost to achieve a prescribed gap. Instead of QSGD's uniform quantization scheme, we use an unbiased nonuniform logarithmic scheme, similar to those introduced in telephony systems for audio compression . We call the ing algorithm nonuniformly quantized stochastic gradient descent (NUQSGD). Like QSGD, NUQSGD is a quantized data-parallel SGD algorithm with strong theoretical guarantees that allows the user to trade off communication costs with convergence speed. Unlike QSGD, NUQSGD has strong empirical performance on deep models and large datasets, matching that of QSGDinf. In particular, we provide a new efficient implementation for these schemes using a modern computational framework (Pytorch), and benchmark it on classic large-scale image classification tasks. The intuition behind the nonuniform quantization scheme underlying NUQSGD is that, after L 2 normalization, many elements of the normalized stochastic gradient will be near-zero. By concentrating quantization levels near zero, we are able to establish stronger bounds on the excess variance. In the overparametrized regime of interest, these bounds decrease rapidly as the number of quantization levels increases. Combined with a bound on the expected code-length, we obtain a bound on the total communication costs of achieving an expected suboptimality gap. The ing bound is slightly stronger than the one provided by QSGD. To study how quantization affects convergence on state-of-the-art deep models, we compare NUQSGD, QSGD, and QSGDinf, focusing on training loss, variance, and test accuracy on standard deep models and large datasets. Using the same number of bits per iteration, experimental show that NUQSGD has smaller variance than QSGD, as expected by our theoretical . This smaller variance also translates to improved optimization performance, in terms of both training loss and test accuracy. We also observe that NUQSGD matches the performance of QSGDinf in terms of variance and loss/accuracy. Further, our distributed implementation shows that the ing algorithm considerably reduces communication cost of distributed training, without adversely impacting accuracy. Our empirical show that NUQSGD can provide faster end-to-end parallel training relative to data-parallel SGD, QSGD, and Error-Feedback SignSGD on the ImageNet dataset. • We establish stronger theoretical guarantees for the excess variance and communication costs of our gradient quantization method than those available for QSGD's uniform quantization method. • We then establish stronger convergence guarantees for the ing algorithm, NUQSGD, under standard assumptions. • We demonstrate that NUQSGD has strong empirical performance on deep models and large datasets, both in terms of accuracy and scalability. Thus, NUQSGD closes the gap between the theoretical guarantees of QSGD and the empirical performance of QSGDinf. proposed signSGD, an efficient heuristic scheme to reduce communication costs drastically by quantizing each gradient component to two values. later provided convergence guarantees for signSGD. Note that the quantization employed by signSGD is not unbiased, and so a new analysis was required. As the number of levels is fixed, SignSGD does not provide any trade-off between communication costs and convergence speed. introduced Buckwild!, a lossy compressed SGD with convergence guarantees. The authors provided bounds on the error probability of SGD, assuming convexity and gradient sparsity. proposed TernGrad, a stochastic quantization scheme with three levels. TernGrad also significantly reduces communication costs and obtains reasonable accuracy with a small degradation to performance compared to full-precision SGD. Convergence guarantees for TernGrad rely on a nonstandard gradient norm assumption. As discussed, proposed QSGD, a more general stochastic quantization scheme, for which they provide both theoretical guarantees and experimental validation (although for different variants of the same algorithm). We note that their implementation was only provided in Microsoft CNTK; by contrast, here we provide a more generic implementation in Horovod , a communication back-end which can support a range of modern frameworks such as Tensorflow, Keras, Pytorch, and MXNet. NUQSGD uses a logarithmic quantization scheme. Such schemes have long been used in telephony systems for audio compression . Logarithmic quantization schemes have appeared in other contexts recently: studied weight distributions of long short-term memory networks and proposed to use logarithm quantization for network compression. proposed a gradient compression scheme and introduced an optimal quantization scheme, but for the setting where the points to be quantized are known in advance. As a , their scheme is not applicable to the communication setting of quantized data-parallel SGD. We consider a high-dimensional machine learning model, parametrized by a vector w ∈ R d. Let Ω ⊆ R d denote a closed and convex set. Our objective is to minimize f: Ω → R, which is an unknown, differentiable, convex, and β -smooth function. The following summary is based on. where · denotes the Euclidean norm. Let (S, Σ, µ) be a probability space (and let E denote expectation). Assume we have access to stochastic gradients of f, i.e., we have access to a function In the rest of the paper, we let g(w) denote the stochastic gradient for notational simplicity. The update rule for conventional full-precision projected SGD is w t+1 = P Ω (w t − αg(w t)), where w t is the current parameter input, α is the learning rate, and P Ω is the Euclidean projection onto Ω. We say the stochastic gradient has a second-moment upper bound B when E[g(w) 2 ] ≤ B for all w ∈ Ω. Similarly, the stochastic gradient has a variance upper bound Note that a second-moment upper bound implies a variance upper bound, because the stochastic gradient is unbiased. We have classical convergence guarantees for conventional full-precision SGD given access to stochastic gradients at each iteration: Theorem 1 (Bubeck 2015, Theorem 6.3). Let f: Ω → R denote a convex and β -smooth function and let R 2 sup w∈Ω w − w 0 2. Suppose that the projected SGD update is executed for T iterations with α = 1/(β + 1/γ) where γ = r 2/T /σ. Given repeated and independent access to stochastic gradients with a variance upper bound σ 2, projected SGD satisfies Minibatched (with larger batch sizes) and data-parallel SGD are two common SGD variants used in practice to reduce variance and improve computational efficiency of conventional SGD. At each iteration, each processor computes its own stochastic gradient based on its local data and then broadcasts it to all peers. Each processor receives and aggregates the stochastic gradients from all peers to obtain the updated parameter vector. In detail, the update rule for full-precision dataparallel SGD is where g l (w t) is the stochastic gradient computed and broadcasted by processor l. Provided that g l (w t) is a stochastic gradient with a variance upper bound σ 2 for all l, then Data-parallel SGD is described in Algorithm 1. Full-precision data-parallel SGD is a special case of Algorithm 1 with identity encoding and decoding mappings. Otherwise, the decoded stochastic gradientĝ i (w t) is likely to be different from the original local stochastic gradient g i (w t). By Theorem 1, we have the following convergence guarantees for full-precision data-parallel SGD: Corollary 1 (Alistarh et al. 2017, Corollary 2.2). Let f, R, and γ be as defined in Theorem 1 and let ε > 0. Suppose that the projected SGD update is executed for T iterations with α = 1/(β + √ K/γ) on K processors, each with access to independent stochastic gradients of f with a second-moment bound B. The smallest T for the full-precision data-parallel SGD that guarantees Data-parallel SGD reduces computational costs significantly. However, the communication costs of broadcasting stochastic gradients is the main performance bottleneck in large-scale distributed systems. In order to reduce communication costs and accelerate training, introduced a compression scheme that produces a compressed and unbiased stochastic gradient, suitable for use in SGD. At each iteration of QSGD, each processor broadcasts an encoding of its own compressed stochastic gradient, decodes the stochastic gradients received from other processors, and sums all the quantized vectors to produce a stochastic gradient. In order to compress the gradients, every coordinate (with respect to the standard basis) of the stochastic gradient is normalized by the Euclidean norm of the gradient and then stochastically quantized to one of a small number quantization levels distributed uniformly in the unit interval. The stochasticity of the quantization is necessary to not introduce bias. give a simple argument that provides a lower bound on the number of coordinates that are quantized to zero in expectation. Encoding these zeros efficiently provides communication savings at each iteration. However, the cost of their scheme is greatly increased variance in the gradient, and thus slower overall convergence. In order to optimize overall performance, we must balance communication savings with variance. By simple counting arguments, the distribution of the (normalized) coordinates cannot be uniform. Indeed, this is the basis of the lower bound on the number of zeros. These arguments make no assumptions on the data distribution, and rely entirely on the fact that the quantities being quantized are the coordinates of a unit-norm vector. Uniform quantization does not capture the properties of such vectors, leading to substantial gradient variance. In this paper, we propose and study a new scheme to quantize normalized gradient vectors. Instead of uniformly distributed quantization levels, as proposed by, we consider quantization levels that are nonuniformly distributed in the unit interval, as depicted in Figure 1. In order to obtain a quantized gradient that is suitable for SGD, we need the quantized gradient to remain unbiased. achieve this via a randomized quantization scheme, which can be easily generalized to the case of nonuniform quantization levels. Using a carefully parametrized generalization of the unbiased quantization scheme introduced by Alistarh et al., we can control both the cost of communication and the variance of the gradient. Compared to a uniform quantization scheme, our scheme reduces quantization error and variance by better matching the properties of normalized vectors. In particular, by increasing the number of quantization levels near zero, we obtain a stronger variance bound. Empirically, our scheme also better matches the distribution of normalized coordinates observed on real datasets and networks. We now describe the nonuniform quantization scheme: Let s ∈ {1, 2, · · ·} be the number of internal quantization levels, and let L = (l 0, l 1, · · ·, l s+1) denote the sequence of quantization levels, where, lets(r) and p(r) satisfy ls (r) ≤ r ≤ ls (r)+1 and r = 1 − p(r) ls (r) + p(r)ls (r)+1, respectively. Define τ(r) = ls (r)+1 − ls (r). Note thats(r) ∈ {0, 1, · · ·, s}. where, letting r i = |v i |/ v, the h i (v, s)'s are independent random variables such that h i (v, s) = ls (r i) with probability 1 − p(r i) and h i (v, s) = ls (r i)+1 otherwise. We note that the distribution of h i (v, s) satisfies E[h i (v, s)] = r i and achieves the minimum variance over all distributions that satisfy E[h i (v, s)] = r i with support L. In the following, we focus on a special case of nonuniform quantization withL = (0, 1/2 s, · · ·, 2 s−1 /2 s, 1) as the quantization levels. The intuition behind this quantization scheme is that it is very unlikely to observe large values of r i in the stochastic gradient vectors of machine learning models. Stochastic gradients are observed to be dense vectors . Hence, it is natural to use fine intervals for small r i values to reduce quantization error and control the variance. After quantizing the stochastic gradient with a small number of discrete levels, each processor must encode its local gradient into a binary string for broadcasting. We describe this encoding in Appendix A. In this section, we provide theoretical guarantees for NUQSGD, giving variance and code-length bounds, and using these in turn to compare NUQSGD and QSGD. Please note that the proofs of Theorems 2, 3, 4, and 5 are provided in Appendices B, C, D, and E respectively. where The in Theorem 2 implies that if g(w) is a stochastic gradient with a second-moment bound η, then Q s (g(w)) is a stochastic gradient with a variance upper bound ε Q η. In the range of interest where d is sufficiently large, i.e., s = o(log(d)), the variance upper bound decreases with the number of quantization levels. To obtain this data-independent bound, we establish upper bounds on the number of coordinates of v falling into intervals defined byL. where Theorem 3 provides a bound on the expected number of communication bits to encode the quantized stochastic gradient. Note that 2 2s + √ d2 s ≤ d/e is a mild assumption in practice. As one would expect, the bound,, increases monotonically in d and s. In the sparse case, if we choose s = o(log d) levels, then the upper bound on the expected code-length is Combining the upper bounds above on the variance and code-length, Corollary 1 implies the following guarantees for NUQSGD: Theorem 4 (NUQSGD for smooth convex optimization). Let f and R be defined as in Theorem 1, let ε Q be defined as in Theorem 2, let ε > 0,B = (1 + ε Q)B, and let γ > 0 be given by γ 2 = 2R 2 /(BT). With ENCODE and DECODE defined as in Appendix A, suppose that Algorithm 1 is executed for T iterations with a learning rate α = 1/(β + √ K/γ) on K processors, each with access to independent stochastic gradients of f with a second-moment bound B. Then In addition, NUQSGD requires at most N Q communication bits per iteration in expectation. On nonconvex problems, (weaker) convergence guarantees can be established along the lines of, e.g., (, Theorem 2.1). How do QSGD and NUQSGD compare in terms of bounds on the expected number of communication bits required to achieve a given suboptimality gap ε? The quantity that controls our guarantee on the convergence speed in both algorithms is the variance upper bound, which in turn is controlled by the quantization schemes. Note that the number of quantization levels, s, is usually a small number in practice. On the other hand, the dimension, d, can be very large, especially in overparameterized networks. In Figure 2, we show that the quantization scheme underlying NUQSGD in substantially smaller variance upper bounds for plausible ranges of s and d. Note that these bounds do not make any assumptions on the dataset or the structure of the network. For any (nonrandom) number of iterations T, an upper bound, N A, holding uniformly over iterations k ≤ T on the expected number of bits used by an algorithm A to communicate the gradient on iteration k, yields an upper bound T N A, on the expected number of bits communicated over T iterations by algorithm A. Taking T = T A,ε to be the (minimum) number of iterations needed to guarantee an expected suboptimality gap of ε based on the properties of A, we obtain an upper bound, ζ A,ε = T A,ε N A, on the expected number of bits of communicated on a run expected to achieve a suboptimality gap of at most ε. Theorem 5 (Expected number of communication bits). Provided that s = o(log(d)) and Focusing on the dominant terms in the expressions of overall number of communication bits required to guarantee a suboptimality gap of ε, we observe that NUQSGD provides slightly stronger guarantees. Note that our stronger guarantees come without any assumption about the data. In this section, we examine the practical performance of NUQSGD in terms of both convergence (accuracy) and speedup. The goal is to empirically show that NUQSGD can provide the same performance and accuracy compared to the QSGDInf heuristic, which has no theoretical compression guarantees. For this, we implement and test these three methods (NUQSGD, QSGD, and QSGDInf), together with the distributed full-precision SGD baseline, which we call SuperSGD. We split our study across two axes: first, we examine the convergence of the methods and their induced variance. Second, we provide an efficient implementation of all four methods in Pytorch using the Horovod communication back-end , adapted to efficiently support quantization, and examine speedup relative to the full-precision baseline. We investigate the impact of quantization on training performance by measuring loss, variance, accuracy, and speedup for ResNet models applied to ImageNet and CIFAR10 (Krizhevsky). We evaluate these methods on two image classification datasets: ImageNet and CIFAR10. We train ResNet110 on CIFAR10 and ResNet18 on ImageNet with mini-batch size 128 and base learning rate 0.1. In all experiments, momentum and weight decay are set to 0.9 and 10 −4, respectively. The bucket size and the number of quantization bits are set to 8192 and 4, respectively. We observe similar in experiments with various bucket sizes and number of bits. We simulate a scenario with k GPUs for all three quantization methods by estimating the gradient from k independent mini-batches and aggregating them after quantization and dequantization. In Figure 3 (left and middle), we show the training loss with 8 GPUs. We observe that NUQSGD and QSGDinf improve training loss compared to QSGD on ImageNet. We observe significant gap in training loss on CIFAR10 where the gap grows as training proceeds. We also observe similar performance gaps in test accuracy (provided in Appendix F). In particular, unlike NUQSGD, QSGD does not achieve test accuracy of full-precision SGD. Figure 3 (right) shows the mean normalized variance of the gradient (defined in Appendix F) versus training iteration on the trajectory of single-GPU SGD on CIFAR10. These observations validate our theoretical that NUQSGD has smaller variance for large models with small number of quantization bits. Efficient Implementation and Speedup. To examine speedup behavior, we implemented all quantization methods in Horovod , a communication back-end supporting Pytorch, Tensorflow and MXNet. Doing so efficiently requires non-trivial refactoring of this framework, since it does not support communication compression-our framework will be open-sourced upon publication. Our implementation diverges slightly from the theoretical analysis. First, Horovod applies "tensor fusion" to multiple layers, by merging the ing gradient tensors for more efficient transmission. This causes the gradients for different layers to be quantized together, which can lead to loss of accuracy (due to e.g. different normalization factors across the layers). We addressed this by tuning the way in which tensor fusion is applied to the layers such that it minimizes the accuracy loss. Second, we noticed that quantizing the gradients corresponding to the biases has a significant adverse effect on accuracy; since the communication impact of biases is negligible, we transmit them at full precision. We apply this for all methods considered. Finally, for efficiency reasons, we directly pack the quantized values into 32-bit numbers, without additional encoding. We implemented compression and de-compression via efficient CUDA kernels. Our baselines are full-precision SGD (SuperSGD), Error-Feedback SignSGD , and the QSGDinf heuristic, which we compare against the 4-bit and 8-bit NUQSGD variants executing the same pattern. The implementation of the QSGDinf heuristic provides almost identical convergence numbers, and is sometimes omitted for visibility. (QSGD yields inferior convergence on this dataset and is therefore omitted.) All variants are implemented using a standard all-to-all reduction pattern. Figures 4 (left), (middle) show the execution time per epoch for ResNet34 and ResNet50 models on ImageNet, on a cluster machine with 8 NVIDIA 2080 Ti GPUs, for the hyperparameter values quoted above. The confirm the efficiency and scalability of the compressed variant, mainly due to the reduced communication volume. We note that the overhead of compression and decompression is less than 1% of the batch computation time for NUQSGD. Figure 4 (right) presents end-to-end speedup numbers (time versus accuracy) for ResNet50/ImageNet, executed on 4 GPUs, under the same hyperparameter settings as the full-precision baseline, with bucket size 512. First, notice that NUQSGD variants match the target accuracy of the 32-bit model, with non-trivial speedup over the standard data-parallel variant, directly proportional to the perepoch speedup. The QSGDinf heuristic yields similar accuracy and performance, and is therefore omitted. Second, we found that unfortunately EF-SignSGD does not converge under these standard hyperparameter settings. To address this issue, we performed a non-trivial amount of hyperparameter tuning for this algorithm: in particular, we found that the scaling factors and the bucket size must be carefully adjusted for convergence on ImageNet. We were able to recover full accuracy with EF-SignSGD on ResNet50, but that the cost of quantizing into buckets of size 64. Unfortunately, in this setting the algorithm transmits a non-trivial amount of scaling data, and the GPU implementation becomes less efficient due to error computation and reduced parallelism. The end-to-end speedup of this tuned variant is inferior to NUQSGD-4bit, and only slightly superior to that of NUQSGD-8bit. Please see Figure 9 in the Appendix and the accompanying text for details. We study data-parallel and communication-efficient version of stochastic gradient descent. Building on QSGD, we study a nonuniform quantization scheme. We establish upper bounds on the variance of nonuniform quantization and the expected code-length. In the overparametrized regime of interest, the former decreases as the number of quantization levels increases, while the latter increases with the number of quantization levels. Thus, this scheme provides a trade-off between the communication efficiency and the convergence speed. We compare NUQSGD and QSGD in terms of their variance bounds and the expected number of communication bits required to meet a certain convergence error, and show that NUQSGD provides stronger guarantees. Experimental are consistent with our theoretical and confirm that NUQSGD matches the performance of QSGDinf when applied to practical deep models and datasets including ImageNet. Thus, NUQSGD closes the gap between the theoretical guarantees of QSGD and empirical performance of QSGDinf. One limitation of our study which we aim to address in future work is that we focus on all-to-all reduction patterns, which interact easily with communication compression. In particular, we aim to examine the interaction between more complex reduction patterns, such as ring-based reductions , which may yield superior performance in bandwidthbottlenecked settings, but which interact with communication-compression in non-trivial ways, since they may lead a gradient to be quantized at each reduction step. Read that bit plus N following bits; The encoding, ENCODE(v), of a stochastic gradient is as follows: We first encode the norm v using b bits where, in practice, we use standard 32-bit floating point encoding. We then proceed in rounds, r = 0, 1, · · ·. On round r, having transmitted all nonzero coordinates up to and including t r, we transmit ERC(i r) where t r+1 = t r + i r is either (i) the index of the first nonzero coordinate of h after t r (with t 0 = 0) or (ii) the index of the last nonzero coordinate. In the former case, we then transmit one bit encoding the sign ρ t r+1, transmit ERC(log(2 s+1 h t r+1)), and proceed to the next round. In the latter case, the encoding is complete after transmitting ρ t r+1 and ERC(log(2 s+1 h t r+1)). The DECODE function (for Algorithm 1) simply reads b bits to reconstruct v. Using ERC −1, it decodes the index of the first nonzero coordinate, reads the bit indicating the sign, and then uses ERC −1 again to determines the quantization level of this first nonzero coordinate. The process proceeds in rounds, mimicking the encoding process, finishing when all coordinates have been decoded. Like, we use Elias recursive coding (, ERC) to encode positive integers. ERC is simple and has several desirable properties, including the property that the coding scheme assigns shorter codes to smaller values, which makes sense in our scheme as they are more likely to occur. Elias coding is a universal lossless integer coding scheme with a recursive encoding and decoding structure. The Elias recursive coding scheme is summarized in Algorithm 2. For any positive integer N, the following are known for ERC We first find a simple expression of the variance of Q s (v) for every arbitrary quantization scheme in the following lemma: ), and fix s ≥ 1. The variance of Q s (v) for general sequence of quantization levels is given by where r i = |v i |/ v and p(r),s(r), τ(r) are defined in Section 3.1. Proof. Noting the random quantization is i.i.d over elements of a stochastic gradient, we can decom- where In the following, we consider NUQSGD algorithm withL = (0, 1/2 s, · · ·, 2 s−1 /2 s, 1) as the quantization levels. Then, h i (v, s)'s are defined in two cases based on which quantization interval r i falls into: where p 1 r, s = 2 s r. where p 2 r, s = 2 s− j r − 1. Note that Q s = 0. Let S j denote the coordinates of vector v whose elements fall into the (j + 1)-th bin, i.e., S 0 {i : Applying the of Lemma 1, we have where τ j l j+1 − l j for j ∈ {0, · · ·, s}. Substituting τ 0 = 2 −s and τ j = 2 j−1−s for j ∈ {1, · · ·, s} into, we have We first note that ∑ i∈S 0 p 1 (r i, s) ≤ d and ∑ i∈S j+1 p 2 (r i, s) ≤ d for all j, i.e., an upper bound on the variance of Substituting the upper bounds in and into, an upper bound on the variance of Q s (v) is given by The upper bound in cannot be used directly as it depends on {d 0, · · ·, d s}. Note that d j's depend on quantization intervals. In the following, we obtain an upper bound on E[Q s (v) − v 2 ], which depends only on d and s. To do so, we need to use this lemma inspired by (, Lemma A.5): Let · 0 count the number of nonzero components. Lemma 2. Let v ∈ R d. The expected number of nonzeros in Q s (v) is bounded above by For each i ∈ S 0, Q s (v i) becomes zero with probability 1 − 2 s r i, which in Using a similar argument as in the proof of Lemma 2, we have We defined...... Noting that the coefficients of the additive terms in the upper bound in are monotonically increasing with j, we can find an upper bound on, which gives and completes the proof. Let | · | denote the length of a binary string. In this section, we find an upper bound on E[|ENCODE(v)], i.e., the expected number of communication bits per iteration. Recall from Appendix A that the quantized gradient Q s (v) is determined by the tuple (v, ρ ρ ρ, h). Write i 1 < i 2 < · · · < i h 0 for the indices of the h 0 nonzero entries of h. Let i 0 = 0. The encoding produced by ENCODE(v) can be partitioned into two parts, R and E, such that, for j = 1,..., h 0, • R contains the codewords ERC(i j − i j−1) encoding the runs of zeros; and • E contains the sign bits and codewords ERC(log{2 s+1 h i j}) encoding the normalized quantized coordinates. (, Lemma A.3), the properties of Elias encoding imply that We now turn to bounding |E|. The following in inspired by (, Lemma A.3). Lemma 3. Fix a vector q such that q p p ≤ P, let i 1 < i 2 <... i q 0 be the indices of its q 0 nonzero entries, and assume each nonzero entry is of form of 2 k, for some positive integer k. Then Proof. Applying property for ERC (end of Appendix A), we have where the last bound is obtained by Jensen's inequality. Taking q = 2 s+1 h, we note that q 2 = 2 2s+2 h 2 and By Lemma 3 applied to q and the upper bound, Combining and, we obtain an upper bound on the expected code-length: where It is not difficult to show that, for all k > 0, g 1 (x) x log k x is concave. Note that g 1 is an increasing function up to x = k/e. Defining g 2 (x) x log log C x and taking the second derivative, we have Hence g 2 is also concave on x < C. Furthermore, g 2 is increasing up to some C/5 < x * < C/4. We note that E[h 0] ≤ 2 2s + √ d2 s following Lemma 2. By assumption 2 2s + √ d2 s ≤ d/e, and so, Jensen's inequality and lead us to. Let g(w) andĝ(w) denote the full-precision and decoded stochastic gradients, respectively. Then By Theorem 2, The follows by Corollary 1. Notice that the variance for NUQSGD and QSGDinf is lower than SGD for almost all the training and it decreases after the learning rate drops. All methods except SGD simulate training using 8 GPUs. SuperSGD applies no quantization to the gradients and represents the lowest variance we could hope to achieve. Ignoring all but terms depending on d and s, we have T ε = O(B/ε 2). Following Theorems 2 and 3 for NUQSGD, ζ NUQSGD,ε = O(N Q ε Q B/ε 2). For QSGD, following the of In overparameterized networks, where d ≥ 2 2s+1, we have Figure 6: Accuracy on the hold-out set on CIFAR10 (left) and on ImageNet (right) for training ResNet models from random initialization until convergence. For CIFAR10, the hold-out set is the test set and for ImageNet, the hold-out set is the validation set. In this section, we present further experimental in a similar setting to Section 5. In Figure 6, we show the test accuracy for training ResNet110 on CIFAR10 and validation accuracy for training ResNet34 on ImageNet from random initialization until convergence (discussed in Section 5). Similar to the training loss performance, we observe that NUQSGD and QSGDinf outperform QSGD in terms of test accuracy in both experiments. In both experiments, unlike NUQSGD, QSGD does not recover the test accuracy of SGD. The gap between NUQSGD and QSGD on ImageNet is significant. We argue that this is achieved because NUQSGD and QSGDinf have lower variance relative to QSGD. It turns out both training loss and generalization error can benefit from the reduced variance. For different methods, the variance is measured on their own trajectories. Note that the normalized variance of NUQSGD and QSGDinf is lower than SGD for almost the entire training. It decreases on CIFAR10 after the learning rate drops and does not grow as much as SGD on ImageNet. Since the variance depends on the optimization trajectory, these curves are not directly comparable. Rather the general trend should be studied. We also measure the variance and normalized variance at fixed snapshots during training by evaluating multiple gradient estimates using each quantization method. All methods are evaluated on the same trajectory traversed by the single-GPU SGD. These plots answer this specific question: What would the variance of the first gradient estimate be if one were to train using SGD for any number of iterations then continue the optimization using another method? The entire future trajectory may change by taking a single good or bad step. We can study the variance along any trajectory. However, the trajectory of SGD is particularly interesting because it covers a subset of points in the parameter space that is likely to be traversed by any first-order optimizer. For multi-dimensional parameter space, we average the variance of each dimension. Figure 5 (left), shows the variance of the gradient estimates on the trajectory of single-GPU SGD on CIFAR10. We observe that QSGD has particularly high variance, while QSGDinf and NUQSGD have lower variance than single-GPU SGD. We also propose another measure of stochasticity, normalized variance, that is the variance normalized by the norm of the gradient. The mean normalized variance can be expressed as where l(w; z) denotes the loss of the model parametrized by w on sample z and subscript A refers to randomness in the algorithm, e.g., randomness in sampling and quantization. Normalized variance can be interpreted as the inverse of Signal to Noise Ratio (SNR) for each dimension. We argue that the noise in optimization is more troubling when it is significantly larger than the gradient. For sources of noise such as quantization that stay constant during training, their negative impact might only be observed when the norm of the gradient becomes small. Figure 5 (right) shows the mean normalized variance of the gradient versus training iteration. Observe that the normalized variance for QSGD stays relatively constant while the unnormalized variance of QSGD drops after the learning rate drops. It shows that the quantization noise of QSGD can cause slower convergence at the end of the training than at the beginning. In Figure 7, we show the mean normalized variance of the gradient versus training iteration on CIFAR10 and ImageNet. For different methods, the variance is measured on their own trajectories. Since the variance depends on the optimization trajectory, these curves are not directly comparable. Rather the general trend should be studied. ResNet152 Weak Scaling. In Figure 8, we present the weak scaling for ResNet152/ImageNet. Each of the GPUs receives a batch of size 8, and we therefore scale up the global batch size by the number of nodes. The exhibit the same superior scaling behavior for NUQSGD relative to the uncompressed baseline. EF-SignSGD Convergence. In Figure 9, we present a performance comparison for NUQSGD variants (bucket size 512) and a convergent variant of EF-SignSGD with significant levels of parameter tuning for convergence. We believe this to be the first experiment to show convergence of the latter method at ImageNet scale, as the original paper only considers the CIFAR dataset. For convergence, we have tuned the choice of scaling factor and the granularity at which quantization is applied (bucket size). We have also considered learning rate tuning, but that did not appear to prevent divergence in the early stages of training for this model. We did not attempt warm start, since that would significantly decrease the practicality of the algorithm. We have found that bucket size 64 is the highest at which the algorithm will still converge on this model and dataset, and found 1-bit SGD scaling , which consists of taking sums over positives and over negatives for each bucket, to yield good . The experiments are executed on a machine with 8 NVIDIA Titan X GPUs, and batch size 256, and can be found in Figure 9. Under these hyperparameter values the EF-SignSGD algorithm sends 128 bits per each bucket of 64 values (32 for each scaling factor, and 64 for the signs), doubling its baseline communication cost. Moreover, the GPU implementation is not as efficient, as error feedback must be computed and updated at every step, and there is less parallelism to leverage inside each bucket. This explains the fact that the end-to-end performance is in fact close to that of the 8-bit NUQSGD variant, and inferior to 4-bit NUQSGD. In the following theorem, we show that for any given set of levels, there exists a distribution of points with dimension d such that the variance is in Ω(√ d), and so our bound is tight in d. The variance optimization problem R 2 is an integer nonconvex problem. We can obtain an upper bound on the optimal objective of problem R 2 by relaxing the integer constraint as follows. The ing QSQP is shown as follows: Note that problem Q 1 can be solved efficiently using standard standard interior point-based solvers, e.g., CVX . In the following, we develop a coarser analysis that yields an upper bound expressed as the optimal value to an LP. Theorem 8 (LP bound). Let v ∈ R d. An upper bound on the nonuniform quantization of v is given by ε LP v 2 where ε LP is the optimal value of the following LP: Corollary 2 (Optimal level). For the special case with s = 1, the optimal level to minimize the worst-case bound obtained from problem P 1 is given by l * 1 = 1/2. Proof. For s = 1, problem P 1 is given by P 0: max Note that the objective of P 0 is monotonically increasing in (d 0, d 1). It is not difficult to verify that the optimal (d * 0, d * 1) is a corner point on the boundary line of the feasibility region of P 0. Geometrical representation shows that that candidates for an optimal solution are (d − (1/l 1) 2, (1/l 1) 2 ) and (d, 0). Substituting into the objective of P 0, the optimal value of P 0 is given by Finally, note that τ 0 = τ 1 = 1/2 minimizes the optimal value of P 0. In this section, we focus on the special case of exponentially spaced collection of levels of the form L p = (0, p s, · · ·, p 2, p, 1) for p ∈ and an integer number of levels, s. In this case, we have τ 0 = p s and τ j = (1 − p)p s− j for j = 1, · · ·, s. For any given s and d, we can solve the corresponding quadratic and linear programs efficiently to find the worst-case variance bound. As a bonus, we can find the optimal value of p that minimizes the worst-case variance bound. In Figure 10, we show the numerical obtained by solving QCQP Q 1 with L p versus p using CVX . In Figure 10 (left), we fix d and vary s, while in Figure 10 (right), we fix s and vary d. As expected, we note that the variance upper bound increases as d increases and the variance upper bound decreases as s increases. We observe that our current scheme is nearly optimal (in the worst-case sense) in some cases. Further, the optimal value of p shifts to the right as d increases and shifts to the left as s increases. We can obtain convergence guarantees to various learning problems where we have convergence guarantees for SGD under standard assumptions. On nonconvex problems, (weaker) convergence guarantees can be established along the lines of, e.g., (, Theorem 2.1). In particular, NUQSGD is guaranteed to converge to a local minima for smooth general loss functions. Theorem 9 (NUQSGD for smooth nonconvex optimization). Let f: Ω → R denote a possibly nonconvex and β -smooth function. Let w 0 ∈ Ω denote an initial point, ε Q be defined as in Theorem 2, T ∈ Z >0, and f * = inf w∈Ω f (w). Suppose that Algorithm 1 is executed for T iterations with a learning rate α = O(1/β) on K processors, each with access to independent stochastic gradients of f with a second-moment bound B. Then there exists a random stopping time R ∈ {0, · · ·, T} such that NUQSGD guarantees E[∇ f (w R) 2 ] ≤ ε where ε = O β (f (w 0) − f * )/T + (1 + ε Q)B.

represent the short summary for finding the corresponding scientific article (long)
Fewshot example: "first deep neural network for modeling Egocentric Spatial Memory inspired by neurophysiological discoveries of navigation cells in mammalian brain" == "Inspired by neurophysiological discoveries of navigation cells in the mammalian brain, we introduce the first deep neural network architecture for modeling Egocentric Spatial Memory (ESM). It learns to estimate the pose of the agent and progressively construct top-down 2D global maps from egocentric views in a spatially extended environment. During the exploration, our proposed ESM network model updates belief of the global map based on local observations using a recurrent neural network. It also augments the local mapping with a novel external memory to encode and store latent representations of the visited places based on their corresponding locations in the egocentric coordinate. This enables the agents to perform loop closure and mapping correction. This work contributes in the following aspects: first, our proposed ESM network provides an accurate mapping ability which is vitally important for embodied agents to navigate to goal locations. In the experiments, we demonstrate the functionalities of the ESM network in random walks in complicated 3D mazes by comparing with several competitive baselines and state-of-the-art Simultaneous Localization and Mapping (SLAM) algorithms. Secondly, we faithfully hypothesize the functionality and the working mechanism of navigation cells in the brain. Comprehensive analysis of our model suggests the essential role of individual modules in our proposed architecture and demonstrates efficiency of communications among these modules. We hope this work would advance research in the collaboration and communications over both fields of computer science and computational neuroscience. Egocentric spatial memory (ESM) refers to a memory system that encodes, stores, recognizes and recalls the spatial information about the environment from an egocentric perspective BID24. Such information is vitally important for embodied agents to construct spatial maps and reach goal locations in navigation tasks. For the past decades, a wealth of neurophysiological have shed lights on the underlying neural mechanisms of ESM in mammalian brains. Mostly through single-cell electrophysiological recordings studies in mammals BID23, there are four types of cells identified as specialized for processing spatial information: head-direction cells (HDC), border and boundary vector cells (BVC), place cells (PC) and grid cells (GC). Their functionalities are: According to BID38, HDC, together with view cells BID5, fires whenever the mammal's head orients in certain directions. The firing behavior of BVC depends on the proximity to environmental boundaries BID22 and directions relative to the mammals' heads BID1. PC resides in hippocampus and increases firing rates when the animal is in specific locations independent of head orientations BID1. GC, as a metric of space BID35, are regularly distributed in a grid across the environment BID11. They are updated based on animal's speed and orientation BID1. The corporation of these cell types enables mammals to navigate and reach goal locations in complex environments; hence, we are motivated to endow artificial agents with the similar memory capability but a computational architecture for such ESM is still absent. Inspired by neurophysiological discoveries, we propose the first computational architecture, named as the Egocentric Spatial Memory Network (ESMN), for modeling ESM using a deep neural network. ESMN unifies functionalities of different navigation cells within one end-to-end trainable framework and accurately constructs top-down 2D global maps from egocentric views. To our best knowledge, we are the first to encapsulate the four cell types respectively with functionally similar neural networkbased modules within one integrated architecture. In navigation tasks, the agent with the ESMN takes one egomotion from a discrete set of macro-actions. ESMN fuses the observations from the agent over time and produces a top-down 2D local map using a recurrent neural network. In order to align the spatial information at the current step with all the past predicted local maps, ESMN estimates the agent's egomotion and transforms all the past information using a spatial transformer neural network. ESMN also augments the local mapping module with a novel spatial memory capable of integrating local maps into global maps and storing the discriminative representations of the visited places. The loop closure component will then detect whether the current place was visited by comparing its observation with the representations in the external memory which subsequently contributes to global map correction. Neuroscience-inspired AI is an emerging research field BID12. Our novel deep learning architecture to model ESMN in the mammalian navigation system attempts to narrow the gap between computer science (CS) and computational neuroscience (CN) and bring interests to both communities. On one hand, our novel ESMN outperforms several competitive baselines and the state-of-the-art monocular visual SLAMs. Our outstanding performance in map construction brings great advancements in robotics and CS. It could also have many potential engineering applications, such as path planning for robots. In CN, the neuroplausible navigation system with four types of cells integrated is still under development. In our work, we put forward bold hypothesis about how these navigation cells may cooperate and perform integrated navigation functions. We also faithfully propose several possible communication links among them in the form of deep architectures. We evaluate ESMN in eight 3D maze environments where they feature complex geometry, varieties of textures, and variant lighting conditions. In the experiments, we demonstrate the acquired skills of ESMN in terms of positional inference, free space prediction, loop closure classification and map correction which play important roles in navigation. We provide detailed analysis of each module in ESMN as well as their functional mappings with the four cell types. Lastly, we conduct ablation studies, compare with state-of-the-art Simultaneous Localization And Mapping (SLAM) algorithms and show the efficacy of our integrated framework on unifying the four modules. There is a rich literature on computational models of egocentric spatial memory (ESM) primarily in cognitive science and AI. For brevity, we focus on the related works in machine learning and robotics. Reward-based learning is frequently observed in spatial memory experiments BID14. In machine learning, reinforcement learning is commonly used in attempts to mathematically formalize the reward-based learning. Deep Q-networks, one of the reinforcement learning frameworks, have been employed in the navigation tasks where the agent aims to maximize the rewards while navigating to the goal locations BID26; BID28. The representation of spatial memory is expressed implicitly using long short term memory (LSTM) BID13. In one of the most relevant works BID10, Gupta et al.introduced a mapper-planner pipeline where the agent is trained in a supervised way to produce a top-down belief map of the world and thus plans path; however, the authors assume that the agents perform mapping tasks in an ideal scenario where the agent knows which macro-action to take and takes the macro-action without control errors at each time step. Different from their work, our ESM network takes into account the action uncertainty and predicts the agent's pose based on egocentric views. Moreover, we propose the mechanism of loop closure and map correction for long-term exploration. Apart from reinforcement learning, there are works on navigation in the domain of robotics where the spatial memory is often explicitly represented in the form of grid maps or topological structuresElfes; BID33; BID20. SLAM tackles the problem of positional inference and mapping BID36 BID4; BID3; BID39. While classical SLAM achieves good performances with the aid of multimodal sensors BID21, SLAM using monocular cameras still has limitations where the feature matching process relies heavily on hand-crafted features extracted from the visual scenes BID40; BID34. As great strides have been made using deep learning in computer vision tasks which in a significant performance boost, several works endeavor to replace parts of the classical SLAM workflows with deep learning based modules BID17; BID0; BID16. However, there is no existing end-to-end neural network for visual SLAM to our best knowledge. Inspired by the neurophysiological , we propose the first unified ESM model using deep neural networks. This model simultaneously solves the problems of positional inference, intrinsic representation of places and space geometry during the process of 2D map construction. Before we elaborate on our proposed architecture, we introduce the Egocentric Spatial Memory Network (ESMN) modeling problem and relevant notations. ESM involves an object-to-self representational system which constantly requires encoding, transforming and integrating spatial information from the first-person view into a global map. At the current time step t, the agent, equipped with one RGB camera, takes the camera view I t as the current visual input. Without the aid of any other sensors, it predicts the egomotion from the pair of camera views (I t−1, I t). At time step t, the agent is allowed to take only one egomotion A θ,d,l,t out of a discrete set of macro-actions which include rotating left/right by θ degrees and moving in the directions l: forward/backward/left/right by the distance of d relative to the agent's current pose. We assume the agent moves in a 2D grid world and the camera coordinate is fixed with respect to the agent's body. The starting location p 0 of the agent is always where the triplet denotes the positions along the x and y-axis and the orientation in the world coordinate. The problem of modeling ESM is to learn a global map in the egocentric coordinate based on the visual input pairs (I t−1, I t) for t = 1,..., T. We define the global map as a top-view 2D probabilistic grid map where the probability infers the agent's belief of free space. For precise controls of the experimental variables, we consider the egocentric spatial memory (ESM) modeling problem in artificial 3D maze environments. In order to tackle this problem, we propose a unified neural network architecture named as ESMN. The architecture of ESMN is illustrated in FIG0 and is elaborated in details in Figure 2. Inspired by the navigation cells mentioned in the introduction, our proposed ESMN comprises four modules: Head Direction Unit (HDU), Boundary Vector Unit (BVU), Place Unit (PU), and Grid Unit (GU). This is to incorporate multiple objectives for modeling ESM. spatial information to the current egocentric coordinate via a 2D-CNN. PU learns to encode the latent representation of visited places in a 2D-CNN pre-trained using a triplet loss. GU, as a memory, integrates the predicted local maps from BVU over time and keeps track of all the visited places. In GU, we add a gating mechanism to constantly detect the loop closure and thus eliminate the accumulated errors during long-term mapping. In this paper, we adopt a "divide and conquer" approach by composing different navigation modules within one framework systematically. The division of these modules are inspired by scientific supports which have shown the advantages in biological systems BID31. Leveraging on rich features extracted from deep networks, the learnt features are suitable for recognizing visual scenes and hence, boost up the map construction and correction performances. The efficiency and robustness of our algorithm are demonstrated by comparing with other spatial navigation methods replied on visual inputs. At each time step, the RGB camera image (or frame) is the only input to the agent. In order to make spatial reasoning about the topological structure of the spatially extended environment, the agent has to learn to take actions to explore its surroundings and predict their own poses by integrating the egomotion over time. The HDU module in our proposed ESMN model employs a 2D-CNN to predict the macro-action the agent has taken at the time step t based on the inputs of two camera views {I t−1, I t}. Though there are two possible implementations for the loss in training the HDU module, i.e. regression and classification, we consider the latter in order to reduce the dimensionality of action space. The macro-actions A θ,d,l are discretized into 2 classes for rotation and 4 classes for translation as explained at the beginning of Section 3. Though we solve the problem of positional inference using a feed-forward neural network based on two most recent observations alone, the extensions to use recurrent neural network would be interesting to explore in the future. We explain how ESMN integrates egocentric views into a top-down 2D representation of the environment using a recurrent neural network. Similar to BID10, the BVU in ESMN serves as a local mapper and maintains the accumulative free space representations in the egocentric coordinate for a short-term period. Given the current observation I t, function g first encodes its geometric representation about space g(I t) via a 2D-CNN and then transform g(I t) into egocentric top-down view m t via de-convolutional layers. Together with the accumulative space representation m t−1 at the previous step and the estimated egomotion A θ,d,l,t from t − 1 to t, BVU estimates the current local space representation m t using the following update rule: DISPLAYFORM0 where W is a function that transforms the previous accumulative space representation m t−1 to the current egocentric coordinate based on the estimated egomotion A θ,d,l,t. We parameterize W by using a spatial transformer network BID15 composing of two key elements: it generates the sampling grid which maps the input coordinates of m t−1 to the corresponding output locations after egomotion A θ,d,l,t transformation; the sampling kernel then takes the bilinear interpolation of the values on m t−1 and outputs the transformed m t−1 in the current egocentric coordinate. U is a function which merges the free space prediction m t from the current observation with the accumulative free space representation at the previous step. Specifically, we simplify merging function U as a weighted summation parameterized by λ followed by hyperbolic tangent operation: DISPLAYFORM1 Loop closure is valuable for the agents in the navigation tasks as it encourages efficient exploration and spatial reasoning. In order to detect loop closure during an episode, given the current observation I t, ESMN learns to encode the discriminative representation h(I t) of specific places independent of scaling and orientations via an embedding function F. Based on the similarity of all the past observations Ω t = {I 1, I 2, ..., I t} at corresponding locations P t = {p 1, p 2, ..., p t}, we create training targets by making an analogy to the image retrieval tasks BID8 and define the triplet (I t, I +, I −) as anchor sample (current camera view), positive and negative samples drawn from Ω t respectively. PU tries to minimize the triplet loss: DISPLAYFORM0 where we parameterize F using a three-stream 2D-CNN where the weights are shared across streams. D is a distance measure between pairs of embedding. Here, mean squared error is used. A loop closure label equals 1 if the mean squared error is below the threshold α which is set empirically. Apart from the similarity comparison of observations, we consider two extra criteria for determining whether the place was visited: the current position of the agent p t is near to the positions visited at the earlier times. We empirically set a threshold distance between the current and the recently visited locations based on the loop closure accuracy during training. We implemented it via a binary mask in the center of the egocentric global map where the binary states denote the accepted "closeness". the agent only compares those positions which are far from the most recent visited positions to avoid trivial loop closure detection at consecutive time steps. It is implemented via another binary mask which tracks these recently visited places. These criterion largely reduce the false alarm rate and improves the searching speed during loop closure classification. While BVU provides accumulative local free space representations in high resolution for a short-term period, we augment the local mapping framework with memory for long-term integration of the local maps and storage of location representations. Different from Neural Turing Machine BID9 where the memory slots are arranged sequentially, our addressable memory, of size F × H × W, is indexed by spatial coordinates {(i, j): i ∈ {1, 2, ..., H}, j ∈ {1, 2, ..., W}} with memory vector M (i, j) of size F at location (i, j). Because ESM is often expressed in the coordinate frame of the agent itself, we use location-based addressing mechanism and the locations of reading or writing heads are fixed to be always in the center of the memory. Same as BVU, all the past spatial information in the memory is transformed based on the estimated egomotion A θ,d,l,t. Mathematically, we formulate the returned reading vector r h,w as DISPLAYFORM0 where the memory patch covers the area of memory vectors with the width w and height h. We simplify the writing mechanism for GU and use Equation 4 for writing the vector r h,w. Figure 3: Overview of maze layouts, with differing geometries, textures and lighting conditions. Each maze is stimulated with normal, weak and strong lighting conditions. Maze 5_S and 5_W refers to Maze 5 with strong and weak lighting conditions respectively. Maze 1 is adopted from. Maze 8 is inspired by radial arm maze used to test spatial memory in rats BID32. The digits are pasted on walls along specific pathways for loop closure classification tasks. In our case, two writing heads and two reading heads are necessary to achieve the following objectives: one reading head returns the memory vectors m t−1 in order for BVU to predict m t using Equation 1; GU performs updates by writing the predicted local accumulative space representation m t back into the memory to construct the global map in the egocentric coordinate; GU keeps track of the visited places by writing the discriminative representation h(I t) at the center of the egocentric global map denoted as (DISPLAYFORM1 W 2); GU returns the memory vectors near to the current location for loop closure classification where the size of the searching area is parameterized by w and h. The choice of w and h are application-based. We simplify the interaction between local space representations m t and m t−1 with GU and set w and h to be the same size as m t and m t−1.If the loop closure classifies the current observation as "visited", GU eliminates the discrepancies on the global map by merging the two places together. The corrected map has to preserve the topological structure in the discovered areas and ensure the connectivity of the different parts on the global map is maintained. To realize this, we take three inputs in a stack of 3D convolution and de-convolution layers for map correction. The inputs are: the local map predicted at the anchor place; the local map predicted at the recalled place in the current egocentric coordinate; all the past integrated global maps. To make the training targets, we perturb the sequence of ground truth egomotions with random ones and generate synthetic integrated global maps with rotation and scaling augmentations. We minimize regression loss between the predicted maps and the ground truth. We train ESMN end-to-end by stochastic gradient descent with learning rate 0.002 and momentum 0.5. Adam Optimizer BID18 is used. At each time step, the ground truths are provided: local map, egomotion and loop closure classification label. For faster training convergence, we first train each module separately and then load these pre-trained networks into ESMN for fine-tuning. The input frame size is 3 × 64 × 64. We normalize all the input RGB images to be within the range [−1, 1]. The batch size is 10. The discriminative representation h(I t) is of dimension 128. The size of the local space representation m t is h × w = 32 × 32 covering the area 7.68 meters × 7.68 meters in the physical world whereas the global map (the addressable memory) is of size H × W = 500 × 500. We set λ = 0.5 in BVU and α = 0.01 in PU. The memory vector in GU is of size F = 128. Refer to Supplementary Material for detailed architecture. We conduct experiments on eight 3D mazes in Gazebo for robotic simulation BID19. We will make our simulation source code public once our paper is accepted. Figure 3 shows an overview of eight mazes. They feature complex geometry and rich varieties of textures. To prevent over-fitting in a single-modality environment, we augment these observations with different illumination conditions classified into three categories: strong, normal and weak, and train our ESMN in these mixed lighting conditions. Based on the in Section 4.1, our ESMN performs equally well in the test set across permutations of these conditions which shows its capacity of generalizing in various environments. For loop closure classification tasks, we create digits on walls as the unique features of specific pathways. The agent randomly walks through the mazes with obstacle avoidance. The ground truths for 2D top-view of mazes are obtained by using one virtual 2D laser scanner attached to the agent. We use the data collected in maze 1 to 5 for training and validation and the data in maze 6 to 8 for testing. The macro-actions for egomotions are divided into 2 classes for rotation by θ = 10 degrees and 4 classes for translation with d = 0.1 meters relative to the agent's current pose in the physical world. The simulation environment enables us to conduct exhaustive evaluation experiments and train individual module separately using fully supervised signals for better functionality evaluation. The integrated framework is then trained jointly after loading the pre-trained weights from individual modules. Head Direction Unit (HDU): Positional Inference. Head Direction Cells fire whenever the animal's head orients in certain directions. The study BID25 suggests that the rodent's orientation system is as a of neural integration of head angular velocities derived from vestibular system. Thus, after jointly training ESMN, we decode HDU for predicting head direction in the world coordinate by accumulating the estimated egomotions over time. FIG2 shows the head direction prediction in the world coordinate compared with the agent's ground truth global poses. We observe that the head directions integrated from the estimated egomotions are very similar to the ground truths. Moreover, we compute the egomotion classification errors from 2 classes of rotation and 4 classes of translation under three illumination conditions, see Table 1. There is a slight degradation of rotation + translation compared with rotation alone. One possible reason is that the change of egocentric camera views after 10 degrees of rotation is more observable than the one induced from translation by 0.1 meters. We also provide quantitative analysis of our predicted local maps across the first 32 time steps in FIG4 using Mean Squared Error (MSE), Correlation (Cor), Mutual Information (MI) which are standard image similarity metrics. At each time step, the predicted local maps are compared with the ground truth maps at t = 32. As the agent continues to explore in the environment, the area of the predicted free space expands leading to the decrease of MSE and the increase of correlation and MI in our test set. This validates that ESMN is able to accurately estimate the proximity to the physical obstacles relative to the agent itself and continuously accumulate the belief of free space representations based on the egomotions. Place Unit (PU): Loop Closure Classification. One of the critical aspects of ESM is the ability to recall and recognize a previously visited place independent of head orientation. Place cell (PC) resides in hippocampus and increases firing patterns when the animal is in specific locationsBurgess. Recent anatomical connectivity research BID37 implies that grid cells (GC) are the principle cortical inputs to PC. In our proposed ESMN, we design a novel memory unit (GU) for storing the discriminative representations of visited places based on their relative locations with respect to the agent itself in the egocentric coordinate. GU interacts constantly with PU by reading and writing the memory vectors. We evaluate the learnt discriminative representations of visited Table 2: Ablation study on the global map performance in Maze 6 at t = 448 and t = 1580 using metrics in Section 4.1. From top to bottom, the models are: 3D-CNN baseline, LSTM baseline, our ablated model with PU and GU removed, our proposed architecture. The best values are highlighted in bold.places by comparing the current observation (anchor input) with all the past visited places. FIG5 presents example pairs of observations when the loop closure is detected. Qualitative imply that our ESMN can accurately detect loop closure when the agent is at the previously visited places irrespective of large differences between these camera views. Grid Unit: Mapping Integration and Correction. After the loop closure is detected, ESMN performs map correction to eliminate the discrepancies during long-term mapping. FIG6 presents the example of the predicted global maps after map correction. It is shown that the map gets corrected at t = 448 (Column 1). Thus, the predicted global map (Row 3) is structurally more accurate compared with the one without loop closure (Row 2). Our ablation analysis is as follows: FORMULA0 To study the necessity of egomotion estimation from HDU, we take a sequence of camera views at all the previous time steps as inputs to predict the global map directly. We implement this by using a feed-forward 3D convolution neural network (3D-CNN). Practically, since it is hard to take all the past camera views across very long time period, we choose the input sequence with one representative frame every 15 time steps. As ESM requires sustainable mapping over long durations, we create one more baseline by taking the sequence of camera views as inputs and using Long Short Term Memory architecture to predict global maps directly (LSTM Direct). To maintain the same model complexity, we attach the same 2D-CNN in our BVU module before LSTM and fully connected layers after LSTM. To explore the effect of loop closure and thus map correction, we create one ablated model with PU and GU removed (HDU + BVU). FORMULA3 We present the of our integrated architecture with loop closure classification and map correction enabled (HDU + BVU + PU + GU). We report the evaluation in Table 2 using the metrics MSE, correlation and MI as introduced in Section 4.1.We observe that our proposed architecture surpasses the competitive baselines and the ablated models. At t = 448, compared with the first baseline (3D-CNN), there is decrease of 0.03 in MSE and increase of 0.17 in correlation and 0.09 in MI. The significant improvement infers that it is necessary to estimate egomotion for better map construction. Additionally, the integration of local maps based on the egomotion makes the computation more flexible and efficient by feeding back the accumulative maps to the system for future time steps. In the second baseline (LSTM Direct), we observe that the performance drops significantly when it constructs global maps for longer durations. As GU serves as an external memory to integrate local maps, the baseline confirms GU has advantages over LSTMDirect in terms of long-lasting memory. To explore the effect of loop closure and thus map correction, we have the ablated model with PU and GU removed (HDU + BVU Table 3 : Comparison of our method with state-of-the-art SLAM algorithms. The global map after the agent completes one loop closure in Maze 6, 7, and 8 are reported using metrics introduced in Section 4.1. The numbers highlighted in bold are the best.our proposed architecture with all four modules enabled at t = 448 and t = 1580, the decreased performance validates that these steps are necessary to eliminate the errors during long-term mapping. We evaluate our ESMN by comparing with state-of-the-art monocular visual SLAM methods, , and . We used the codes provided by the authors and replaced the intrinsic camera parameters with ours. In order to construct maps in the world scale, we provide explicit scaling factor for both methods. Results show that our ESMN significantly outperforms the rest in terms of MSE, Cor and MI as shown in Table 3 . In the experiments, we also observe that ORBslam has high false positives in loop closure detection which leads to incorrect map correction. It may be caused by the limitations of local feature descriptors. Different from them, our ESMN can robustly detect loop closure and correct maps over time. Furthermore, we notice that both SLAMs tend to fail to track feature descriptors between frames especially during rotations. The parameters of searching windows in order to match features have to be manually set and they cannot be adapted in large motion cases. The in Maze 8 using EKFslam are not provided in Table 3 as the original algorithm requires the dense local feature matching and it is not applicable in very low texture environments. We get inspirations from neurophysiological discoveries and propose the first deep neural network architecture for modeling ESM which unifies the functionalities of the four navigation cell types: head-direction cells, border cells and boundary vector cells, place cells and grid cells. Our learnt model demonstrates the capacity of estimating the pose of the agent and constructing a top-down 2D spatial representations of the physical environments in the egocentric coordinate which could have many potential applications, such as path planning for robot agents. Our ESMN accumulates the belief about the free space by integrating egocentric views. To eliminate errors during mapping, ESMN also augments the local mapping module with an external spatial memory to keep track of the discriminative representations of the visited places for loop closure detection. We conduct exhaustive evaluation experiments by comparing our model with some competitive baselines and state-of-the-art SLAM algorithms. The experimental demonstrate that our model surpasses all these methods. The comprehensive ablation study suggests the essential role of individual modules in our proposed architecture and the efficiency of communications among these modules. We introduce the anatomy of our model for reproducibility in this section. We follow exactly the same convention as Torch. Inspired by neurophysilogical discoveries, our framework consists of four units: Head Direction Unit (HDU), Boundary Vector Unit (BVU), Place Unit (PU), and Grid Unit (GU). Refer to Table 7 for HDU, Table 6 for BVU, Table 8 for PU, Table 9 for global map correction. In GU, as explained in Section 3.5 in the main manuscript, egocentric spatial memory is often expressed in the coordinate frame of the agent itself, we use location-based addressing mechanism and the locations of reading or writing heads are fixed to be always in the center of the external memory. As the receptive field parameterized by w and h for reading and writing are fixed. We designed a 2D binary masking map where 1 denotes the place Table 5: Reading Head 2 and Writing Head 2 in Grid Unit (GU). It either reads memory vectors or writes the discriminative spatial representation h(I t) of size 1 × 128 from or into the global map of size 128 × 500 × 500 in GU.to read and write and vice versa. We use element-wise multiplication between each feature map of the external memory and the binary masking map for reading and writing operations. Refer to TAB5 for reading and writing local maps in GU, and Table 5 for reading and writing spatial representations of visited places in GU.Figure 9: Example of predicted local maps over first 32 time steps in Maze 6. Frames #1, 5, 9, 13, 17, 21, 25, 29, 32 are shown (left to right columns). The topmost row shows the camera view. Row 2 shows the ground truth with red arrows denoting the agent's position and orientation from the top view. The white region denotes free space while the black denotes unknown areas. Row 3 shows the corresponding top-view accumulative belief of the predicted local maps where the red color denotes higher belief of the free space. Best viewed in color. current camera view estimated egomotion predicted local map at previous time step nn. Sequential nn. ParallelTable nn. SpatialConvolution nn. AffineGridGeneratorBHWD(A, A) nn. Reshape(batchSize, 1, 32, 32) nn. ReLU(true) nn. Transpose({3,4},{2,4}) nn. SpatialConvolution nn. BilinearSamplerBHWD nn. SpatialBatchNormalization (256,1e-3) nn. Transpose({2,4},{3,4}) nn. ReLU(true) nn. Reshape(batchSize, 32, 32) nn. SpatialConvolution nn. ReLU nn. SpatialBatchNormalization (512,1e-3) nn. ReLU(true) nn. SpatialConvolution nn. SpatialBatchNormalization(1024,1e-3) nn. ReLU(true) nn. SpatialFullConvolution nn. SpatialBatchNormalization(512,1e-3) nn. ReLU(true) nn. SpatialFullConvolution nn. SpatialBatchNormalization(256,1e-3) nn. ReLU(true) nn. SpatialFullConvolution nn. SpatialBatchNormalization(128,1e-3) nn. ReLU(true) nn. SpatialFullConvolution nn. SpatialBatchNormalization(1,1e-3) nn. ReLU(true) nn. Squeeze nn. View (batchSize, 32, 32) nn. CAddTable nn. HardTanh Table 6: Architecture of Boundary Vector Unit (BVU) FIG0: Example of predicted local maps over first 32 time steps in Maze 7. Frames #1, 5, 9, 13, 17, 21, 25, 29, 32 are shown (left to right columns). The topmost row shows the camera view. Row 2 shows the ground truth with red arrows denoting the agent's position and orientation from the top view. The white region denotes free space while the black denotes unknown areas. Row 3 shows the corresponding top-view accumulative belief of the predicted local maps where the red color denotes higher belief of the free space. Best viewed in color. 5, 9, 13, 17, 21, 25, 29, 32 are shown (left to right columns). The topmost row shows the camera view. Row 2 shows the ground truth with red arrows denoting the agent's position and orientation from the top view. The white region denotes free space while the black denotes unknown areas. Row 3 shows the corresponding top-view accumulative belief of the predicted local maps where the red color denotes higher belief of the free space. Best viewed in color.contacted two RGB camera views at t and t − 1 nn. Sequential nn. SpatialConvolution nn. ReLU(true) nn. SpatialConvolution nn. SpatialBatchNormalization(256,1e-3) nn. ReLU(true) nn. SpatialConvolution nn. SpatialBatchNormalization(512,1e-3) nn. ReLU(true) nn. SpatialConvolution nn. SpatialBatchNormalization(1024,1e-3) nn. ReLU(true) nn. SpatialConvolution nn. SpatialBatchNormalization(1024,1e-3) nn. ReLU(true) nn. Reshape nn. Linear nn. ReLU nn. Linear nn. ReLU nn. Linear nn. CrossEntropyCriterion Table 7: Architecture of Head Direction Unit (HDU)"

NUQSGD closes the gap between the theoretical guarantees of QSGD and the empirical performance of QSGDinf.

represent the tldr for finding the corresponding scientific article

We propose a extension of the batch normalization, show a first-of-its-kind convergence analysis for this extension and show in numerical experiments that it has better performance than the original batch normalizatin.