Question: Represent the text to find a very short summary (1-2 sentences max)

Question: We propose a novel deep network architecture for lifelong learning which we refer to as Dynamically Expandable Network (DEN), that can dynamically decide its network capacity as it trains on a sequence of tasks, to learn a compact overlapping knowledge sharing structure among tasks. DEN is efficiently trained in an online manner by performing selective retraining, dynamically expands network capacity upon arrival of each task with only the necessary number of units, and effectively prevents semantic drift by splitting/duplicating units and timestamping them. We validate DEN on multiple public datasets in lifelong learning scenarios on multiple public datasets, on which it not only significantly outperforms existing lifelong learning methods for deep networks, but also achieves the same level of performance as the batch model with substantially fewer number of parameters. Lifelong learning BID13, the problem of continual learning where tasks arrive in sequence, is an important topic in transfer learning. The primary goal of lifelong learning is to leverage knowledge from earlier tasks for obtaining better performance, or faster convergence/training speed on models for later tasks. While there exist many different approaches to tackle this problem, we consider lifelong learning under deep learning to exploit the power of deep neural networks. Fortunately, for deep learning, storing and transferring knowledge can be done in a straightforward manner through the learned network weights. The learned weights can serve as the knowledge for the existing tasks, and the new task can leverage this by simply sharing these weights. Therefore, we can consider lifelong learning simply as a special case of online or incremental learning, in case of deep neural networks. There are multiple ways to perform such incremental learning BID12 BID17. The simplest way is to incrementally fine-tune the network to new tasks by continuing to train the network with new training data. However, such simple retraining of the network can degenerate the performance for both the new tasks and the old ones. If the new task is largely different from the older ones, such as in the case where previous tasks are classifying images of animals and the new task is to classify images of cars, then the features learned on the previous tasks may not be useful for the new one. At the same time, the retrained representations for the new task could adversely affect the old tasks, as they may have drifted from their original meanings and are no longer optimal for them. For example, the feature describing stripe pattern from zebra, may changes its meaning for the later classification task for classes such as striped t-shirt or fence, which can fit to the feature and drastically change its meaning. Then how can we ensure that the knowledge sharing through the network is beneficial for all tasks, in the online/incremental learning of a deep neural network? Recent work suggests to either use a regularizer that prevents the parameters from drastic changes in their values yet still enables to find a good solution for the new task BID4, or block any changes to the old task BID4 retrains the entire network learned on previous tasks while regularizing it to prevent large deviation from the original model. Units and weights colored in red denote the ones that are retrained, and black ones are ones that remain fixed. (b) Non-retraining models such as Progressive Network BID12 expands the network for the new task t, while withholding modification of network weights for previous tasks. (c) Our DEN selectively retrains the old network, expanding its capacity when necessary, and thus dynamically deciding its optimal capacity as it trains on. parameters BID12. Our strategy is different from both approaches, since we retrain the network at each task t such that each new task utilizes and changes only the relevant part of the previous trained network, while still allowing to expand the network capacity when necessary. In this way, each task t will use a different subnetwork from the previous tasks, while still sharing a considerable part of the subnetwork with them. FIG0 illustrates our model in comparison with existing deep lifelong learning methods. There are a number of challenges that need to be tackled for such incremental deep learning setting with selective parameter sharing and dynamic layer expansion. 1) Achieving scalability and efficiency in training: If the network grows in capacity, training cost per task will increasingly grow as well, since the later tasks will establish connections to a much larger network. Thus, we need a way to keep the computational overhead of retraining to be low.2) Deciding when to expand the network, and how many neurons to add: The network might not need to expand its size, if the old network sufficiently explains the new task. On the other hand, it might need to add in many neurons if the task is very different from the existing ones. Hence, the model needs to dynamically add in only the necessary number of neurons.3) Preventing semantic drift, or catastrophic forgetting, where the network drifts away from the initial configuration as it trains on, and thus shows degenerate performance for earlier examples/tasks. As our method retrains the network, even partially, to fit to later learned tasks, and add in new neurons which might also negatively affect the prior tasks by establishing connections to old subnetwork, we need a mechanism to prevent potential semantic drift. To overcome such challenges, we propose a novel deep network model along with an efficient and effective incremental learning algorithm, which we name as Dynamically Expandable Networks (DEN). In a lifelong learning scenario, DEN maximally utilizes the network learned on all previous tasks to efficiently learn to predict for the new task, while dynamically increasing the network capacity by adding in or splitting/duplicating neurons when necessary. Our method is applicable to any generic deep networks, including convolutional networks. We validate our incremental deep neural network for lifelong learning on multiple public datasets, on which it achieves similar or better performance than the model that trains a separate network for each task, while using only 11.9%p − 60.3%p of its parameters. Further, fine-tuning of the learned network on all tasks obtains even better performance, outperforming the batch model by as much as 0.05%p − 4.8%p. Thus, our model can be also used for structure estimation to obtain optimal performance over network capacity even when batch training is possible, which is a more general setup. Lifelong learning Lifelong learning BID13 is the learning paradigm for continual learning where the model learns from a sequence of tasks while transferring knowledge obtained from earlier tasks to later ones. Since its inception of idea by BID13, it has been extensively studied due to its practicality in scenarios where the data arrives in streams, such as in autonomous driving or learning of robotic agents. Lifelong learning is often tackled as an online multi-task learning problem, where the focus is on efficient training as well as on knowledge transfer. BID3 suggest an online lifelong learning framework (ELLA) that is based on an existing multi-task learning formulation BID7 ) that efficiently updates latent parameter bases for a sequence of tasks, by removing dependency to previous tasks for the learning of each task predictor, and preventing retraining previous task predictors. Recently, lifelong learning is studied in deep learning frameworks; since lifelong learning of a deep network can be straightforwardly done by simple re-training, the primary focus of research is on overcoming catastrophic forgetting BID4 BID12 BID16 BID10.Preventing catastrophic forgetting Incremental or lifelong learning of deep networks in the problem known as catastrophic forgetting, which describes the case where the retraining of the network for new tasks in the network forgetting what are learned for previous tasks. One solution to this problem is to use a regularizer that prevents the new model from deviating too much from the previous one, such as 2 -regularizer. However, use of the simple 2 -regularizer prevents the model from learning new knowledge for the new tasks, which in suboptimal performances on later tasks. To overcome this limitation, BID4 proposed a method called Elastic Weight Consolidation (EWC) that regularizes the model parameter at each step with the model parameter at previous iteration via the Fisher information matrix for the current task, which enables to find a good solution for both tasks. BID16 proposed a similar approach, but their approach computes the per-synapse consolidation online, and considers the entire learning trajectory rather than the final parameter value. Another way to prevent catastrophic forgetting is to completely block any modifications to the previous network, as done in BID12, where at each learning stage the network is expanded with a subnetwork with fixed capacity that is trained with incoming weights from the original network, but without backpropagating to it. Dynamic network expansion There are few existing works that explored neural networks that can dynamically increase its capacity during training. BID17 propose to incrementally train a denoising autoencoder by adding in new neurons for a group of difficult examples with high loss, and later merging them with other neurons to prevent redundancy. Recently, BID11 propose a nonparametric neural network model which not only learns to minimize the loss but also find the minimum dimensionality of each layer that can reduce the loss, with the assumption that each layer has infinite number of neurons. BID2 also propose a network that can adaptively learn both the structure and the weights to minimize the given loss, based on boosting theory. However, none of these work considered multi-task setting and involves iterative process of adding in neurons (or sets of neurons), while our method only needs to train the network once for each task, to decide how many neurons to add. BID15 propose a method to incrementally train a network for multi-class classification, where the network not only grows in capacity, but forms a hierarchical structure as new classes arrive at the model. The model, however, grows and branches only the topmost layers, while our method can increase the number of neurons at any layer. We consider the problem of incremental training of a deep neural network under the lifelong learning scenario, where unknown number of tasks with unknown distributions of training data arrive at the model in sequence. Specifically, our goal is to learn models for a sequence of T tasks, t = 1,..., t,..., T for unbounded T where the task at time point t comes with training data DISPLAYFORM0. Note that each task t can be either a single task, or comprised of set of subtasks. While our method is generic to any kinds of tasks, for simplification, we only consider the binary classification task, that is, y ∈ {0, 1} for input feature x ∈ R d. It is the main challenge in the lifelong learning setting that all the previous training datasets up to t − 1 are not available at the current time t (only the model parameters for the previous tasks are accessible, if any). The lifelong learning agent at time t aims to learn the model parameter W t by solving following problem: DISPLAYFORM1 Figure 2: Incremental learning of a dynamically expandable network: Left: Selective retraining. DEN first identifies neurons that are relevant to the new tasks, and selectively retrains the network parameters associated with them. Center: Dynamic network expansion. If the selective retraining fails to obtain desired loss below set threshold, we expand the network capacity in a top-down manner, while eliminating any unnecessary neurons using group-sparsity regularization. Right: Network split/duplication. DEN calculates the drift ρ where L is task specific loss function, W t is the parameter for task t, and Ω(W t) is the regularization (e.g. element-wise 2 norm) to enforce our model W t appropriately. In case of a neural network which is our primary interest, DISPLAYFORM2 is the weight tensor. To tackle these challenges of lifelong learning, we let the network to maximally utilize the knowledge obtained from the previous tasks, while allowing it to dynamically expand its capacity when the accumulated knowledge alone cannot sufficiently explain the new task. Figure 2 and Algorithm 1 describes our incremental learning process. In following subsections, we describe each component of our incremental learning algorithm in detail: 1) Selective retraining, 2) Dynamic network expansion, and 3) Network split/duplication. Selective Retraining. A most naive way to train the model for a sequence of tasks would be retraining the entire model every time a new task arrives. However, such retraining will be very costly for a deep neural network. Thus, we suggest to perform selective retraining of the model, by retraining only the weights that are affected by the new task. Initially(t=1), we train the network with 1 -regularization to promote sparsity in the weights, such that each neuron is connected to only few neurons in the layer below: where 1≤l≤L denotes the l th layer of the network, W t l is the network parameter at layer l, and µ is the regularization parameter of the element-wise 1 norm for sparsity on W. For convolutional layers, we apply-norm on the filters, to select only few filters from the previous layer. Throughout our incremental learning procedure, we maintain W t−1 to be sparse, thus we can drastically reduce the computation overheads if we can focus on the subnetwork connected new task. To this end, when a new task t arrives at the model, we first fit a sparse linear model to predict task t using topmost hidden units of the neural network via solving the following problem: DISPLAYFORM0 where W t−1 1:L−1 denotes the set of all other parameters except W t L,t. That is, we solve this optimization to obtain the connections between output unit o t and the hidden units at layer L-1 (fixing all other parameters up to layer L-1 as W t−1). Once we build the sparse connection at this layer, we can identify all units and weights in the network that are affected by the training, while leaving the part of the network that are not connected to o t unchanged. Specifically, we perform breadth-first search on the network starting from those selected nodes, to identify all units (and input feature) that have paths to o t. Then, we train only the weights of the selected subnetwork S, denoted as W DISPLAYFORM1 We use the element-wise 2 regularizer since the sparse connections have been already established 1. This partial retraining will in lower computational overhead and also help with avoiding negative transfer, since neurons that are not selected will not get affected by the retraining process. Algorithm 2 describes the selective retraining process. Add neuron i to S if the weight between i and ot in W t L,t is not zero. DISPLAYFORM0 Add neuron i to S if there exists some neuron j ∈ S such that W t−1 l,ij = 0. Solve Eq. 4 to obtain W t S Dynamic Network Expansion. In case where the new task is highly relevant to the old ones, or aggregated partial knowledge obtained from each task is sufficient to explain the new task, selective retraining alone will be sufficient for the new task. However, when the learned features cannot accurately represent the new task, additional neurons need to be introduced to the network, in order to account for the features that are necessary for the new task. Some existing work BID17 BID12 are based on a similar idea. However, they are either inefficient due to iterative training that requires repeated forward pass BID17, or adds in constant number of units at each task t without consideration of the task difficulty BID12 and thus are suboptimal in terms of performance and network capacity utility. To overcome these limitations, we instead propose an efficient way of using group sparse regularization to dynamically decide how many neurons to add at which layer, for each task without repeated retraining of the network for each unit. Suppose that we expand the l th layer of a network with a constant number of units, say k, inducing two parameter matrices expansions: DISPLAYFORM1 ] for outgoing and incoming layers respectively, where W N is the expanded weight matrix involved with added neurons. Since we do not always want to add in all k units (depending on the relatedness between the new task and the old tasks), we perform group sparsity regularization on the added parameters as follows: DISPLAYFORM2 where g ∈ G is a group defined on the incoming weights for each neuron. For convolutional layers, we defined each group as the activation map for each convolutional filter. This group sparsity regularization was used in BID14 and BID1 to find the right number of neurons for a full network, while we apply it to the partial network. Algorithm 3 describes the details on how expansion works. After selective retraining is done, the network checks if the loss is below certain threshold. If not, then at each layer we expand its capacity by k neurons and solve for Eq. 5. Due to group sparsity regularization in Eq. 5, hidden units (or convolutional filters) that are deemed unnecessary from the training will be dropped altogether. We expect that from this dynamic network expansion process, the model captures new features that were not previously represented by W t−1 l to minimize residual errors, while maximizing the utilization of the network capacity by avoiding to add in too many units. Input: Datatset Dt, Threshold τ Perform Algorithm 2 and compute L if L > τ then Add k units h N at all layers Solve for Eq. 5 at all layers for l = L − 1,..., 1 do Remove useless units in h N l Network Split/Duplication. A crucial challenge in lifelong learning is the problem of semantic drift, or catastrophic forgetting, which describes the problem where the model gradually fits to the later learned tasks and thus forgets what it learned for earlier tasks, ing in degenerate performance for them. The most popular yet simple way of preventing semantic drift is to regularize the parameters from deviating too much from its original values using 2 -regularization, as follows: DISPLAYFORM0 where t is the current task, and W t−1 is the weight tensor of the network trained for tasks {1, . . ., t − 1}, and λ is the regularization parameter. This 2 regularization will enforce the solution W t to be found close to W t−1, by the degree given by λ; if λ is small, then the network will be learned to reflect the new task more while forgetting about the old tasks, and if λ is high, then W t will try to preserve the knowledge learned at previous tasks as much as possible. Rather than placing simple 2 regularization, it is also possible to weight each element with Fisher information BID4. Nonetheless, if number of tasks is large, or if the later tasks are semantically disparate from the previous tasks, it may become difficult to find a good solution for both previous and new tasks. A better solution in such a case, is to split the neuron such that we have features that are optimal for two different tasks. After performing Eq. 6, we measure the amount of semantic drift for each hidden unit i, ρ t i, as the 2 -distance between the incoming weights at t-1 and at t. Then if ρ t i > σ, we consider that the meaning of the feature have significantly changed during training, and split this neuron i into two copies (properly introducing new edges from and to duplicate). This operation can be performed for all hidden units in parallel. After this duplication of the neurons, the network needs to train the weights again by solving Eq. 6 since split changes the overall structure. However, in practice this secondary training usually converges fast due to the reasonable parameter initialization from the initial training. Algorithm 4 describes the algorithm for split operation. Input: Weight W t−1, Threshold σ Perform Eq. 6 to obtain W t for all hidden unit i do ρ DISPLAYFORM0 Copy i into i (w introduction of edges for i) Perform Eq. 6 with the initialization of W t to obtain W t Timestamped Inference. In both the network expansion and network split procedures, we timestamp each newly added unit j by setting {z} j = t to record the training stage t when it is added to the network, to further prevent semantic drift caused by the introduction of new hidden units. At inference time, each task will only use the parameters that were introduced up to stage t, to prevent the old tasks from using new hidden units added in the training process. This is a more flexible strategy than fixing the weights learned up to each learning stage as in BID12, since early tasks can still benefit from the learning at later tasks, via units that are further trained, but not split. Baselines and our model. 1) DNN-STL. Base deep neural network, either feedforward or convolutional, trained for each task separately.2) DNN-MTL. Base DNN trained for all tasks at once.3) DNN. Base DNN. All incremental models use 2 -regularizations.3) DNN-L2. Base DNN, where at each task t, W t is initialized as W t−1 and continuously trained with 2 -regularization between W t and W t−1. Deep network trained with elastic weight consolidation BID4 ) for regularization. 5) DNN-Progressive. Our implementation of the progressive network BID12, whose network weights for each task remain fixed for the later arrived tasks. Base network settings. 1) Feedforward networks: We use a two-layer network with 312-128 neurons with ReLU activations. 2) Convolutional networks. For experiments on the CIFAR-100 dataset, we use a modified version of AlexNet BID6 ) that has five convolutional layers (64-128-256-256-128 depth with 5 × 5 filter size), and three fully-connected layers (384-192-100 neurons at each layer).All models and algorithms are implemented using the Tensorflow BID0 library. We will release our codes upon acceptance of our paper, for reproduction of the . Datasets. 1) MNIST-Variation. This dataset consists of 62, 000 images of handwritten digits from 0 to 9. Unlike MNIST, the handwritten digits are rotated to arbitrary angles and has noise in the , which makes the prediction task more challenging. We use 1, 000/200/5, 000 images for train/val/test split for each class. We form each task to be one-versus-rest binary classification.2) CIFAR-100. This dataset consists of 60, 000 images of 100 generic object classes BID5. Each class has 500 images for training and 100 images for test. We used a CNN as the base network for the experiments on this dataset, to show that our method is applicable to a CNN. Further, we considered each task as a set of 10 subtasks, each of which is a binary classification task on each class.3) AWA (Animals with Attributes). This dataset consists of 30, 475 images of 50 animals BID8. For features, we use DECAF features provided with the dataset, whose dimensionality is reduced to 500 by PCA. We use random splits of 30/30/30 images for training/validation/test. We validate our models for both prediction accuracy and efficiency, where we measure the efficiency by network size at the end of training and training time. We first report the average per-task performance of baselines and our models in the top row of Figure 3. DNN-STL showed best performances on AWA and CIFAR-100 dataset; they are expected to perform well, since they are trained to be optimal for each task, while all other models are trained online which might cause semantic drift. When the number of tasks is small, MTL works the best from knowledge sharing via multi-task learning, but when the number of tasks is large, STL works better since it has larger learning capacity than MTL. Our model, DEN, performs almost the same as these batch models, and even outperforms them on MNIST-Variation dataset. Retraining models combined with regularization, such as L2 and EWC do not perform well, although the latter outperforms the former. This is expected as the two models cannot dynamically increase their capacity. Progressive network works better than the two, but it underperforms DEN on all datasets. The performance gap is most significant on AWA, as larger number of tasks (T = 50) may have made it more difficult to find the appropriate network capacity. If the network is too small, then it will not have sufficient learning capacity to represent new tasks, and if too large, it will become prone to overfitting. We further report the performance of each model over network capacity measured relative to MTL on each dataset, in Figure 3 (bottom row). For baselines, we report the performance of multiple models with different network capacity. DEN obtains much better performance with substantially fewer number of parameters than Progressive network or obtain significantly better performance using similar number of parameters. DEN also obtains the same level of performance as STL using only 18.0%, 60.3%, and 11.9% of its capacity on MNIST-Variation, CIFAR-100, and AWA respectively. This also shows the main advantage of DEN, which is being able to dynamically find its optimal capacity, since it learns a very compact model on MNIST-Variation, whilst learning a substantially large network on CIFAR-100. Further fine-tuning of DEN on all tasks (DEN-Finetune) obtains the best performing model on all datasets, which shows that DEN is not only useful for lifelong learning, but can be also used for network capacity estimation when all tasks are available in the first place. Effect of selective retraining. We further examine how efficient and effective the selective training is, by measuring the training speed and the area under ROC curve on MNIST-Variation dataset. To Figure 5: Semantic drift experiment on the MNIST-Variation dataset. We report the AUROC of different models on t = 1, t = 4, and t = 7 at each training stage to see how the model performance changes over time for these tasks. Reported AUROC is the average over five random splits.this end, we compare the model without network expansion, which we refer to as DNN-Selective, against retraining on DNN-L2 and DNN-L1 (Eq. FORMULA4), for both the accuracy and efficiency. Figure 4 (a) shows both the accuracy over training time measured as actual time spent with GPU computation, for each model. We observe that selective retraining takes significantly less time than the full retraining of the network, and even less than DNN-L1 that comes with sparse network weights. Further, whereas DNN-L1 obtained slightly less accuracy than DNN-L2, DNN-Selective improves the accuracy over the base network by 2%p. This accuracy gain may be due to the suppression of catastrophic forgetting, as DEN trains only a partial subnetwork for each newly introduced task. Figure 4 (b) shows the number of selected neurons at each layer with selective retraining. Note that DNN-selective mostly selects less portion of upper level units which are more task-specific, while selecting larger portion of more generic lower layer units. Effect of network expansion. We also compare the effectiveness of the network expansion against the model with a variant of our model that does selective retraining and layer expansion, but without network split. We refer to this model as DNN-Dynamic. We compare DNN-Dynamic with DNN-L2 used in the main experiment, and DNN-Constant, which is a version of our model that expands its capacity at each layer with fixed number of units, on MNIST-Variation dataset. Figure 4 (c) shows the experimetal . DNN-Dynamic obtains the best mean AU-ROC, significantly outperforming all models including DNN-Constant, while increasing the size of the network substantially less than DNN-Constant (k=20). This may be because having less number of parameters is not only beneficial in terms of training efficiency, but also advantageous in preventing the model from overfitting. We can set the network capacity of DNN-Constant to be similar (k=13) to obtain better accuracy, but it still underperforms DEN which can dynamically adjust the number of neurons at each layer. Effect of network split/duplication and timestamped inference. To see how network split/duplication and unit timestamping help prevent semantic drift (or catastrophic forgetting), while allowing to obtain good performances on later tasks, we compare the performance of our model against baselines and also a variant of our DEN without timestamped inference (DEN-No-Stamp) at different learning stages. Each figure in Figure 5 (a), (b), and (c) shows how the performance of the model changes at each training stage t, for tasks t=1, t=4, and t=7.We observe that DNN-L2 prevents semantic drift of the models learned at early stages, but in increasingly worse performance on later tasks (t=4, 7). DNN-EWC, on the other hand, has better performance on later tasks than DNN-L2, as reported in BID4. However, it shows significantly lower performance than both DNN-Progressive and our model, which may be due to its inability to increase network capacity, that may in limited expressive power. DNN-Progressive shows no semantic drift on old tasks, which is expected because it does not retrain parameters for them. DEN w/o Timestamping works better than DNN-Progressive on later tasks, with slight performance degeneration over time. Finally, our full model with timestamped inference, DEN, shows no sign of noticeable performance degeneration at any learning stage, while significantly outperforming DNN-Progressive. This show that DEN is highly effective in preventing semantic drift as well. We proposed a novel deep neural network for lifelong learning, Dynamically Expandable Network (DEN). DEN performs partial retraining of the network trained on old tasks by exploiting task relatedness, while increasing its capacity when necessary to account for new knowledge required to account for new tasks, to find the optimal capacity for itself, while also effectively preventing semantic drift. We implement both feedforward and convolutional neural network version of our DEN, and validate them on multiple classification datasets under lifelong learning scenarios, on which they significantly outperform the existing lifelong learning methods, achieving almost the same performance as the network trained in batch while using as little as 11.9%p − 60.3%p of its capacity. Further fine-tuning of the models on all tasks in obtaining models that outperform the batch models, which shows that DEN is useful for network structure estimation as well. We provide additional experimental on Permuted MNIST dataset BID9. This dataset consists of 70, 000 images of handwritten digits from 0 to 9, where 60, 000 images are used for training, and 10, 000 images for test. The difference of this dataset from the original MNIST is that each of the ten tasks is the multi-class classification of a different random permutation of the input pixels. FIG4 shows the of this experiment. Our DEN outperforms all lifelong learning baselines while using only 1.39 times of base network capacity. Further, DEN-Finetune achieves the best AUROC among all models, including DNN-STL and DNN-MTL.

Represent the short summary for finding the corresponding scientific article (long)

We propose a novel deep network architecture that can dynamically decide its network capacity as it trains on a lifelong learning scenario.

Represent the TLDR for finding the corresponding scientific article.

We find that deep networks which generalize poorly are more reliant on single directions than those that generalize well, and evaluate the impact of dropout and batch normalization, as well as class selectivity on single direction reliance.


Question: Represent the article for finding a single-sentence TLDR

Question: This paper fosters the idea that deep learning methods can be sided to classical visual odometry pipelines to improve their accuracy and to produce uncertainty models to their estimations. We show that the biases inherent to the visual odom- etry process can be faithfully learnt and compensated for, and that a learning ar- chitecture associated to a probabilistic loss function can jointly estimate a full covariance matrix of the residual errors, defining a heteroscedastic error model. Experiments on autonomous driving image sequences and micro aerial vehicles camera acquisitions assess the possibility to concurrently improve visual odome- try and estimate an error associated to its outputs.

Represent the short summary for finding the corresponding scientific article (long):

This paper discusses different methods of pairing VO with deep learning and proposes a simultaneous prediction of corrections and uncertainty.

Represent this text to find a much much longer article about it with say hundreds-thousands of words:

We introduce an augmented robust feature space for streaming wifi data that is capable of tackling concept drift for indoor localization


Question: Represent the article for finding a single-sentence TLDR

Question: Building robust online content recommendation systems requires learning com- plex interactions between user preferences and content features. The field has evolved rapidly in recent years from traditional multi-arm bandit and collabora- tive filtering techniques, with new methods integrating Deep Learning models that enable to capture non-linear feature interactions. Despite progress, the dynamic nature of online recommendations still poses great challenges, such as finding the delicate balance between exploration and exploitation. In this paper we provide a novel method, Deep Density Networks (DDN) which deconvolves measurement and data uncertainty and predicts probability densities of CTR, enabling us to perform more efficient exploration of the feature space. We show the usefulness of using DDN online in a real world content recommendation system that serves billions of recommendations per day, and present online and offline to eval- uate the benefit of using DDN. In order to navigate the vast amounts of content on the internet, users either rely on active search queries, or on passive content recommendations. As the amount of the content on the internet grows, content discovery becomes an increasingly crucial challenge, shaping the way content is consumed by users. Taboola's content discovery platform aims to perform "reverse search", using computational models to match content to users who are likely to engage with it. Taboola's content recommendations are shown in widgets that are usually placed at the bottom of articles (see FIG0) in various websites across the internet, and serve billions of recommendation per day, with a user base of hundreds of millions of active users. Traditionally recommender systems have been modeled in a multi-arm bandit setting, in which the goal is to a find a strategy that balances exploitation and exploration in order to maximize the long term reward. Exploitation regimes try to maximize the immediate reward given the available information, while exploration seeks to extract new information from the feature space, subsequently increasing the performance of the exploitation module. One of the simplest approaches to deal with multi-arm bandit problems is the -greedy algorithm, in which with probability a random recommendation is chosen, and with probability 1 − the recommendation with the highest predicted reward is chosen. Upper Confidence Bound -UCB- BID0 ) and Thompson sampling techniques use prediction uncertainty estimations in order to perform more efficient exploration of the feature space, either by explicitly adding the uncertainty to the estimation (UCB) or by sampling from the posterior distribution (Thompson sampling). Estimating prediction uncertainty is crucial in order to utilize these methods. Online recommendations are noisy and probabilistic by nature, with measured values being only a proxy to the true underlying distribution, leading to additional interesting challenges when predicting uncertainty estimations. In this paper we present DDN, a unified deep neural network model which incorporates both measurement and data uncertainty, having the ability to be trained end-to-end while facilitating the exploitation/exploration selection strategy. We introduce a mathematical formulation to deconvolve measurement noise, and to provide data uncertainty predictions that can be utilized to improve exploration methods. Finally, we demonstrate the benefit of using DDN in a real world content recommendation system. Over the past decade deep learning has been applied with tremendous success in many different application domains such as computer vision, speech recognition and machine translation. In recent years we have seen a corresponding explosion of deep learning models in the recommender systems landscape, revolutionizing recommendation architectures and providing superior performance over traditional models BID17 BID3; BID4; BID14 ). Deep learning's ability to capture non-linearities has enabled to model complex user-item relations and to integrate higher level representations of data sources such as contextual, textual and visual input. Traditionally recommender systems have been modeled in a multi-arm bandit setting, where the goal is to find an exploitation/exploration selection strategy in order to maximize the long term reward. A similar challenge has been faced in Reinforcement learning (RL) setting, in which an agent has to decide when to forego an immediate reward and to explore its environment. Bayesian neural networks using distributions over the weights were applied by using either sampling or stochastic variational inference BID9; BID15 ). While Bayesian models offer a mathematically grounded framework, they usually entail a prohibitive computational cost. BID2 proposed Bayes by Backprop algorithm for the variational posterior estimation and applied Thompson sampling. BID6 proposed Monte Carlo (MC) dropout, a Bayesian approximation of model uncertainty by extracting estimations from the different sub-models that have been trained using dropout. BID8 separated uncertainty into two types, model and data uncertainty, while studying the effect of each uncertainty separately in computer vision tasks. BID10 formulated the exploration/exploitation trade-off in personalized article recommendation as a contextual bandit problem and proposed LinUCB algorithm, which adapts the UCB strategy to support models based on contextual features. The effect of measurement noise and noisy labels has been studied extensively BID5 ). BID12 proposed a probabilistic model for the conditional probability of seeing a wrong label, where the correct label is a latent variable of the model. explicitly modelled noise by an additional softmax layer that connects the correct labels to the noisy ones. In this paper we model measurement noise using a Gaussian model and combine it with a MDN.3 TABOOLA'S RECOMMENDER SYSTEM OVERVIEW Taboola's revenue stream is facilitated by online advertisers, who pay a fixed amount CPC (Cost Per Click) for each user that is redirected to their site after clicking on a Taboola's recommendation. The algorithm's total value is measured in RPM (Revenue Per Mille) where RP M = CT R * CP C * 1000, is the average revenue for every 1000 recommendations and CTR (Click Through Rate) is the probability of a recommendation being clicked. Content recommendations are ranked according to their predicted RPM; recommendations with the highest predicted RPM will be shown to the user. Taboola's main algorithmic challenge is to provide an estimate of the CTR in any given context. Taboola's recommendation engine needs to provide recommendations within strict time constraints (< 50ms). It is infeasable to rank millions of recommendations in that time frame; in order to support this we have partitioned the system into a two-step process, candidation and ranking (see FIG1). During the candidation phase, we narrow down the list of possible recommendations to thousands based on their RPM prediction in a specific context. CTR prediction in this setting is based on features such as the creative of recommendations (text and image) and empirical click statistics. This relatively small list of recommendations is written to distributed databases in worldwide data centers, and are re-calculated by Taboola's servers continuously throughout the day. When the frontend servers get a request for recommendations from the browser, they retrieve the relevant ready-made recommendation list, and perform an additional ranking of the recommendations based on additional user features using a DNN, further personalizing recommendations. This system architecture shows similarities to BID3 ).The dynamic nature of Taboola's marketplace means that our algorithm constantly needs to evaluate new recommendations, with tens of thousands of new possible recommendations every day. To support this, we split the algorithm into exploration and exploitation modules. The exploitation module aims to choose the recommendations that maximize the RPM, while the exploration module aims to enrich the dataset available for exploitation models by showing new recommendations. In this paper we focus on the candidation phase and the corresponding CTR prediction task, leaving out of this scope the second ranking step. In this section we present Deep Density Network (DDN) and describe its ability to deconvolve measurement noise and integrate it with data uncertainty in a unified model. Employing uncertainty during the training phase can be interpreted as loss attenuation, making our model more robust to noisy data. In addition, accurate uncertainty estimations enable us to employ more efficient exploitation/exploration selection strategies as discussed below. Our deep recommender model is a hybrid of a content-based and a collaborative filtering (CF) recommendation system. A high level overview is depicted in FIG2. We use two separate subnets which model the target and the context features. The target subnet receives as input the content features seen by the user and additional features such as the recommendation age which are unseen to the user. The categorical features are passed through an embedding layer and concatenated along with the numerical features, followed by fully connected layers with a RELU activation function. The is the target feature descriptor. Similarly, the context features are modeled using a separate DNN, taking as input context features such as device type where the target will be recommended, ing in the context feature descriptor. The target and context feature descriptors are then fused using a DNN which outputs both the CTR and its uncertainty prediction, with the measurement being compensated as described in sec. 4.2.In order to train our models, we collect and use historical data which consists of target and context pairs (t, c), where t is the target we recommended in a specific browsing context c. Each row in our training dataset includes the empirical CTR of the target-context pair (t, c), together with additional target and context features. We train models that optimize CTR prediction on this dataset using Maximum Likelihood Estimation (MLE) as discussed below. We separate uncertainty into two different types: measurement and data uncertainty. Measurement uncertainty corresponds to the uncertainty of our observation due to the measurement noise introduced by the binomial recommendation experiment. This type of uncertainty depends on the number of times r, a specific target x = (t, c) pair was recommended, i.e. target t was recommended in context c. Data uncertainty corresponds to the inherent noise of the observations; In contrast to measurement noise, it cannot be reduced even if more data was to be collected, as it corresponds to the inherent variability of the data. Data uncertainty is categorized into homoscedastic and heteroscedastic uncertainty. Homoscedastic is constant over all different inputs, in contrast to heteroscedastic which depends on the inputs, i.e. different input values may have more noisy outputs than others. Simple algorithms like -greedy choose actions indiscriminately during exploration, with no specific preference for targets that have higher probability to be successful in exploitation, or for targets that hold significant information gain about the feature space, for example targets that contain words that weren't previously recommended. It is beneficial to select among the non-greedy actions according to their potential of actually being optimal, taking into account both the expectation and the variance of their CTR estimates. Estimating uncertainty enables us to employ the upper confidence bound (UCB) algorithm for a better and adaptive selection strategy between exploitation/exploration. We estimate both the mean payoff µ t and the standard deviation σ t of each target t and select the target that achieves the highest UCB score, where a is a tunable parameter: DISPLAYFORM0 Our marketplace is defined by a very high recommendation turnover rate, with new content being uploaded everyday and old one becoming obsolete. Probabilistic modeling of the data uncertainty assists us in using the exploration model in order to sample targets that have the highest potential value, by employing the UCB strategy. In contrast to the variance captured by data uncertainty, model uncertaintiy corresponds to what the model "knows" about the feature space. BID6 show that model uncertainty can capture the confidence about different values in the feature space. This however comes at a prohibitive computational cost when calculated using dropout. We explore the feature space by setting to Out Of Vocabulary (OOV) categorical feature values which have been shown less than a minimal threshold. As shown in FIG3, OOV values indeed get larger uncertainty estimations. In order to deconvolve the data and measurement uncertainties we explicitly model them together. Let Y, Y * and be three random variables given x = (t, c). Y corresponds to observed CTR, after recommending (t, c) pair, r times. Y * corresponds to the true/clean CTR without the measurement noise, i.e. the observed CTR had we recommended t infinite times in c. corresponds to the binomial noise error distribution. DISPLAYFORM0 We are modelling data uncertainty by placing a distribution over the output of the model and learning it as a function of the different inputs. To this end, we are using Mixture Density Network (MDN), which employ a Gaussian Mixture Model (GMM) to model Y * . DISPLAYFORM1 For every input the MDN model predicts the coefficients of the GMM; These are the mixing coefficients, α i, µ i and σ i, from which we estimate the expected value and the standard deviation of Y *.The measurement uncertainty corresponds to the measurement noise distribution which we approximate with a Gaussian distribution: DISPLAYFORM2 Due to the fact that data noise is small given x, we enforce constant σ = f (µ, r) for every y * |x where µ is the expected value of y * |x. In this way, Y * |= given x, as σ depends only on r and µ. We can rewrite eq. 2 using eq. 3 and 6 to: DISPLAYFORM3 This enables us to deconvolve and model both data and measurement uncertainties, using a single model which combines MDN and a Gaussian model. Given this probability estimation, the training process uses SGD for minimizing the loss: DISPLAYFORM4 Data: For the purpose of this paper we use the browsed website (i.e. publisher) as the user context. In all of the experiments we used three months of historical data, containing ∼10M records of target-publisher pairs. The dataset contains ∼ 1M unique targets and ∼10K unique publishers. Every experiment has been run on multiple time slots to validate that the were statistically significant. We have experimented with the following models:1. REG is a regression model that outputs a point estimate of the CTR where the loss is the MSE between the actual and the predicted CTR.2. MDN is a model that estimates the distribution over the CTR utilizing a mixture of Gaussians (see sec. 4.2.1).3. DDN is a model that estimates the distribution over the CTR combining the data and measurement uncertainties (see sec. 4.2.1).In order to have a fair comparison, we tuned the hyper-parameters (e.g. embedding sizes, number of layers, number of mixtures) for each model separately; we performed thousands of iterations of random search, and chose the parameters that yielded the best . We evaluate our models using Mean Square Error (MSE). Due to the dynamic nature of online recommendations it is crucial that we evaluate our models online within an A/B testing framework, by measuring the average RPM of models across different publishers. In addition we utilize an online throughput metric which aims to capture the effectiveness of the exploration mechanism. Prior works have put an effort on how to measure exploration; BID11 built an offline simulator that enabled them to test different models to see which one achieves target coverage faster. This is not feasible in our case given the large turnover rate in the recommendation pool. Instead, we use the following targets throughput metric. Let < t i, p i > be the set of target-publisher pairs that accumulated enough data to achieve empiric binomial statistical significance in a given day. A model is said to be contributing to < t i, p i > if it has recommended t i in p i in the previous day more times than a predefined threshold. Our throughput metric is defined by the number of targets that a specific model contributed to this set. 6.5% 9.1% 11.7% Table 3: RPM lift vs targets throughput as a function of different values of c. Feature importance: Understanding the parameters of deep learning networks poses a significant challenge compared to linear and tree based models. We utilize the fact that our models output a full distribution rather than a point estimate to evaluate feature importance. In our analysis, we evaluate the effect on the σ prediction when a feature is "hidden" from the model during inference, by setting it to OOV. For each feature, we calculate statistics over the ratio σ oov /σ, between the predicted σ before and after setting it to OOV.In FIG3 we observe that the analyzed features have a large impact on data uncertainty. The median values of the various features are greater than one, validating our assumption that feature values that did not appear in the training data will obtain a higher uncertainty estimation. In addition, we see a distinct ordering of feature importance, where new advertisers yield a larger ratio than new targets. Using σ in a UCB setting (as in equation 1) will prioritize new targets, especially ones from new advertisers -a desired behaviour both in terms of information gain and advertiser satisfaction. In TAB0 we compare the MDN and DDN models by training them on two different datasets, D1 and D2. D1 differs from D2 by the amount of noise in the training samples; D1 contains noisy data points with relatively small amount of empirical data, while D2 contains examples with higher empirical statistical significance. We observe that DDN improves on MDN performance by 2.7% when using D1 for training, and by 5.3% when using D2. This validates that integrating measurement noise into our modeling is crucial when the training data contains very noisy samples, by attenuating the impact of measurement noise on the loss function. (see sec. 4.2.1) TAB2 we compare the three different models discussed previously in terms of online RPM. We observe that DDN is the best performing model, outperforming MDN and REG by 1.7% and 2.9% respectively. These verify once again that the loss attenuation achieved by DDN during training has enabled it to converge to better parameters, generalizing better to unseen examples. DISPLAYFORM0 We analyzed the effect of the parameter a found in 1. From a theoretical standpoint, increasing this value is supposed to prioritize higher information gain at the expense of RPM, by choosing targets that the model is uncertain about. This trade-off is worthwhile in the long term. In Table 3 we observe that there is an inverse correlation between RPM and throughput which is triggered by different values of a, with targets throughput increasing by 11.7% when setting a = 1.5. Choosing the right trade-off is an application specific concern, and we chose the trade-off induced by a = 0.5, ing in a good throughput gain with a small RPM cost. We have introduced Deep Density Network (DDN), a unified DNN model that is able to predict probability distributions and to deconvolve measurement and data uncertainties. DDN is able to model non-linearities and capture complex target-context relations, incorporating higher level representations of data sources such as contextual and textual input. We have shown the added value of using DNN in a multi-arm bandit setting, yielding an adaptive selection strategy that balances exploitation and exploration and maximizes the long term reward. We presented validating DDN's improved noise handling capabilities, leading to 5.3% improvement on a noisy dataset. Furthermore, we observed that DDN outperformed both REG and MDN models in online experiments, leading to RPM improvements of 2.9% and 1.7% respectively. Finally, by employing DDN's data uncertainty estimation and UCB strategy, we improved our exploration strategy, depicting 6.5% increase of targets throughput with only 0.05% RPM decrease.

Represent the TLDR for finding the corresponding scientific article

We have introduced Deep Density Network, a unified DNN model to estimate uncertainty for exploration/exploitation in recommendation systems.

Represent the short summary for finding the corresponding scientific article (long)

This work aims to provide quantitative answers to the relative importance of concepts of interest via concept activation vectors (CAV). In particular, this framework enables non-machine learning experts to express concepts of interest and test hypotheses using examples (e.g., a set of pictures that illustrate  the concept). We show that CAV can be learned given a relatively small set of examples. Hypothesis testing with CAV can answer whether a particular concept (e.g., gender) is more important in predicting a given class (e.g., doctor) than other sets of concepts. Interpreting networks with CAV does not require any retraining or modification of the network.


Question: represent the article for finding a single-sentence tldr

Question: While it is well-documented that climate change accepters and deniers have become increasingly polarized in the United States over time, there has been no large-scale examination of whether these individuals are prone to changing their opinions as a of natural external occurrences. On the sub-population of Twitter users, we examine whether climate change sentiment changes in response to five separate natural disasters occurring in the U.S. in 2018. We begin by showing that tweets can be classified with over 75% accuracy as either accepting or denying climate change when using our methodology to compensate for limited labelled data; are robust across several machine learning models and yield geographic-level in line with prior research. We then apply RNNs to conduct a cohort-level analysis showing that the 2018 hurricanes yielded a statistically significant increase in average tweet sentiment affirming climate change. However, this effect does not hold for the 2018 blizzard and wildfires studied, implying that Twitter users' opinions on climate change are fairly ingrained on this subset of natural disasters. Much prior work has been done at the intersection of climate change and Twitter, such as tracking climate change sentiment over time, finding correlations between Twitter climate change sentiment and seasonal effects BID1, and clustering Twitter users based on climate mentalities using network analysis BID8. Throughout, Twitter has been accepted as a powerful tool given the magnitude and reach of samples unattainable from standard surveys. However, the aforementioned studies are not scalable with regards to training data, do not use more recent sentiment analysis tools (such as neural nets), and do not consider unbiased comparisons pre-and post-various climate events (which would allow for a more concrete evaluation of shocks to climate change sentiment). This paper aims to address these three concerns as follows. First, we show that machine learning models formed using our labeling technique can accurately predict tweet sentiment (see Section 3.1). We introduce a novel method to intuit binary sentiments of large numbers of tweets for training purposes. Second, we quantify unbiased outcomes from these predicted sentiments (see Section 3.2). We do this by comparing sentiments within the same cohort of Twitter users tweeting both before and after specific natural disasters; this removes bias from over-weighting Twitter users who are only compelled to compose tweets after a disaster. We henceforth refer to a tweet affirming climate change as a "positive" sample (labeled as 1 in the data), and a tweet denying climate change as a "negative" sample (labeled as -1 in the data). All data were downloaded from Twitter in two separate batches using the "twint" scraping tool BID9 to sample historical tweets for several different search terms; queries always included either "climate change" or "global warming", and further included disaster-specific search terms (e.g., "bomb cyclone," "blizzard," "snowstorm," etc.). We refer to the first data batch as "influential" tweets, and the second data batch as "event-related" tweets. The first data batch consists of tweets relevant to blizzards, hurricanes, and wildfires, under the constraint that they are tweeted by "influential" tweeters, who we define as individuals certain to have a classifiable sentiment regarding the topic at hand. For example, we assume that any tweet composed by Al Gore regarding climate change is a positive sample, whereas any tweet from conspiracy account @ClimateHiJinx is a negative sample. The assumption we make in ensuing methods (confirmed as reasonable in Section 3.1) is that influential tweeters can be used to label tweets in bulk in the absence of manually-labeled tweets. Here, we enforce binary labels for all tweets composed by each of the 133 influential tweeters that we identified on Twitter (87 of whom accept climate change), yielding a total of 16,360 influential tweets. The second data batch consists of event-related tweets for five natural disasters occurring in the U.S. in 2018. These are: the East Coast Bomb Cyclone (Jan. 2 -6); the Mendocino, California wildfires (Jul. 27 -Sept. 18); Hurricane Florence (Aug. 31 -Sept. 19); Hurricane Michael (Oct. 7 -16); and the California Camp Fires. For each disaster, we scraped tweets starting from two weeks prior to the beginning of the event, and continuing through two weeks after the end of the event. Summary statistics on the downloaded event-specific tweets are provided in TAB0. Note that the number of tweets occurring prior to the two 2018 sets of California fires are relatively small. This is because the magnitude of these wildfires were relatively unpredictable, whereas blizzards and hurricanes are often forecast weeks in advance alongside public warnings. The first (influential tweet data) and second (event-related tweet data) batches are de-duplicated to be mutually exclusive. In Section 3.1, we perform geographic analysis on the event-related tweets from which we can scrape self-reported user city from Twitter user profile header cards; overall this includes 840 pre-event and 5,984 post-event tweets. To create a model for predicting sentiments of event-related tweets, we divide the first data batch of influential tweets into training and validation datasets with a 90%/10% split. The training set contains 49.2% positive samples, and the validation set contains 49.0% positive samples. We form our test set by manually labelling a subset of 500 tweets from the the event-related tweets (randomly chosen across all five natural disasters), of which 50.0% are positive samples. TAB1.The RNN pre-trained using GloVe word embeddings BID7 achieved the higest test accuracy. We pass tokenized features into the embedding layer, followed by an LSTM BID3 with dropout and ReLU activation, and a dense layer with sigmoid activation. We apply an Adam optimizer on the binary crossentropy loss. Implementing this simple, one-layer LSTM allows us to surpass the other traditional machine learning classification methods. Note the 13-point spread between validation and test accuracies achieved. Ideally, the training, validation, and test datasets have the same underlying distribution of tweet sentiments; the assumption made with our labelling technique is that the influential accounts chosen are representative of all Twitter accounts. Critically, when choosing the influential Twitter users who believe in climate change, we highlighted primarily politicians or news sources (i.e., verifiably affirming or denying climate change); these tweets rarely make spelling errors or use sarcasm. Due to this skew, the model yields a high rate of false negatives. It is likely that we could lessen the gap between validation and test accuracies by finding more "real" Twitter users who are climate change believers, e.g. by using the methodology found in BID8. Our second goal is to compare the mean values of users' binary sentiments both pre-and post-each natural disaster event. Applying our highest-performing RNN to event-related tweets yields the following breakdown of positive tweets: Bomb Cyclone (34.7%), Mendocino Wildfire (80.4%), Hurricane Florence (57.2%), Hurricane Michael (57.6%), and Camp Fire (70.1%). As sanity checks, we examine the predicted sentiments on a subset with geographic user information and compare to the prior literature. In FIG0, we map 4-clustering on three dimensions: predicted sentiments, latitude, and longitude. The clusters correspond to four major regions of the U.S.: the Northeast (green), Southeast (yellow), Midwest (blue), and West Coast (purple); centroids are designated by crosses. Average sentiments within each cluster confirm prior knowledge BID4: the Southeast and Midwest have lower average sentiments (-0.17 and -0.09, respectively) than the West Coast and Northeast (0.22 and 0.09, respectively). In Figure 2, we plot predicted sentiment averaged by U.S. city of event-related tweeters. The majority of positive tweets emanate from traditionally liberal hubs (e.g. San Francisco, Los Angeles, Austin), while most negative tweets come from the Philadelphia metropolitan area. These regions aside, rural areas tended to see more negative sentiment tweeters post-event, whereas urban regions saw more positive sentiment tweeters; however, overall average climate change sentiment pre-and post-event was relatively stable geographically. This map further confirms findings that coastal cities tend to be more aware of climate change BID6.From these mapping exercises, we claim that our "influential tweet" labelling is reasonable. We now discuss our final method on outcomes: comparing average Twitter sentiment pre-event to post-event. In Figure 3, we display these metrics in two ways: first, as an overall average of tweet binary sentiment, and second, as a within-cohort average of tweet sentiment for the subset of tweets by users who tweeted both before and after the event (hence minimizing awareness bias). We use Student's t-test to calculate the significance of mean sentiment differences pre-and post-event (see Section 4). Note that we perform these mean comparisons on all event-related data, since the low number of geo-tagged samples would produce an underpowered study. In Figure 3, we see that overall sentiment averages rarely show movement post-event: that is, only Hurricane Florence shows a significant difference in average tweet sentiment pre-and post-event at the 1% level, corresponding to a 0.12 point decrease in positive climate change sentiment. However, controlling for the same group of users tells a different story: both Hurricane Florence and Hurricane Michael have significant tweet sentiment average differences pre-and post-event at the 1% level. Within-cohort, Hurricane Florence sees an increase in positive climate change sentiment by 0.21 points, which is contrary to the overall average change (the latter being likely biased since an influx of climate change deniers are likely to tweet about hurricanes only after the event). Hurricane Michael sees an increase in average tweet sentiment of 0.11 points, which reverses the direction of tweets from mostly negative pre-event to mostly positive post-event. Likely due to similar bias reasons, the Mendocino wildfires in California see a 0.06 point decrease in overall sentiment post-event, but a 0.09 point increase in within-cohort sentiment. Methodologically, we assert that overall averages are not robust to use in sentiment analyses. We now comment on the two events yielding similar between overall and within-cohort comparisons. Most tweets regarding the Bomb Cyclone have negative sentiment, though sentiment increases by 0.02 and 0.04 points post-event for overall and within-cohort averages, respectively. Meanwhile, the California Camp Fires yield a 0.11 and 0.27 point sentiment decline in overall and within-cohort averages, respectively. This large difference in sentiment change can be attributed to two factors: first, the number of tweets made regarding wildfires prior to the (usually unexpected) event is quite low, so within-cohort users tend to have more polarized climate change beliefs. Second, the root cause of the Camp Fires was quickly linked to PG&E, bolstering claims that climate change had nothing to do with the rapid spread of fire; hence within-cohort users were less vocally positive regarding climate change post-event. There are several caveats in our work: first, tweet sentiment is rarely binary (this work could be extended to a multinomial or continuous model). Second, our are constrained to Twitter users, who are known to be more negative than the general U.S. population BID2. Third, we do not take into account the aggregate effects of continued natural disasters over time. Going forward, there is clear demand in discovering whether social networks can indicate environmental metrics in a "nowcasting" fashion. As climate change becomes more extreme, it remains to be seen what degree of predictive power exists in our current model regarding climate change sentiments with regards to natural disasters.

represent the tldr for finding the corresponding scientific article

We train RNNs on famous Twitter users to determine whether the general Twitter population is more likely to believe in climate change after a natural disaster.


Question: Represent the scientific article to find a short TLDR!

Question: We study the control of symmetric linear dynamical systems with unknown dynamics and a hidden state. Using a recent spectral filtering technique for concisely representing such systems in a linear basis, we formulate optimal control in this setting as a convex program. This approach eliminates the need to solve the non-convex problem of explicit identification of the system and its latent state, and allows for provable optimality guarantees for the control signal. We give the first efficient algorithm for finding the optimal control signal with an arbitrary time horizon T, with sample complexity (number of training rollouts) polynomial only in log(T) and other relevant parameters. Recent empirical successes of reinforcement learning involve using deep nets to represent the underlying MDP and policy. However, we lack any supporting theory, and are far from developing algorithms with provable guarantees for such settings. We can make progress by addressing simpler setups, such as those provided by control theory. Control theory concerns the control of dynamical systems, a non-trivial task even if the system is fully specified and provable guarantees are not required. This is true even in the simplest setting of a linear dynamical system (LDS) with quadratic costs, since the ing optimization problems are high-dimensional and sensitive to noise. The task of controlling an unknown linear system is significantly more complex, often giving rise to non-convex and high-dimensional optimization problems. The standard practice in the literature is to first solve the non-convex problem of system identification-that is, recover a model that accurately describes the system-and then apply standard robust control methods. The non-convex problem of system identification is the main reason that we have essentially no provable algorithms for controlling even the simplest linear dynamical systems with unknown latent states. In this paper, we take the first step towards a provably efficient control algorithm for linear dynamical systems. Despite the highly non-convex and high-dimensional formulation of the problem, we can efficiently find the optimal control signal in polynomial time with optimal sample complexity. Our method is based on wave-filtering, a recent spectral representation technique for symmetric LDSs BID7 ). A dynamical system converts input signals {1, . . .,} ∈ R into output signals {1, . . .,} ∈ R, incurring a sequence of costs 1,..., ∈ R. We are interested in controlling unknown dynamical systems with hidden states (which can be thought of as being partially "observed"via the output signals). A vast body of work focuses on linear dynamical systems with quadratic costs, in which the {} and {} are governed by the following dynamics: DISPLAYFORM0 where ℎ 1,..., ℎ ∈ R is a sequence of hidden states starting with a fixed ℎ 1. Matrices,,,,, of appropriate dimension describe the system and cost objective; the {} are Gaussian noise vectors. All of these matrices, as well as the parameters of the Gaussian, can be unknown. The most fundamental control problem involves controlling the system for some time horizon: find a signal 1,..., that minimizes the sum of these quadratic output costs. Clearly, any algorithm for doing so must first learn the system parameters in some form, and this is often the source of computational intractability (meaning algorithms that take time exponential in the number of system parameters).Previously known algorithms are of two types. The first type tries to solve the non-convex problem, with algorithms that lack provable guarantees and may take exponential time in the worst case: e.g., expectation-maximization (EM) or gradient-based methods (back-propagation through time, like the training of RNNs) which identify both the hidden states and system parameters. Algorithms of the second type rely upon regression, often used in time-series analysis. Since -step dynamics of the system involve the first powers of, the algorithm represents these powers as new variables and learns the -step dynamics via regression (e.g., the so-called VARX(,) model) assuming that the system is well-conditioned; see Appendix A. This has moderate computational complexity but high sample complexity since the number of parameters in the regression scales with, the number of time steps. Our new method obtains the best of both : few parameters to train ing in low sample complexity, as well as polynomial computational complexity. In Section 2 we state the precise algorithm. The informal is as follows. DISPLAYFORM1 Theorem 1.1 (Controlling an unknown LDS; informal). Let D be a linear dynamical system with a symmetric transition matrix and with size |D|. Then, for every > 0, Algorithm 1 produces a sequence of controls (ˆ1, . . .ˆ), ‖ˆ‖ 2 ≤ 1 with ‖ˆ1: ‖ 2 ≤, such that DISPLAYFORM2 Assuming i.i.d. Gaussian noise ∼ (0, Σ), the algorithm samples˜(poly (|D|,, Tr Σ, 1/)) trajectories from D, and runs in time polynomial in the same parameters. The field of optimal control for dynamical systems is extremely broad and brings together literature from machine learning, statistical time-series analysis, dynamical system tracking and Kalman filtering, system identification, and optimal control. For an extensive survey of the field, see e.g. BID12 BID1.Tracking a known system. A less ambitions goal than control is tracking of a dynamical system, or prediction of the output given a known input. For the special case of LDS, the well-known Kalman filter BID9 is an optimal recursive least-squares solution for maximum likelihood estimation (MLE) under Gaussian perturbations to a linear dynamical system. System identification. When the underlying dynamical system is unknown, there are essentially no provably efficient methods for recovering it. For various techniques used in practice, see the classic survey BID10. BID11 suggest using the EM algorithm to learn the parameters of an LDS, nowadays widespread, but it is well-known that optimality is not guaranteed. The recent of BID6 gives a polynomial time algorithm for system recovery, although it applies only to the single-input-single-output case and makes various statistical assumptions on the inputs. Model-free tracking. Our methods depend crucially on a new algorithm for LDS sequence prediction, at the heart of which is a new convex relaxation for the tracking formulation BID7. In particular, this method circumvent the obstacle of explicit system identification. We detail our usage of this in Definition 2.3.We note an intriguing connection to the recently widespread use of deep neural networks to represent an unknown MDP in reinforcement learning: the main algorithm queries the unknown dynamical system with exploration signals, and uses its responses to build a compact representation (denoted byˆin Algorithm 1) which estimates the behavior of the system. Time-series analysis. One of the most common approaches to modeling dynamical systems is the autoregressive-moving average (ARMA) model and its derivatives in the time-series analysis literature BID5 BID2 BID3. At the heart of this method is the autoregressive form of a time series, namely, DISPLAYFORM0 Using online learning techniques, it is possible to completely identify an autoregressive model, even in the presence of adversarial noise BID0. This technique lies at the heart of a folklore regression method for optimal control, given in the second row of table 1.Optimal control. The most relevant and fundamental primitive from control theory, as applied to the control of linear dynamical systems, is the linear-quadratic-Gaussian (LQG) problem. In this setting, the system dynamics are assumed to be known, and the task is to find a sequence of inputs which minimize a given quadratic cost. A common solution, the LQG controller, is to combine Kalman filtering with a linear-quadratic regulator, a controller selected by solving the Bellman equation for the problem. Such an approach admits theoretical guarantees under varied assumptions on the system; see, for example, BID4.Our setting also involves a linear dynamical system with quadratic costs, and thus can be seen as a special case of the LQG setup, in which the process noise is zero, and the transition matrix is assumed to be symmetric. However, our are not analogous: our task also includes learning the system's dynamics. As such, our main algorithm for control takes a very different approach than that of the standard LQR: rather than solving a recursive system of equations, we provide a formulation of control as a one-shot convex program. First, we state the formal definitions of the key objects of interest. Definition 2.1 (Dynamical system). A dynamical system D is a mapping that takes a sequence of input vectors 1,..., ∈ B 2 = {∈ R : ‖ ‖ 2 ≤ 1} to a sequence of output vectors 1,..., ∈ R and costs 1,..., ∈ R. Denote: = [; . . . ;] as the concatenation of all input vectors from time to, and write DISPLAYFORM0 Definition 2.2 (Linear dynamical system). A linear dynamical system (LDS) is a dynamical system whose outputs and costs are defined by DISPLAYFORM1 where ℎ 1,..., ℎ ∈ R is a sequence of hidden states starting with fixed ℎ 1, and,,,,, are matrices (or vectors) of appropriate dimension. We assume ‖ ‖ op ≤ 1, i.e., all singular values of are at most one, and that 0, 0.Our algorithm and its guarantees depend on the construction of a family of orthonormal vectors in R, which are interpreted as convolution filters on the input time series. We define the wave-filtering matrix below; for more details, see Section 3 of BID7. Definition 2.3 (Wave-filtering matrix). Fix any,, and 1 ≤ ≤. Let be the eigenvector corresponding to the -th largest eigenvalue of the Hankel matrix ∈ R ×, with entries:= 2 (+) 3 −(+). The wave-filtering matrix Φ ∈ R × is defined by vertically stacked block matrices {Φ ∈ R × }, defined by horizontally stacked multiples of the identity matrix: DISPLAYFORM2 Then, letting range from 1 to, Φ: − then gives a dimension-wise convolution of the input time series by the filters {} of length. Theorem 3.3 uses a structural from BID7, which guarantees the existence of a concise representation of D in the basis of these filters. The main theorem we prove is the following. Tr(Σ) ln 2 ã, produces a sequence of controls (ˆ1, . . .ˆ) ∈ B 2, such that with probability at least 1 −, DISPLAYFORM3 assuming that DISPLAYFORM4 Further, the algorithm samples poly 1, log 1, log, log, 1,,,, Tr(Σ) trajectories from the dynamical system, and runs in time polynomial in the same parameters. We remark on some of the conditions. is bounded away from 0 when we suffer loss in all directions of. In condition, Tr(Σ) is inevitable loss due to noise, so is an assumption on the system's controllability. We set up some notation for the algorithm. Let Φ ∈ R × be the wave-filtering matrix from Definition 2.3. Let = 0 for ≤ 0, and let =: − +1. Let = max(‖ ‖, ‖ ‖, ‖ ‖, ‖ ‖) be an upper bound for the matrices involved in generating the outputs and costs. To prove Theorem 2.4, we invoke Lemma 3.1 and Lemma 3.2, proved in Subsection 3.1 and Subsection 3.2, respectively. DISPLAYFORM0 Lemma 3.2 (Robustness of control to uncertainty in dynamics). Let 1: = arg min 1: DISPLAYFORM1 whereˆ∈ R × is such that for every sequence of input signals (1, . . .,) ∈ B 2 with ‖ 1: ‖ 2 ≤ and ≤, Equation holds withˆ=D (1:). Assume. Then DISPLAYFORM2 Moreover, the minimization problem posed above is convex (for, 0), and can be solved to within opt acccuracy in poly(,, 1/ opt) time using the ellipsoid method. Proof of Theorem 2.4. Use Lemma 3.1 with ← ». Note that 1: = is a valid input to the LDS because ‖ ‖ 2 = 1. Now use Lemma 3.2 on the of Lemma 3.1. To prove Lemma 3.1, we will use the following structural from BID7 restated to match the setting in consideration. DISPLAYFORM0 Proof. This follows from Theorem 3b in BID7 after noting four things.1. E(−) are the outputs when the system is started at ℎ 1 = 0 with inputs 1: and no noise. A linear relationship − = ′ holds by unfolding the recurrence relation.2. Examining the construction of in the proof of Theorem 3b, the is exactly the projection of ′ onto the subspace defined by Φ. (Note we are using the fact that the rows of Φ are orthogonal.) 3. Theorem 3b is stated in terms of the quantity, which is bounded by 2 by Lemma F.5 in BID7.4. We can modify the system so that = by replacing (,,,) with (, , , ). This replaces by (max(, √)). Theorem 3b originally had dependence of on both Φ and, but if = then the dependence is only on Φ.Proof of Lemma 3.1. DISPLAYFORM1. Letting ′ be the matrix such that − = ′, and = ′ Φ ⊤ as in Theorem 3.3, we have that DISPLAYFORM2 Let 1 ≤ ≤. We bound the error under controls 1: ∈ B 2, ‖ 1: ‖ 2 ≤ using the triangle inequality. Letting 1: be the output under 1:, DISPLAYFORM3 By Theorem 3.3, for = Ω(log 2 log( /)), choosing constants appropriately, the first term is ≤ 4.To bound the second term in, we show concentrates around E. We have DISPLAYFORM4 By concentration of sums of 2 random variables (see BID8, for example), DISPLAYFORM5 Take ′ = 2 and ′ = 4 √ and note was chosen to satisfy. Use the union bound to get that DISPLAYFORM6 To bound the third term in, we first show thatˆconcentrates around. We havê DISPLAYFORM7 By 2 concentration, DISPLAYFORM8 We also have (E −)1 DISPLAYFORM9 With ≥ 1 − probability, we avoid both the bad events in FORMULA2 and FORMULA2 DISPLAYFORM10 and for all ‖ 1: ‖ ≤, the third term of FORMULA2 is bounded by (note ‖Φ‖ op = 1 because it has orthogonal rows) DISPLAYFORM11 Thus by, E[] − −ˆΦ 2 ≤ with probability ≥ 1 −. To prove Lemma 3.2, we need the following helpful lemma. Lemma 3.4. For a symmetric LDS D with = max(‖ ‖, ‖ ‖, ‖ ‖, ‖ ‖ op) and,, and an approximationD whose predictionsˆsatisfy the of Lemma 3.1, we have that for every sequence of controls (1, . . .,) ∈ B 2, and for every 1 ≤ ≤, DISPLAYFORM0 where costD (DISPLAYFORM1 DISPLAYFORM2 using Cauchy-Schwarz. Proof of Lemma 3.2. Define DISPLAYFORM3 1: = arg min 1: DISPLAYFORM4 DISPLAYFORM5 . By assumption, ≤. We have DISPLAYFORM6 By Lemma 3.4 for inputs DISPLAYFORM7 Lettingˆ1: be the outputs underD under the controlˆ1:, note that similar to, DISPLAYFORM8 becauseˆ1: is optimal forD. By Lemma 3.4 for inputsˆ1:, DISPLAYFORM9 Now by,, and, DISPLAYFORM10 We have presented an algorithm for finding the optimal control inputs for an unknown symmetric linear dynamical system, which requires querying the system only a polylogarithmic number of times in the number of such inputs, while running in polynomial time. Deviating significantly from previous approaches, we circumvent the non-convex optimization problem of system identification by a new learned representation of the system. We see this as a first step towards provable, efficient methods for the traditionally non-convex realm of control and reinforcement learning. In this section, we verify the statement made in Section 1 on the time and sample complexity of approximating a linear dynamical system with an autoregressive model. Although this is wellknown (see, for example, Section 6 of BID6), we present a self-contained presentation for convenience and to unify notation. The vector autoregressive model with exogenous variables, or VARX(,), is a touchstone in timeseries analysis. Given a time series of inputs (sometimes known as biases) {}, it generates the time series of responses {} by the following recurrence: DISPLAYFORM0 Here, and are memory parameters, the {} and {} are matrices of appropriate dimension, and the {} are noise vectors. In the special case of = 0, the problem can be solved efficiently with linear regression: in this case, is a linear function of the concatenated inputs [; −1 ; . . ., − +1].A VARX(0,) model is specified by = [,..., (−1) ] ∈ R × and predicts =: − +1. We quantify the relationship between VARX(0,) and linear dynamical systems, with a statement analogous to Theorem 3.1: Theorem A.1. Let D be an LDS with size ℒ, fixed ℎ 1, and noise = 0, producing outputs {1, . . .,} from inputs {1, . . .,}. Suppose that the transition matrix of D has operator norm at most < 1. Then, for each > 0, there is a VARX(0,) model with = (1 1− log(ℒ/)), specified by a matrix, such that DISPLAYFORM1 Proof. By the modification of D given in the proof of Theorem 3.3, we may assume without loss of generality that =. Also, as in the discussion of Theorem 3.1, it can be assumed that the initial hidden state ℎ 1 is zero. Then, we construct the block of corresponding to lag as DISPLAYFORM2 This is well-defined for all 1 ≤ ≤. Note that when ≥, the autoregressive model completely specifies the system D, which is determined by its (infinite-horizon) impulse response function. Furthermore, by definition of, we have DISPLAYFORM3 Noting that DISPLAYFORM4 we conclude that DISPLAYFORM5 implying the claim by the stated choice of.VARX(0,) only serves as a good approximation of an LDS whose hidden state decays on a time scale shorter than; when the system is ill-conditioned (is close to 1), this can get arbitrarily large, requiring the full time horizon =.On the other hand, it is clear that both the time and sample complexity of learning a VARX(0,) model grows linearly in. This verifies the claim in the introduction.

Represent this text to find a much much longer article about it with say hundreds-thousands of words

Using a novel representation of symmetric linear dynamical systems with a latent state, we formulate optimal control as a convex program, giving the first polynomial-time algorithm that solves optimal control with sample complexity only polylogarithmic in the time horizon.

General analysis of sign-based methods (e.g. signSGD) for non-convex optimization, built on intuitive bounds on success probabilities.