Question: Represent the article for finding a single-sentence TLDR

Question: Deep learning yields great across many fields, from speech recognition, image classification, to translation. But for each problem, getting a deep model to work well involves research into the architecture and a long period of tuning. We present a single model that yields good on a number of problems spanning multiple domains. In particular, this single model is trained concurrently on ImageNet, multiple translation tasks, image captioning (COCO dataset), a speech recognition corpus, and an English parsing task. Our model architecture incorporates building blocks from multiple domains. It contains convolutional layers, an attention mechanism, and sparsely-gated layers. Each of these computational blocks is crucial for a subset of the tasks we train on. Interestingly, even if a block is not crucial for a task, we observe that adding it never hurts performance and in most cases improves it on all tasks. We also show that tasks with less data benefit largely from joint training with other tasks, while performance on large tasks degrades only slightly if at all. Recent successes of deep neural networks have spanned many domains, from computer vision BID16 to speech recognition BID7 and many other tasks. Convolutional networks excel at tasks related to vision, while recurrent neural networks have proven successful at natural language processing tasks, e.g., at machine translation BID31 BID2. But in each case, the network was designed and tuned specifically for the problem at hand. This limits the impact of deep learning, as this effort needs to be repeated for each new task. It is also very different from the general nature of the human brain, which is able to learn many different tasks and benefit from transfer learning. The natural question arises:Can we create a unified deep learning model to solve tasks across multiple domains?The question about multi-task models has been studied in many papers in the deep learning literature. Natural language processing models have been shown to benefit from a multi-task approach a long time ago BID5, and recently multi-task machine translation models have even been shown to exhibit zero-shot learning when trained on multiple languages . Speech recognition has also been shown to benefit from multi-task training BID27, as have some vision problems, such as facial landmark detection BID36. But all these models are trained on other tasks from the same domain: translation tasks are trained with other translation tasks, vision tasks with other vision tasks, speech tasks with other speech tasks. Multi-modal learning has been shown to improve learned representations in the unsupervised setting BID22 and when used as a-priori known unrelated tasks BID24. But no competitive multi-task multi-modal model has been proposed, so the above question remains unanswered. In this work, we take a step toward positively answering the above question by introducing the MultiModel architecture, a single deep-learning model that can simultaneously learn multiple tasks from various domains. Concretely, we train the MultiModel simultaneously on the following 8 corpora:Code available at redacted. WSJ speech corpus , used for sentence-level speech recognition. ImageNet dataset BID25, used for image classification. COCO image captioning dataset BID17, used for image captioning. WSJ parsing dataset BID18, used for constituency parsing. These corpora were chosen as they are commonly used for machine learning the respective tasks: speech-to-text, image classification, captioning, parsing and translation. The model learns all of these tasks and achieves good performance: not state-of-the-art at present, but above many task-specific models studied in recent past (see the Section 3 for details). FIG0 illustrates some decodes taken directly from the model: it is clear that it can caption images, categorize them, translate to French and German and construct parse trees. While the MultiModel is only a first step and will be improved in the future, two key insights are crucial to making it work at all and are our main contributions. Small modality-specific sub-networks convert into a unified representation and back from it. To allow training on input data of widely different sizes and dimensions, such as images, sound waves and text, we need sub-networks to convert inputs into a joint representation space. We call these sub-networks modality nets as they are specific to each modality (images, speech, text) and define transformations between these external domains and a unified representation. We design modality nets to be computationally minimal, promoting heavy feature extraction and ensuring that the majority of computation is performed within the domain-agnostic body of the model. Since our model is auto-regressive, modality nets need to both convert the inputs into the unified representation and later convert from this representation into the output space. Two design decisions were important:• The unified representation is variable-size. While a fixed-size representation is tempting and easier to implement, it creates a bottleneck and limits the performance of the model. • Different tasks from the same domain share modality nets. We avoid creating a sub-network for every task, and prefer only to create one for every input modality. For example, all translation tasks share the same modality-net (and vocabulary), no matter for which language pair. This encourages generalization across tasks and allows to add new tasks on the fly. Computational blocks of different kinds are crucial for good on various problems. The body of the MultiModel incorporates building blocks from mutiple domains. We use depthwiseseparable convolutions, an attention mechanism, and sparsely-gated mixture-of-experts layers. These blocks were introduced in papers that belonged to different domains and were not studied before on tasks from other domains. For example, separable convolutions were introduced in the Xception Figure 2: The MultiModel, with modality-nets, an encoder, and an autoregressive decoder.architecture BID4 and were not applied to text or speech processing before. On the other hand, the sparsely-gated mixture-of-experts BID29 had been introduced for language processing tasks and has not been studied on image problems. We find that each of these mechanisms is indeed crucial for the domain it was introduced, e.g., attention is far more important for languagerelated tasks than for image-related ones. But, interestingly, adding these computational blocks never hurts performance, even on tasks they were not designed for. In fact we find that both attention and mixture-of-experts layers slightly improve performance of MultiModel on ImageNet, the task that needs them the least. The MultiModel consists of a few small modality-nets, an encoder, I/O mixer, and an autoregressive decoder, as depicted in Figure 2. As already said above, the encoder and decoder are constructed using 3 key computational blocks to get good performance across different problems: Convolutions allow the model to detect local patterns and generalize across space. Attention layers allow to focus on specific elements to improve performance of the model. Sparsely-gated mixture-of-experts gives the model capacity without excessive computation cost. We start by describing the architecture of each of these 3 blocks and then introduce the encoder, decoder and the architecture of our modality-nets. To perform local computation, we use blocks of convolutions with ReLU non-linearities and normalization. A block of convolutions gets as input a tensor of shape [batch size, sequence length, feature channels] and returns a tensor of the same shape, processed as follows. For convolution operations, we use depthwise separable convolutions, studied for images in BID4, in a way similar to BID12. Depthwise separable convolutions are a parameterand computationally-efficient variant of the traditional convolution. They are defined by a convolution on each feature channel separately, followed by a pointwise convolution to project to the desired feature depth. We refer the reader to BID4 for a complete definition; here we will denote a depthwise separable convolution with weights W h×w corresponding to f kernels of size h × w applied to an input tensor x with stride s and dilated by a factor d (see BID35) as SepConv d,s,f (W, x). Note that subscripts for stride, dilation and output size are omitted when dilation d or stride s are equal to 1, or output size f is equal to the input's feature depth. We use convolutions in blocks that consist of three components: a ReLU activation of the inputs, followed by a SepConv, followed by layer normalization. Layer normalization BID1 acts over the h hidden units of the layer below, computing layer-wise statistics for each batch example and normalizing accordingly. These normalized units are then scaled and shifted by scalar learned parameters G and B respectively, producing the final units to be activated by a non-linearity. The complete convolution step is therefore defined as: DISPLAYFORM0 The convolutional steps are composed into blocks by stacking them and adding residual connections BID9 as depicted in FIG3. We use stacks of four convolutional blocks with two skip-connections between the stack input and the outputs of the second and fourth convolutional steps, and with the first two having 3 × 1 kernels and the next two having 15 × 1 kernels, with the final one dilated by 8 to provide a wide receptive field. We also add 40% dropout at the end of each block, so the complete block is defined as follows: DISPLAYFORM1 For attention, we use a multi-head dot-product attention mechanism inspired by BID2 and similar to , as depicted in FIG3. The inputs to the attention layer are two tensors: a source tensor and a target tensor both with the shape [batch size, sequence length, feature channels] The target tensor is additively composed with a timing signal and mixed using two convolutional blocks. This mixed tensor is then self-attended using a multi-head dot-product attention, which is a dot-product attention with inputs split into g = 8 separate tensors representing each attention head, as shown in FIG3. The timing signals are the main difference between this attention mechanism and the ones used previously. They allow this content-based attention to focus based on their position. They are constructed by concatenating sine and cosine curves: The source tensor is finally passed through two different pointwise convolutions to generate the memory keys K and values V and the query keys, memory keys and memory values are used to apply the attention mechanism between the self-attended target and the source (see FIG3). DISPLAYFORM0 We use sparsely-gated mixture-of-experts layers of the same kind as introduced in BID29: A mixture-of-experts layer consists of a number of simple feed-forward neural networks (experts) and a trainable gating network which selects a sparse combination of the experts to process each input. We refer the reader to BID29 for details as we use exactly the architecture described there. In particular, during training we select k = 4 experts out of the whole expert pool and add the additional load-balancing cost as in BID29. In each of the two mixture-of-experts layers in our model, we use a pool of 240 experts when training on 8 problems jointly, and 60 experts when training on each problem separately. The body of the MultiModel consists of 3 parts: the encoder that only processes the inputs, the mixer that mixes the encoded inputs with previous outputs (autoregressive part), and a decoder that processes the inputs and the mixture to generate new outputs. The encoder, mixer and decoder are structured similarly to previous fully convolutional sequence to sequence models such as ByteNet or WaveNet (van den), but differ in the computational blocks that are used. We depict their architecture in FIG3. As can be seen there, the encoder consists of 6 repeated convolutional blocks (described before) with a mixture-of-experts layer in the middle. The mixer consists of an attention block and 2 convolutional blocks. The decoder consists of 4 blocks of convolutions and attention, with a mixture-of-experts layer in the middle. Crucially, the convolutions in the mixer and decoder are padded on the left, so they can never access any information in the future. This allows the model to be autoregressive, and this convolutional autoregressive generation scheme offers large receptive fields over the inputs and past outputs, which are capable of establishing long term dependencies. To allow the decoder to produce outputs for different tasks even with the same modality, we always start decoding with a command-token, such as To-English or To-Parse-Tree. We learn an embedding vector corresponding to each of the tokens during training. We have 4 modality nets, for language (text data), images, audio, and categorical data. For all predictions, we use the cross-entropy loss, per subword-unit on text, per category on classification. Our language-based data is all tokenized using the same vocabulary with 8k subword-units, following the method from BID28. The language input modality takes a sequence of tokens ending in a termination token. This sequence of tokens is mapped to the correct dimensionality for the body using a learned embedding. On the output side, the language modality takes the decoded output of the body and performs a learned linear mapping, followed by a Sof tmax, ing in a probability distribution over the token vocabulary. DISPLAYFORM0 The image input modality is analogous to the Xception entry flow BID4. The input image's feature depth is gradually deepened using residual convolution blocks which we call ConvRes and define as follows: DISPLAYFORM0 DISPLAYFORM1 The categorical output modality is analogous to the Xception exit flow BID4. If the network inputs are two-dimensional data such as image or spectral audio data, then the one-dimensional output from the model body is first reshaped into two-dimensions again, followed by progressive down-sampling: DISPLAYFORM0 GlobalAvgP ool denotes a mean taken across all spatial and temporal dimensions. We accept audio input in the form of a 1-dimensional waveform over time BID8 BID23 BID26 or as a 2-dimensional spectrogram. Both the waveform and spectral input modalities use a stack of 8 ConvRes blocks from the ImageInputM odality (Section 2.5.2). The i th block has the form: l i = ConvRes(l i−1, 2 i). The spectral modality does not perform any striding along the frequency bin dimension, preserving full resolution in the spectral domain. The modalities of the MultiModel allows to perform a training step on a batch of data from any of the 8 tasks we consider. For example, when making a training step on a batch of translation data, only the language modality sub-network will be activated. Training will then update the parameters of the language modality and all shared parameters, i.e., those in input encoder, mixer and decoder. MultiModel can be trained on a single machine, but we used distributed training for the multi-task runs. When training jointly on 8 tasks, we had a separate worker training on each task, while the shared parameters of the model were on a parameter server and were updated asynchronously. When training on a single task, we used only a single worker training for a similar number of steps. In all training runs report below we used the same set of hyper-parameters and the Adam optimizer BID15 with gradient clipping. We will release the implementation as open-source together with the details of our setup and all used hyper-parameters. The MultiModel architecture draws from eariler encoder-decoder architectures applied to neural machine translation. Earlier sequence-to-sequence models for translation BID31 BID2 BID32 ).memory cells BID10 ). Convolutional architectures yielded good on word-level neural machine translation starting from BID13 and later in BID20. These early models used a standard RNN on top of the convolution to generate the output and had a bottleneck there that hurt performance, especially on longer sentences, similarly to the limitations of RNN sequence-to-sequence models without attention BID31. Fully convolutional neural machine translation without this bottleneck was presented in BID11. The model in BID11 ) (Extended Neural GPU) used a recurrent stack of gated convolutional layers, while the model in ) (ByteNet) did away with recursion and used left-padded convolutions in the decoder. This idea, introduced in WaveNet (van den and also used in MultiModel (see above) significantly improves efficiency. Depthwise separable convolutions were first studied by Sifre BID30 ) and later they were used to get good on large-scale image classification with Xception BID4. We implemented the MultiModel architecture described above using TensorFlow and trained it in a number of configurations. We focused our experiments so as to answer the following questions: How far is the MultiModel trained on 8 tasks simultaneously from state-of-the-art ? How does training on 8 tasks simultaneously compare to training on each task separately? How do the different computational blocks discussed above influence different tasks?In answering the above questions, we don't always consider all 8 problems. Especially the 4 translation problems behave very similarly, so we decided to not include them all in each comparison but we focused on the more varied problems instead. To answer question, we compare the performance of the 8-problem MultiModel with state-of-theart in Table 1. We use the standard top-5 accuracy metric for ImageNet and the standard BLEU metric for translation (scored with MOSES on newstest2014 while newstest2013 was used as the development set). We did not invest much time yet in tuning hyper-parameters of the MultiModel, so we believe that the difference seen there will become much smaller with more tuning. The we achieve are similar to the ones task-specific models get without heavy tuning, e.g., on English-French translation we improve on the recent Extended Neural GPU BID11.To answer question, we compare the MultiModel trained jointly with MultiModel trained separately just on a single task. Since we are comparing different instantiations of the same model, we report two internal metrics: the negative log-perplexity and per-token accuracy (measured on the development set). As can be seen from the in TAB3, the joint 8-problem model performs similarly to single-model on large tasks, and better, sometimes significantly, on tasks where less data is available, such as parsing. The large improvement on parsing seen in TAB3 is not that surprising taking into account the large number of text data in translation tasks. But we were curious if training parsing just with ImageNet, a seemingly unrelated task, would also bring any improvements. This is indeed the case, as can be seen in Table 3. The difference in performance is significant, and since we use both dropout and early stopping, we conjecture that it is not related to over-fitting. Rather, it seems, there are computational primitives shared between different tasks that allow for some transfer learning even between such seemingly unrelated tasks as ImageNet and parsing. Table 3: Results on training parsing alone, with ImageNet, and with 8 other tasks. We report log-perplexity, per-token accuracy, and the percentage of fully correct parse trees. To answer question, we check how training without the mixture-of-experts layers or without the attention mechanism influences performance on different problems. Since both these mechanisms were designed with machine translation in mind, we check the English-French translation. But we also include ImageNet, since this is the problem that stands the least to benefit from those blocks. In fact, one could expect that removing these blocks will improve performance on ImageNet alone if they were truly useless for this task. In contrast, we see in TAB5 that these blocks either don't affect or slightly improve performance. This leads us to conclude that mixing different computation blocks is in fact a good way to improve performance on many various tasks.

represent the short summary for finding the corresponding scientific article (long)

Examples:
Provided: "We show the first successful use of Transformer in generating music that exhibits long-term structure. " Match: "Music relies heavily on repetition to build structure and meaning. Self-reference occurs on multiple timescales, from motifs to phrases to reusing of entire sections of music, such as in pieces with ABA structure. The Transformer , a sequence model based on self-attention, has achieved compelling in many generation tasks that require maintaining long-range coherence. This suggests that self-attention might also be well-suited to modeling music. In musical composition and performance, however, relative timing is critically important. Existing approaches for representing relative positional information in the Transformer modulate attention based on pairwise distance . This is impractical for long sequences such as musical compositions since their memory complexity is quadratic in the sequence length. We propose an algorithm that reduces the intermediate memory requirements to linear in the sequence length. This enables us to demonstrate that a Transformer with our modified relative attention mechanism can generate minute-long (thousands of steps) compositions with compelling structure, generate continuations that coherently elaborate on a given motif, and in a seq2seq setup generate accompaniments conditioned on melodies. We evaluate the Transformer with our relative attention mechanism on two datasets, JSB Chorales and Piano-e-competition, and obtain state-of-the-art on the latter. A musical piece often consists of recurring elements at various levels, from motifs to phrases to sections such as verse-chorus. To generate a coherent piece, a model needs to reference elements that came before, sometimes in the distant past, and then repeat, vary, and further develop them to create contrast and surprise. Intuitively, self-attention could be a good match for this task. Self-attention over its own previous outputs allows an autoregressive model to access any part of the previously generated output at every step of generation. By contrast, recurrent neural networks have to learn to proactively store elements to be referenced in a fixed size state or memory, making training potentially much more difficult. We believe that repeating self-attention in multiple, successive layers of a Transformer decoder BID17 can help capture the multiple levels at which self-referential phenomena exist in music. In its original formulation, the Transformer relies on absolute position representations, using either positional sinusoids or learned position embeddings that are added to the per-position input representations. Recurrent and convolutional neural networks instead model position in relative terms: RNNs through their recurrence over the positions in their input, and CNNs by applying kernels that effectively choose which parameters to apply based on the relative position of the covered input representations. Music has multiple dimensions along which relative differences arguably matter more than their absolute values; the two most prominent are timing and pitch. To capture such pairwise relations between representations, BID13 introduce a relation-aware version of self-attention which they use successfully to modulate self-attention by the distance between two positions. We extend this approach to capture relative timing and optionally also pitch, which yields improvement in both sample quality and perplexity for the JSB Chorales dataset. As opposed to the original Transformer, samples from a Transformer with our relative attention mechanism maintain the regular timing grid present in this dataset. The model furthermore captures global timing, giving rise to regular phrases. The original formulation of relative attention BID13 requires O(L 2 D) memory where L is the sequence length and D is the dimension of the model's hidden state. This is prohibitive for long sequences such as those found in the Maestro dataset of human-performed virtuosic, classical piano music BID7. In Section 3.4, we show how to reduce the memory requirements to O(LD), making it practical to apply relative attention to long sequences. The Maestro dataset consists of MIDI recorded from performances of competition participants, bearing expressive dynamics and timing on a less than 10-millisecond granularity. Discretizing time in a fixed grid on such a resolution would yield unnecessarily long sequences as not all events change on the same timescale. We hence adopt a sparse, MIDI-like, event-based representation from , allowing a minute of music with a 10-millisecond resolution to be represented at lengths around 2K. This is in contrast to a 6K to 18K length that would be needed on a serialized multi-attribute fixed-grid representation. As position in sequence no longer corresponds to time, a priori it is not obvious that relative attention should work as well with such a representation. However, we will show in Section 4.2 that it does improve perplexity and sample quality over strong baselines. We speculate that idiomatic piano gestures such as scales, arpeggios and other motifs all exhibit a certain grammar and recur periodically, hence knowing their relative positional distances makes it easier to model this regularity. This inductive bias towards learning relational information, as opposed to patterns based on absolute position, suggests that the Transformer with relative attention could generalize beyond the lengths it was trained on, which our experiments in Section 4.2.1 confirm. We show the first successful use of Transformers in generating music that exhibits long-term structure. Before our work, LSTMs were used at timescales of 15s (~500 tokens) of piano performances . Our work demonstrates that Transformers not only achieve state-of-the-art perplexity on modeling these complex expressive piano performances, but can also generate them at the scale of minutes (thousands of tokens) with remarkable internal consistency. Our relative self-attention formulation is essential to the model's quality. In listening tests (see Section 4.2.3), samples from models with relative self-attention were perceived as more coherent than the baseline Transformer model BID17. Relative attention not only enables Transformers to generate continuations that elaborate on a given motif, but also to generalize and generate in consistent fashion beyond the length it was trained on (see Section 4.2.1). In a seq2seq setup, Transformers can generate accompaniments conditioned on melodies, enabling users to interact with the model. The space complexity of the relative self-attention mechanism in its original formulation BID13 made it infeasible to train on sequences of sufficient length to capture long-range structure in longer musical compositions. To address this, we present a crucial algorithmic improvement to the relative self-attention mechanism, dramatically reducing its memory requirements from O(L 2 D) to O(LD). For example, the memory consumption per layer is reduced from 8.5 GB to 4.2 MB (per head from 1.1 GB to 0.52 MB) for a sequence of length L = 2048 and hidden-state size D = 512 (per head D h = D H = 64, where number of heads is H = 8) (see TAB1), allowing us to use GPUs to train the relative self-attention Transformer on long sequences. Sequence models have been the canonical choice for modeling music, from Hidden Markov Models to RNNs and Long Short Term Memory networks (e.g., BID3 ;), to bidirectional LSTMs (e.g., BID6 . Successful application of sequential models to polyphonic music often requires serializing the musical score or performance into a single sequence, for example by interleaving different instruments or voices. Alternatively, a 2D pianoroll-like representation (see A.1 for more details) can be decomposed into a sequence of multi-hot pitch vectors, and their joint probability distributions can be captured using Restricted Boltzmann Machines BID14 BID8 or Neural Autoregressive Distribution Estimators (NADE; BID10 . Pianorolls are also image-like and can be modeled by CNNs trained either as generative adversarial networks (GAN; BID4 . (e.g., BID2 or as orderless NADEs BID15 BID19) (e.g., BID9 . use self-similarity in style-transfer fashion, where the self-similarity structure of a piece serves as a template objective for gradient descent to impose similar repetition structure on an input score. Self-attention can be seen as a generalization of self-similarity; the former maps the input through different projections to queries and keys, and the latter uses the same projection for both. Dot-product self-attention is the mechanism at the core of the Transformer, and several recent works have focused on applying and improving it for image generation, speech, and summarization BID12). A key challenge encountered by each of these efforts is scaling attention computationally to long sequences. This is because the time and space complexity of self-attention grows quadratically in the sequence length. For relative self-attention BID13 this is particularly problematic as the space complexity also grows linearly in the dimension, or depth, of the per-position representations. 3.1 DATA REPRESENTATION We take a language-modeling approach to training generative models for symbolic music. Hence we represent music as a sequence of discrete tokens, with the vocabulary determined by the dataset. Datasets in different genres call for different ways of serializing polyphonic music into a single stream and also discretizing time. The JSB Chorale dataset consists of four-part scored choral music, which can be represented as a matrix where rows correspond to voices and columns to time discretized to sixteenth notes. The matrix's entries are integers that denote which pitch is being played. This matrix can than be serialized in raster-scan fashion by first going down the rows and then moving right through the columns (see A.1 for more details). Compared to JSB Chorale, the piano performance data in the Maestro dataset includes expressive timing information at much finer granularity and more voices. For the Maestro dataset we therefore use the performance encoding proposed by which consists of a vocabulary of 128 NOTE_ON events, 128 NOTE_OFFs, 100 TIME_SHIFTs allowing for expressive timing at 10ms and 32 VELOCITY bins for expressive dynamics (see A.2 for more details). The Transformer decoder is a autoregressive generative model that uses primarily self-attention mechanisms, and learned or sinusoidal position information. Each layer consists of a self-attention sub-layer followed by a feedforward sub-layer. DISPLAYFORM0 DISPLAYFORM1 The attention outputs for each head are concatenated and linearly transformed to get Z, a L by D dimensional matrix. A upper triangular mask ensures that queries cannot attend to keys later in the sequence. For other details of the Transfomer model, such as residual connections and learning rates, the reader can refer BID17. The feedforward (FF) sub-layer then takes the output Z from the previous attention sub-layer, and performs two layers of point-wise dense layers on the depth D dimension, as shown in Equation 2. W 1, W 2, b 1, b 2 are weights and biases of those two layers. DISPLAYFORM2 3.3 RELATIVE POSITIONAL SELF-ATTENTION As the Transformer model relies solely on positional sinusoids to represent timing information, BID13 introduced relative position representations to allow attention to be informed by how far two positions are apart in a sequence. This involves learning a separate relative position embedding E r of shape (H, L, D h), which has an embedding for each possible pairwise distance r = j k − i q between a query and key in position i q and j k respectively. The embeddings are ordered from distance −L + 1 to 0, and are learned separately for each head. In BID13, the relative embeddings interact with queries and give rise to a S rel, an L × L dimensional logits matrix which modulates the attention probabilities for each head as: DISPLAYFORM3 We dropped head indices for clarity. Our work uses the same approach to infuse relative distance information in the attention computation, while significantly improving upon the memory footprint for computing S rel. For each head, BID13 DISPLAYFORM4, containing the embeddings that correspond to the relative distances between all keys and queries. Q is then reshaped to an (L, 1, D h) tensor, and S rel = QR. 2 This incurs a total space complexity of O(L 2 D), restricting its application to long sequences. We improve the implementation of relative attention by reducing its intermediate memory requirement DISPLAYFORM0, with example lengths shown in TAB1. We observe that all of the terms we need from QR are already available if we directly multiply Q with E r, the relative position embedding. After we compute QE r, its (i q, r) entry contains the dot product of the query in position i q with the embedding of relative distance r. However, each relative logit (i q, j k) in the matrix S rel from Equation 3 should be the dot product of the query in position i q and the embedding of the relative distance j k − i q, to match up with the indexing in QK. We therefore need to "skew" QE r so as to move the relative logits to their correct positions, hence S rel = Skew(QE r). The "skewing" procedure is illustrated in FIG0 and will be detailed in the next section. The time complexity for both methods are O(L 2 D), while in practice our method is 6x faster at length 650 as prior work still requires manipulating larger tensors. Hence, we propose a "skewing" procedure to transform an absolute-by-relative (i q, r) indexed matrix into an absolute-by-absolute (i q, j k) indexed matrix. The row indices i q stay the same while the columns indices are shifted according to the following equation: FIG0 the upper right green dot in position of QE r after skewing has a column index of 2 − (3 − 1) + 0 = 0, ing in a position of in S rel. DISPLAYFORM1 We outline the steps illustrated in FIG0 below.1. Pad a dummy column vector of length L before the leftmost column.2. Reshape the matrix to have shape (L+1, L). (This step assumes NumPy-style row-major ordering.)3. Slice that matrix to retain only the last l rows and all the columns, ing in a (L, L) matrix again, but now absolute-by-absolute indexed, which is the S rel that we need. For very long sequences, the quadratic memory requirement of even baseline Transformer is impractical. Local attention has been used for example in Wikipedia and image generation (; by chunking the input sequence into non-overlapping blocks. Each block then attends to itself and the one before, as shown by the smaller thumbnail on the top right corner of FIG2 .To extend relative attention to the local case, we first note that the right block has the same configuration as in the global case (see FIG0) but much smaller: DISPLAYFORM0 where M is the number of blocks, and N be the ing block length) as opposed to L 2. The left block is unmasked with relative indices running from -1 (top right) to -2N + 1 (bottom left). Hence, the learned E r for the local case has shape (2N − 1, N).Similar to the global case, we first compute QE r and then use the following procedure to skew it to have the same indexing as QK, as illustrated in FIG2 1. Pad a dummy column vector of length N after the rightmost column.2. Flatten the matrix and then pad with a dummy row of length N − 1. 4.1 J.S. BACH CHORALES J.S. Bach Chorales is a canonical dataset used for evaluating generative models for music 3 (e.g., BID0 BID1 ; BID5 BID9 . It consists of score-based four-part chorales. We first discretize the scores onto a 16th-note grid, and then serialize them by iterating through all the voices within a time step and then advancing time (see A.1 for more details). As there is a direct correspondence between position in sequence and position on the timing/instrument grid in a piece, adding relative position representations could make it easier to learn this grammar. We indeed see relative attention drastically improve negative log-likelihood (NLL) over baseline Transformer TAB3 ). This improvement is also reflected in sample quality. The samples now maintain the necessary timing/instrument grid, always advancing four steps before advancing in time. As local timing is maintained, the model is able to capture timing on a more global level, giving rise to regular phrasing, as shown in the right score of FIG1. In addition to relative attention, we explored enhancing absolute timing through concatenating instead of adding the sinusoids to the input embeddings. This allows the model to more directly learn its absolute positional mapping. This further improves performance for both the baseline and relative transformer TAB3. We compare against COCONET as it is one of the best-performing models that has also been evaluated on the 16-note grid using the canonical dataset split. To directly compare, we re-evaluated COCONET to obtain note-wise losses on the validation set 4. For the Transformer models (abbreviated as TF), we implemented our attention mechanisms in the Tensor2Tensor framework. We use 8 heads, and keep the query, key (att) and value hidden size (hs) fixed within a config. We tuned number of layers (L in {4,5,6}), attention hidden size (att in {256, 512}) and pointwise feedforward hidden size (ff in {512, 1024}). A musical event bears multiple attributes, such as timing, pitch, instrument etc. To capture more relational information, we extend relative attention to capture pairwise distances on additional attributes. We learn separate relative embeddings for timing E t and also pitch E p. E t has entries corresponding to how many sixteenth notes apart are two positions in time, while E p embeds the pairwise pitch interval. However this approach is not directly scalable beyond J.S. Bach Chorales because it involves explicitly gathering relative embeddings for R t and R p, ing in a memory complexity of O(L 2 D) as in BID13. This is due to relative information being computed based on content as opposed to content-invariant information such as position in sequence. It was sufficient to add these extra relational information to the first layer, perhaps because it is closest to the raw input content. Here, the relative logits are computed from three terms, We train on random crops of 2048-token sequences and employ two kinds of data augmentation: pitch transpositions uniformly sampled from {−3, −2, . . ., 2, 3} half-steps, and time stretches uniformly sampled from the set {0.95, 0.975, 1.0, 1.025, 1.05}. For evaluation, we segment each sequence sequentially into 2048 length subsequences and also keep the last subsequences that are of shorter lengths. This in 1128 and 1183 subsequences in the validation and test set respectively. Each subsequence is then evaluated by running the model forward once with teaching forcing. As the subsequences vary in length, the overall negative loglikelihood (NLL) is averaged entirely on the token level. We compare our to PerformanceRNN (LSTM, which first used this dataset) and LookBack RNN (LSTM with attention) BID19. LookBack RNN uses an input representation that requires monophonic music with barlines which is information that is not present 5 Maestro dataset: urlhttps://magenta.tensorflow.org/datasets/maestro 6 Piano-e-Competition: http://www.piano-e-competition.com/ 7 COCONET is an instance of OrderlessNADE, which approximates a mixture model over orderings where orderings are assumed to be uniformly distributed. Hence, its loss is computed by averaging losses over multiple random orderings. The current row in the table reports this loss. In contrast, the row above corresponds to evaluating Coconet as an autoregressive model under the chronological ordering. BID9 show that sample quality is better when using Gibbs sampling (which uses conditionals from multiple orderings) as opposed to autoregressive generation (which only uses conditionals from one ordering). DISPLAYFORM0 in performed polyphonic music data, hence we simply adopt their architecture. TAB4 shows that Transformer-based architectures fits this dataset better than LSTM-based models. We implemented our attention mechanisms in the Tensor2Tensor framework, and used the default hyperparameters for training, with 0.1 learning rate and early stopping. We compare four architectures, varying on two axes: global versus local, and regular versus relative attention. We found that reducing the query and key channel size (att) to three forth of the hidden size (hs) works well for this dataset and used this setup for all of the models, while tuning on number of layers (L) and dropout rate (d). We use block size (bs) 512 for local attention (; . For relative global attention, the maximum relative distance to consider is set to half the training sequence length. For relative local attention, it is set to the full memory length which is two blocks long. TAB4 shows that our memory-efficient relative attention formulations outperform regular attention in both the global and the local case. When looking at the other axes, we see global attention outperforming local attention in both the relative and regular case. Global attention may have the advantage of being able to directly look back for repeating motifs. With a larger dataset, local attention may fare well as it allows for much deeper models and longer sequences, as seen in text and image generation work . In turn, both domains could benefit from the use of relative local attention. When primed with an initial motif (Chopin's Étude Op. 10, No. 5) shown in the top left corner of FIG4, we see the models perform qualitatively differently. Transformer with relative attention elaborates the motif and creates phrases with clear contour which are repeated and varied. Baseline Transformer uses the motif in a more uniform fashion, while LSTM uses the motif initially but soon drifts off to other material. Note that the generated samples are twice as long as the training sequences. Relative attention was able to generalize to lengths longer than trained but baseline Transformer deteriorates beyond its training length. See Appendix C for visualizations of how our Relative Transformer attends to past motifs. To explore the sequence-to-sequence setup of Transformers, we experimented with a conditioned generation task where the encoder takes in a given melody and the decoder has to realize the entire performance, i.e. melody plus accompaniment. The melody is encoded as a sequence of tokens as in BID19, quantized to a 100ms grid, while the decoder uses the performance encoding described in Section 3.1 (and further illustrated in A.2). We use relative attention on the decoder side and show in TAB5 that it also improves performance. To compare the perceived sample quality of models trained on the Maestro dataset, and their ability to generate a continuation for a priming sequence, we carried out a listening test study comparing the baseline Transformer, our Transformer with relative-attention, PerformanceRNN (LSTM), and the validation set. Participants were presented with two musical excerpts (from two different models that were given the same priming sequence) and asked to rate which one is more musical on a Likert scale. For each model, we generated 10 samples each with a different prime, and compared them to three other models, ing in 60 pairwise comparisons. Each pair was rated by 3 different participants, yielding a total of 180 comparisons. Figure 5 shows the number of comparisons in which an excerpt from each model was selected as more musical. The improvement in sample quality from using relative attention over the baseline Transformer model was statistically significant (see Appendix B for the analysis), both in aggregate and between the pair. Even though in aggregate LSTMs performed better in the study than the Transformer, despite having higher perplexity, but when compared against each other head to head, the were not statistically significant (see TAB6 in Appendix B).(Ours) Figure 5: Samples from the Transformer with our efficient relative attention were rated more musical more times than LSTM and baseline Transformer. Error bars show standard deviation of the mean. In this work we demonstrated that the Transformer equipped with relative attention is very well-suited for generative modeling of symbolic music. The compelling long-term structure in the samples from our model leaves us enthusiastic about this direction of research. Moreover, the ability to expand upon a prime, in particular, suggests potential applications as creative tool. The significant improvement from relative attention highlights a shortcoming of the original Transformer that might also limit its performance in other domains. Improving the Transformer's ability to capture periodicity at various time scales, for instance, or relations between scalar features akin to pitch could improve time-series models. Our memory-efficient implementation enables the application of relative attention to much longer sequences such as long texts or even audio waveforms, which significantly broadens the range of problems to which it could be applied. Adapting sequence models for music requires making decisions on how to serialize a polyphonic texture. The data type, whether score or performance, makes certain representations more natural for encoding all the information needed while still ing in reasonable sequence lengths. A.1 SERIALIZED INSTRUMENT/TIME GRID (USED FOR THE J.S.BACH CHORALES DATASET) The first dataset, J.S. Bach Chorales, consists of four-part score-based choral music. The time resolution is sixteenth notes, making it possible to use a serialized grid-like representation. Figure 6 shows how a pianoroll (left) can be represented as a grid (right), following BID9. The rows show the MIDI pitch number of each of the four voices, from top to bottom being soprano (S), alto (A), tenor (T) and bass (B), while the columns is discretized time, advancing in sixteenth notes. Here longer notes such as quarter notes are broken down into multiple repetitions. To serialize the grid into a sequence, we interleave the parts by first iterating through all the voices at time step 1, and then move to the next column, and then iterate again from top to bottom, and so on. The ing sequence is S 1 A 1 T 1 B 1 S 2 A 2 T 2 B 2..., where the subscript gives the time step. After serialization, the most common sequence length is 1024. Each token is represented as onehot in pitch. S: 67, 67, 67, 67 A: 62, 62, 62, 62 T: 59, 59, 57, 57 B: 43, 43, 45, 45 Figure 6: The opening measure of BWV 428 is visualized as a pianoroll (left, where the x-axis is discretized time and y-axis is MIDI pitch number), and encoded in grid representation with sixteenth note resolution (right). The soprano and alto voices have quarter notes at pitches G4 and D4, the tenor has eighth notes at pitches B3 and A3, and the bass has eighth notes at pitches A2 and G2. The second dataset, Maestro BID7, consists of polyphonic piano performances with expressive timing and dynamics. The time resolution here is on the millisecond level, so a grid representation would in sequences that are too long. Instead, the polyphonic performance is serialized into a sequence of one hot encoded events as proposed in.First, the input MIDI files are preprocessed to extend note durations based on sustain pedal control events. The sustain pedal is considered to be down whenever a sustain control change is encountered with a value >= 64; the sustain pedal is then considered up after a control change with a value < 64. Within a period where the sustain pedal is down, the duration of each note is extended to either the beginning of the next note of the same pitch or the end of the sustain period, whichever happens first. If the original duration extends beyond the time when the sustain pedal is down, that original duration is used. Next, the MIDI note events are converted into a sequence from the following set of vocabulary: 128 NOTE_ON events for starting a note of with one of the 128 MIDI pitches, 128 NOTE_OFF events for ending a note with one of the 128 MIDI pitches, 100 TIME_SHIFT events representing forward time shifts in 10ms increments from 10ms to 1s, and 32 SET_VELOCITY events representing the velocity for future NOTE_ON events in the form of the 128 possible MIDI velocities quantized into 32 bins. An example performance encoding is illustrated in Figure 7. TAB1 TIME_SHIFT<500>, NOTE_OFF<65> Figure 7: A snippet of a piano performance visualized as a pianoroll (left) and encoded as performance events (right, serialized from left to right and then down the rows). A C Major chord is arpeggiated with the sustain pedal active. At the 2-second mark, the pedal is released, ending all of the notes. At the 3-second mark, an F is played for.5 seconds. The C chord is played at velocity 80 and the F is played at velocity 100. Participants were presented with two musical excerpts that shared a common priming sequence. For each excerpt, the priming sequence was played, followed by 2.5 seconds of silence, followed by the priming sequence again and a continuation of that sequence. The continuations were either sampled from one of the models or extracted from our validation set. We evaluated all possible pairs in the space of data and model samples, except from the same model. Each continuation had a length of 512 events using the encoding described in Section A.2. This corresponds to the length the models were trained on to remove the deteriorating effect that happens with baseline Transformer when asked to generate beyond the length it was trained on. Participants were asked which excerpt they thought was more musical on a Likert scale of 1 to 5. The pair is laid out left versus right, with 1 indicating the left is much more musical, 2 the left is slightly more musical, 3 being a tie, 4 being the right is slightly more musical, and 5 the right is much more musical. For each model, we generated 10 samples each with a different prime, and compared them to three other models, ing in 60 pairwise comparisons. Each pair was rated by 3 different participants, yielding a total of 180 comparisons. A Kruskal-Wallis H test of the ratings showed that there was a statistically significant difference between the models: χ 2 = 63.84, p = 8.86e-14< 0.01. TAB6 show a post-hoc analysis on the comparisons within each pair, using the Wilcoxon signed-rank test for matched samples. TAB7 shows a post-hoc analysis of how well each model performed when compared to all pairs, and compares each model's aggregate against each other, using the Mann-Whitney U test for independent samples. We use a Bonferroni correction on both to correct for multiple comparisons. The win and loss counts bucket scores 4, 5 and scores 1, 2 respectively, while the tieing score is 3.Both within pairs and between aggregates, participants rated samples from our relative Transformer as more musical than the baseline Transformer with p < 0.01/6.For within pairs, we did not observe a consistent statistically significant difference between the other model pairs, baseline transformer versus LSTM and LSTM versus relative Transformer. When comparing between aggregates, LSTM was overall perceived as more musical than baseline Transformer. Relative Transformer came a bit close to outperforming LSTM with p = 0.018. When we listen to the samples from the two, they do sound qualitatively different. Relative Transformer often exhibits much more structure (as shown in FIG4), but the effects were probably less pronounced in the listening test because we used samples around 10s to 15s, which is half the length of those shown in FIG4 to prevent the baseline Transformer from deteriorating. This weakens the comparison on long-term structure. When compared to real music from the validation set, we see that in aggregates, real music was better than LSTM and baseline Transformer. There was no statistical significant difference between real music and relative Transformer. This is probably again due to the samples being too short as real music is definitely still better. One advantage of attention-based models is that we can visualize its attention distribution 3. This gives us a glimpse of how the model might be building up recurring structures and how far it is attending back. The pianorolls in the visualizations below is a sample generated from Transformer with relative attention. Each figure shows a query (the source of all the attention lines) and previous memories being attended to (the notes that are receiving more softmax probabiliy is highlighted in). The coloring of the attention lines correspond to different heads and the width to the weight of the softmax probability. Figure 8: This piece has a recurring triangular contour. The query is at one of the latter peaks and it attends to all of the previous high notes on the peak, all the way to beginning of the piece. Steps 2,3:Figure 10: Relative global attention: Steps (from left to right) for "skewing" an absolute-by-relative (i q, r) indexed matrix into absolute-by-absolute (i q, j k). Grey indicates self-attention masks or entries introduced by the skewing procedure. Positions with relative distance zero are marked. Entries outlined by purple are removed in step 3. Steps 1, 2Steps 3 Steps 4Figure 11: Relative local attention: Steps (from left to right) for "skewing" an (i q, r) indexed matrix with 2N − 1 ranged relative indices r into (i q, j k indexed. Shapes are indicated above the boxes, while indices in the boxes give relative distances."

Large scale multi-task architecture solves ImageNet and translation together and shows transfer learning.

Represent this text to find a much much longer article about it with say hundreds-thousands of words.

We propose a new module that improves any ResNet-like architectures by enforcing "channel selective" behavior to convolutional layers


Question: Represent the text to find a very short summary (1-2 sentences max)

Question: Machine learning algorithms for controlling devices will need to learn quickly, with few trials. Such a goal can be attained with concepts borrowed from continental philosophy and formalized using tools from the mathematical theory of categories. Illustrations of this approach are presented on a cyberphysical system: the slot car game, and also on Atari 2600 games. There is a growing need for algorithms that control cyberphysical systems to learn with very little data how to operate quickly in a partially-known environment. Many reinforcement-learning (RL) solutions using neural networks (NN) have proved to work well with emulators, for instance with the Atari 1 2600 games BID17, or with real systems such as robots BID11. However, these state-of-the-art approaches need a lot of training data, which may not be obtainable within the allowed time frame or budget. This work thus started as an alternative approach to teach computers to learn quickly to perform as efficiently as the existing solution with approximately one percent of the training data, time, and computing resources. We first review reinforcement learning methods for Markov Decision Processes (MDP) and Partially Observable MDP (POMDP). We then explain the motivation behind our continental-philosophyinspired approach. We describe the two classes of problems on which we focus: the bijective case, which may lead to playing by imitating, and the category-based approach, which should lead to a more innovative behavior of the control algorithms. Both approaches rely on knowledge accumulated during previous experiences, as in Lifelong Machine Learning BID6.These two approaches are illustrated by from both a commercial slot car game controlled by an 8-bit Arduino system, and from Atari 2600 video games running within the Arcade Learning Environment (ALE, see BID1). The development of Artificial Intelligence (AI) owes much to games, which have become one of the classical test-beds for algorithms. Slot car games, for instance, are used to evaluate the performance of decision-making systems. The image-processing, RL-based approach presented in BID11 estimates a car's position on the track thanks to a multilayer perceptron with convolutional layers. It controls the car by applying one out of four possible voltage levels. Training the perceptron takes twelve hours, and learning the control strategy needs another half an hour. The faster solution by BID22 to autonomous slot cars relies on added acceleration sensors and an embedded microcontroller to first create a map of curved and straight tracks. The control algorithm then sets the target velocity and controls it using a Phase-Locked Loop. TM are trademarks of their respective owners and will be written without the trademark symbol for clarity in the remainder of this document. Our system learns to rank with human players in less than a minute, without embedded sensors, for both known and unknown circuits, which correspond to the aforementioned bijective and categorybased approaches. The sensors are a lap counter, and voltage and current from the track. Video games also become increasingly useful for providing a cyber-physical representation of our environment. Indeed, realism turns out to be one of the main focuses for game and character designers (depending on the game intentions). Nevertheless, due to technical limitations and the need for easily described problems, older video-gaming systems, such as the Atari 2600 console, are the go-to systems. They provide a wide variety of situations ranging from mazes (Pac-Man-style games) and action games (such as Frogger) to ball-and-paddle games (Breakout, Pong). Although these different problems require varied strategies to be tackled by a standard human player, they all involve decision-making and, therefore, have been modeled as MDPs BID17.This framework allows the implementation of many different methods. Some works, such as BID17 use a rescaled picture of the playing area as an input for a deep Q-network (DQN) in order to select the best action available to the agent. Another possibility is the use of classic searching and planning methods in order to guide the agent, such as the Iterated Width algorithm BID15 or tree search algorithms such as Monte-Carlo Tree Search (MCTS) BID20 to compute the best possible action for the agent. A less common method is Shallow Reinforcement Learning BID14: although this relies on a simpler linear representation, it obtains similar to those of the non-linear approaches. Finally, Apprenticeship Learning BID2 and Inverse RL BID12 can also be used to train a more humanlike agent which is close to the aforementioned methods in terms of efficiency. We will show that our system learns how to play unknown games in a few thousand frames with a score on par with or better than humans. The success of NN is partially due to the very large amount of data that is used, even if the programmer does not know exactly what happens in the NN (like in black box systems). Two problems are thus the huge quantity of data needed, and the training time required. Moreover, the attribution of good coefficients in the learning phase is very difficult -or impossible -to be interpreted, making validation very difficult. NN are able to learn and generalize, but we do not know exactly how. To improve or complete the NN approach, we propose an approach that tries to explain and use explicitly how an AI can learn, extract features, categorize and generalize. To do this, we place the theoretical elements necessary for such high level abilities directly in the method. These abilities may also emerge in NN after many elementary computations (additions, multiplications, comparisons) occur at each artificial neuron. If this is what effectively happens in each biological neuron, we postulate that intelligence also consists in higher level intellectual operations. In other words, we do not want to reduce intelligence to basic computation -even if it is biologically the case. As we do not want higher level abilities to emerge (or not) after long training times, we explicitly place these high level abilities (such as categorizing and generalizing) directly in our theoretical framework. Thus we can follow some aspects of Dreyfus' critique of AI presented in BID8. This author claims that AI researchers should focus more on what human intelligence is in itself and not only refer to the computer model: considering the brain as a computer, and intelligence as the use of software. More precisely, in the case of RL with NN: considering intelligence as a collection of elementary computations that organize themselves after much training to reach a reward goal. We postulate that it could have happened in such a way over the course of human development, but human intelligence has much evolved. It can produce categorization and generalization not merely for a simple reward, but for the goal of understanding. This is what our AI tries to do. Dreyfus also often refers to authors such as Heidegger, Husserl or Foucault, whose work later became known as continental philosophy. This name was given by analytic philosophers who were often Anglo-Saxon in origin, the earliest being Russell, Frege and Wittgenstein. Analytic philosophy received much influence from mathematical logic that emerged at the end of the 19th century. It tries to clarify philosophical issues by logical analysis, postulating that only philosophical statements verifiable through empirical observations are meaningful (principle of logical positivism). This principle, according to analytic philosophers, is not respected by continental philosophers. Continental philosophy includes a range of French and German doctrines from the 19th and 20th centuries: German idealism, phenomenology, existentialism (influenced by Kierkegaard and Nietzsche), hermeneutics, structuralism, post-structuralism, psychoanalytic theory and object-oriented ontology. These philosophies are all contrary to the analytic movement. If we had to project AI in this debate (analytic versus continental), we could say that Dreyfus criticizes early AI for favoring the analytic tradition and for neglecting the continental one. Of course machines are computers, and computing is closer in nature to logic than phenomenology, metaphysics or psychoanalysis. Continental philosophy, however, can perhaps help understand and describe what human intelligence is, especially for high level abilities, like learning, categorizing, generalizing and understanding. It could possibly then improve the quality and efficiency of human-intelligence-based AI.To summarize, we propose to design our AI using an approach based on certain elements of continental philosophy. This philosophy is described, for lack of a more precise and widely-accepted definition, in terms of its opposition to analytical philosophy. In the next sections, we will propose some connections between this philosophy and existing mathematical theories. We express the logic of our AI at the level of entities, and not at a sample or at a pixel level. In a way, this is similar to working at the morpheme level in structural linguistics, as defined by de , which is the smallest meaningful unit of a language. That implies to start with an analysis of the sampled signals (in one or two dimensions) to detect entities. These entities are like our everyday life objects: tracks (straights and curves), cars, balls, paddles, walls. They are geometrically organized in a space and can be described by cartesian coordinates. We have defined a distance between them that measures how far two entities E and F are one from one another 2. The data is collected at each sample time so that we can construct a timeline and provide an elementary cinematic newtonian model of the situation. This comes from a very old idea of developmental psychology (see for example the works of BID21) that the child starts his cognitive development by the skill of experiencing the world through movement and senses (Piaget called it the sensorimotor stage). But the perceptive world is not a wild set of disordered primitive sensations. They are organized in objects (we call them entities) that take place in a space and can move during time 3. Thus we do not want to take into account all the samples (voltage and current for the slot car, pixels' colors for images) as the fundamental level of knowledge. We shall try to organize them as soon as possible as entities that occur in space and time (and not wait for them to emerge, or not, after a very long learning process). These entities, like the objects of cognitive psychologists, have some properties: relative consistency, continuity of movement, permanence of existence and characteristics (sizes, color, shape). These properties are part of our approach, in the sense that our AI can look for rectangular entities with a particular position, speed and size 4. In more complex games, this rectangular form approach could be too simple, but it is adequate for the Atari 2600 games that we study. One of the main critiques formulated by Dreyfus against the old AI philosophy is the epistemological assumption that claims that all activities can be formalized in terms of predictive rules or laws. In this context, the learning phase consists of determining these rules (that is, their parameters). Then, the system has to apply them by looking for objects or general characteristics of the whole organization of samples, that are like those used in the learning phase. But what about new objects, never seen before, that could appear? Such a strict and trivial application of the epistemological assumption would lead to ignore them. It could also be a principle of precaution to ignore new objects. On the other hand, there could be a principle of curiosity or adventure. Clever machines could be more efficient were they curious as explained in BID19. Referring to the work of Alison Gopnik and Laura Schulz, developmental psychologists at Berkeley and at the Massachusetts Institute of Technology, respectively, it explains that babies naturally gravitate to objects that surprise them rather than to those they are used to, to achieve some extrinsic goal. An AI that only focuses on application of predictive rules will miss the advantages of curiosity. We will use this curiosity to further develop our AI in our next work. If the epistemological assumption of usual AI could be useful for chemistry or physics, because they are context-free, it could be a contradiction in terms with psychology, and behavior understanding. Dreyfus argued that human problem solving depends rather on our sense of the context, that is the natural feeling, understanding or intuition of what is important and interesting given a situation. The world is not just made of objects: it contains subjects. In particular, in the games we consider, there is a representative of what we call the "Me": the entity that is controlled by actions. This point of view allows for a more efficient approach than computing all the possible combinations of the available symbols. This is exactly what we do when we ask our AI to look as soon as possible for some important features (entities and the "Me"). BID7 referred to the Heideggerian concept of Dasein (which means "being there", for a human being confronted with such issues as personhood and mortality), which is a specific way of Being-in-the-world (another Heideggerian concept that considers it as a unity, saying that it is not appropriate to distinguish strictly between the Being and the world that it is in) BID9.In other words, one of the first things that our AI must do is to identify the "Me" from amongst all the listed entities. It is not an implicit potential of a huge number of trainings, like in some RL processes. Moreover, being the "Me" does not mean only to be lead by actions. It also implies to struggle for life. We can say that the "Me" is driven by some life impulses, and that it is attracted to the good objects (that we call friends) and wants to go away from the bad ones (the enemies). Thus postulate that among all the entities, some are friends (those whose contact implies a reward or avoids loss of lives) and some are enemies (those whose contact implies loss of lives). The AI has to distinguish as soon as possible the friends versus the enemies of the "Me", without waiting for this to emerge from millions of trials. After that, the survival strategy is simple: try to meet the friends unless there is an enemy close to the "Me", in which case the first thing to do is to flee. One of the most efficient tools that humans use to understand new situations is the ability to make analogies between past and present. For example, if the AI knows how to play the game Breakout, we expect that it will be able to transpose this ability to a (partially) analogous game, Pong. In particular, we hope to soon use mathematical tools to transpose a policy from one problem (for instance a game) to another. Such a theory is proposed by BID3 for PONDP (Partially Observable Non Deterministic Problems).The problem is that ideal situations where two problems have exactly the same number of states and isomorphic structures are very rare. Nevertheless, there are mathematical tools that can be used to identify non isomorphic structures like equivalence of categories in category theory BID16 5. The theory of category is a powerful tool in modern mathematics that appeared in the mid20th century in topological and geometrical contexts, after the mathematical logic, based on set theory. If mathematical logic was a great source of inspiration for analytic philosophy, category theory could inspire and support continental ideas. The association between category theory and continental philosophy is proposed by and we will follow this path in our work. In a very simplistic way, we could say that if analytic philosophy analyses situations, by distinguishing states (or objects), continental philosophy provides syntheses, setting higher new levels of being (Beings, concepts, types). Whereas in set-theory-based logic, identification is reduced to identity and bijective relations, category theory provides richer descriptions of objects by the introduction of arrows between objects, allowing new kind of identifications. The reader familiar with category theory may find obvious the rest of this paragraph. However, as most Machine Learning tools rely on set theory and not category theory, we try to illustrate below the added value of this mathematical framework. The reader is nevertheless referred to for a thorough and in-depth explanation of category theory. A category C is a collection of objects with arrows between some of them, so that we can compose them. It is something like an oriented graph. In C, an arrow a: A → B is called an isomorphism if it is invertible, that is if there is an arrow b: B → A, such that ba = Id B and ab = Id A. If it is the case we say that the object A and B are isomorphic. The relation of isomorphism defines an equivalence relation on the collection of objects of C. We note the quotient C/. If F: C → C is an equivalence of categories, it induces a real bijection F: (C/) → (C /) between the classes of isomorphic objects even if F is not bijective. We do not identify the objects (or the states) of two situations one-to-one, we identify the types (or classes) of these states. This process can be very useful in the context of observable problems. Let's consider two nonempty sets (of states) C and C not necessary of the same cardinals. Let's suppose that we have two functions of observation f: C → O and f: C → O. Let's assume that they are surjective (if not, we can restrict O and O). The sets of observations O and O will define some types of states. For each o ∈ O, we say that all the states x ∈ C that are observed as o (f (x) = o), have the type T o. This defines a natural equivalence relation R f on the set C: ∀x, y ∈ C, x R f y if and only if f (x) = f (y).In terms of categories, we put an invertible arrow between two objects x and y of C iff x R f y (iff stands for if and only if). This makes C a category, where all arrows are invertible and such that C/ is exactly the quotient C/R f: the set of types of states of C. It is well known that the surjection f: C → O induces a bijectionf: (C/) → O between the set of type and the set of observations. This is obvious since the types as been defined by the observations.f is actually the inverse of DISPLAYFORM0 We do exactly the same with C and f.Suppose now, and this is very important, that the sets of observations O ad O have the same cardinality by the means of a bijection G: O → O. Thus, we can define a bijection F =f DISPLAYFORM1. This bijection between the sets of types can be induced by an equivalence of categories F: C → C defined as follows: for every x ∈ C, let's call o = f (x) and chose an arbitrary x ∈ f −1 (G(o)), and define F (x) = x. If C and C do not have the same cardinality, F has no chance to be bijective, but F is. F sends every state x to a state x of the "same" type (up to G). This is the way that we identify (not necessarily bijectively) C and C. Thus, if we have a strategy to play in C, we can transpose it in C thanks to F.The use of the theory of category in the ability to formalize a wide variety of games and situations. An illustration of this would be the ease with which a human player can switch from the Atari 2600 game Breakout to the very similar Pong. This ease can be transposed into the formalism of categories. However, even a much more concrete system such as the slot car described in section 2.1 and experimented on in section 4.1 can be transcribed into the formalism of categories 6.Let us define the following sets:• {C, C} is the set of categories (one per configuration of the track).• {N, N} is the number of sections per configuration of the track.• {s, s ∈ [1, N]}, {s, s ∈ [1, N]} are the possible locations of the car on the circuit. The location is obtained by counting the number of sections the car has passed in its current lap. We note (u, i) s (Resp. (u, i) s ) the voltage and current measured when the car crosses section s (Resp. s) Let 1 ≤ s 0 ≤ N (Resp. 1 ≤ s 0 ≤ N) be the current position of the car in configuration C (Resp. C).• Let k be a straight section and l a curve section of C.• The player influences (u, i) s with the controller, which leads to the policy π defined by. DISPLAYFORM2 We want to identify C and C, to transpose the policy π from C to C. The states of C are the locations s of the car in the circuit. Similarly, the states of C are the s. If N = N and s 0 = s 0, we can define a bijection between C and C and easily transpose π. But if N = N or s 0 = s 0 it is impossible to define such a bijection. Nevertheless, if we turn C and C into categories by defining some arrows, we will be able to define an equivalence of categories F: C → C. To define these arrows, let's use the observable f defined on the states s of C as follows f: C → {1, 2} with f = h • g where g and h are such that g(s) = (u, i) s and h ((u, i) s ) = 1 if s is a curve, and 2 otherwise. We define f on the s of C the same way, i.e. f: C → {1, 2} with f = h • g and g and h playing the same roles as g and h on the states of C.We can put an invertible arrow between two states of C iff they have the same image by f, and an invertible arrow between two states of C iff they have the same image by f. We then define F: C → C by equation. DISPLAYFORM3 It is easy to see (if the exact definitions are known) that FORMULA3 is an equivalence of categories that allows to transfer π from C to C. F induces a bijection F between the sets of classes (or types of position): DISPLAYFORM4 We finally obtain F (C) = C (types of curves) and F (S) = S (types of straights).This example of systematic categorization and generalization proves that we do not work at the level of states but that type of states are considered instead. Results of this approach are presented for a cyberphysical system: a slot car circuit, and for a simulated system: Atari 2600 video games. The focus on the slot car experimental setup arose from the need to validate the approach on a cyberphysical system. With its imperfect actuators such as a brushed, direct-current (DC) motor, imperfect contacts such as metallic brushes on strips, it allowed us to evaluate the approach while dealing with a wide range of signals from a real system. Moreover, its wide availability and low cost allowed to duplicate the test-bed so as to widen the span of the validations. On the other hand, the configuration is simple, as there is only one entity with dynamic behavior: the "Me' is the slot car. The enemies are located at unknown curvilinear abscissas where a high velocity is detrimental to the "Me". The setup is based on a Scalextric MINI Challenge Set C1320T. We have replaced the mechanical lap counter by a digital omnipolar Hall effect sensor DVR5033 from Texas Instruments. The current is sensed via a 1 Ω resistor in series with the metal strips carrying the power. A spectrum analysis of both the voltage and the current showed components in these signals above 350 Hz. The antialiasing, second-order filter was designed with a cut-off frequency f c = 31 Hz. The Design-to-Cost approach, classic in high-volume manufacturing, led to the now unusual choice of a real-pole filter G(s) = 1/(sRC +1) 2, where s is the Laplace variable, approximated by a Cauer Resistor Capacitor (RC) ladder network BID0. The values R and C are chosen thanks to f c = 1/(2πRC). Moreover, the scaled values of the second RC network, R/d and Cd with {d ∈ R : d > 0}, are computed to meet the specifications of the maximum magnitude error e(d) between G(s) and G a (s), the transfer function of the Cauer RC ladder defined by G a (s) = 1/ (sRC) 2 + s(d + 2)RC + 1. The value of e(d) is given by equation. We chose d = 0.1 to have less than 0.5 dB error, with no sensible impact on the later computations. An implementation with two identical RC sections (i.e. d = 1) would lead to e = 3.5 dB, which would degrade the overall performance. Both the voltage and the current are filtered by such ladder networks before being sampled at f s = 100 Hz:as there are no components in the power spectrum between f s /2 and 350 Hz, there is no aliasing. DISPLAYFORM0 The algorithms are written in C language and run in real-time on an Arduino Mega 2560 which has 8192 bytes of Random Access Memory (RAM). The analog signals are sampled and quantized by the integrated analog to digital converter in the microcontroller, with the sampling period defined by t s = 1/f s, and the sampling time being kt s with k ∈ N.The bijective case for the slot car relies on an three-step imitation procedure:1. A human player first drives the car for n laps, with n = 3 in our experiments.2. The K sampled voltages v(kt s) and currents i(kt s) of the shortest lap (with corresponding t best lap time) are stored in RAM for 0 ≤ k < K, to be replayed by the AI.3. An optimization method (Newton) minimizes the difference between the AI's lap time and t best by scaling the recorded samples v(kt s) used to generate the Pulse-Width Modulation (PWM) control signal. The analogy-based approach relies on two modules: the reward module, and the decision module described below. As in traditional RL, our approach relies on a reward from the environment. This reward is based on three variables: the lap time (measured directly with the lap counter), the presence of the car on the track (binary information), and the fact that the car is moving (also binary information). The algorithm that we designed to provide this reward constantly monitors the car so as to detect that it did not crash (i.e. that it did not leave the track when the velocity was too high) or that it did not stop (when the current was too low to move the car). Both detectors are based on k-nearest neighbors algorithms (k-NN) applied to the voltage and the current. They are implemented as boolean tests on the signals after comparison with some thresholds, to speed up the execution of the algorithm on the microcontroller. As an illustration, a crash can be detected when the voltage is high and the current is near zero: it means that there is no more load (no DC motor) in contact with the strips, even though the voltage is still applied. Using this reward model, the AI can successfully pilot the car on previously unencountered tracks. It does not replay scaled samples of any human driving. The only information reused by the algorithm is the safe speed: it does not trigger the "car crash" reward signal, yet it maintains the car in motion, thus not triggering the "car stop" reward signal. As the circuit is unknown, the bijective case cannot be used: there is no bijection between circuits. The algorithm only relies on the analogy-based approach and transposes knowledge previously acquired for a different circuit configuration thanks to equation FORMULA2. This knowledge -a safe speed for a given s -is transposed via non-bijective analogies presented in 3.3 with the function h((u, i) s ) evaluated with a classifier. Any classifier can be used, including unsupervised learning methods, as the two classes are clustered and separated. For simplicity, we used a k-NN.In practice, the analogy-based approach starts on the unknown circuit with the safe speed. The algorithm infers in real time, from only current and voltage measurements, whether the car is in a configuration that we humans call either curve or straight. The algorithm then chooses the best control signal based on its previous experiences (best in order to reach the goal of decreasing lap time while staying on the track). Even though we use the terms "straight" and "curve" in our explanation, the algorithm simply classifies current and voltage to choose a control signal so as to stay on the track while decreasing the lap time. The algorithm uses this past knowledge (the control signal for each class) in a previously unencountered situation. In this way, it generalizes its strategy and adapts to a radically different case: circuit 2 differs from circuit 1, and a replay of a recorded strategy learned on one circuit or scaled recorded samples of the human driving would fail on the second circuit. The experiments described in this article are conducted on two circuit configurations of different complexity presented in FIG0. The of our experiments for the bijective and the analogy cases are summarized in table 1. Values are tabulated as the mean and the standard deviation from the mean, except for the best lap which is the shortest lap time among all laps. We noticed that the first of eight consecutive laps is always the slowest one for the eight human subjects. The AI, which starts with no previous information, only relies on a safe speed as described in 4.1.1 using a constant PWM of 39% of the full speed. The analogy-based AI, which does not replay any recorded samples of a human driving, improves lap times in less than ten laps, even on an unknown track. On the longest and most complex circuit configuration (circuit 2), it almost ranks best, as tabulated on the line "Final lap". While the final human lap time is lower than the final AI lap time (2.29 s vs 2.52 s for circuit 1, 3.08 s vs 3.13 s for circuit 2), the human unsurprisingly exhibits a higher standard deviation from the mean (140 ms vs 80 ms for circuit 1, 540 ms vs 20 ms for circuit 2). Future improvements of the AI on the unknown track will include an optimization of the two speeds transposed by the function h((u, i) s ): only a safe speed was used during our experiments, leading to no car crash for the AI, contrary to some laps by the humans and thus not taken into account. Lastly, the bijective strategy -imitating the best human lap -also leads to the best lap time. However, contrary to the solution with analogies, it only works for an identical circuit. This means that while the best bijective (imitation) lap time (2.65 s) for circuit 2 is lower, thus better than the final lap time for the analogy (adaptive speed) AI (3.13 s), this strategy can only be used on circuit 2 and cannot lead to a generalization. It is only mentioned here as it gives an empirical lower bound for the lap time on a given circuit. To summarize the slot car case, we implemented the theoretical method exposed in section 3.3 that allows the AI to reuse previously acquired knowledge on a new circuit where a replay of recorded samples of a human driving would lead to an immediate car crash. Even though there is no bijection between the different circuits, in practice this theory allows to generalize knowledge to any different circuit (within the limits imposed by the size of the available RAM). While the slot car allowed us to validate the approach on real analog signals in a simple configuration, the ALE allowed us to validate the approach on more complex configurations while dealing with signals already sampled coming from the emulator. The concepts of entities with "Me" and life impulse introduced in 3.2 are also used to play Atari 2600 games. Our proof-of-concept is based on the detection of such entities thanks to image processing: Sobel operator (center image on figure 2) and bounding-box detection (right image on figure 2). It relies on the library. The entity "Me" is found using system identification. Signals such as impulses and pseudorandom sequences BID13 are sent to ALE to first detect the entities affected by these signals, then to build a dynamic model of the "Me". One or a few entities are controllable: they are the "Me". Their shapes can change during the gameplay, such as the paddle in Breakout, thus the possibility to identify different entities as the "Me". These measurements also update the probability functions p(E, F) for entities E and F that the contact between these entities changes the score, in a way similar to the reward function in RL. From these functions p, friends and enemies are inferred, leading to a basic survival strategy outlined in 3.2.The tests are carried out with the settings from BID17: the AI plays for a maximum of 5 minutes. We choose to use the DQN as the reference: the reason being that this publication is one of the most cited in relation to Atari 2600 games, and is the de-facto benchmark to which one must refer. Although we aim to control cyberphysical systems, we needed to validate the versatility of our approach by first testing it on this standard. We fully replicated the setup using code made publicly available by the authors, and we obtained the same as the publication. We were thus able to extract the score for the DQN for a low number of training frames, so as to compare with our approach. TAB2 for a training time of 10 000 frames (less than 3 minutes), which is 20 000 times less frames than the average training standard reported in BID17. While the DQN achieves better with millions of training frames, our AI reaches decent scores with comparatively much fewer frames, as plotted for Breakout on figure 3. A preliminary analysis of what really occurs while our algorithm learns is as follows: during the first thousand frames, the "me" is not yet correctly identified, as the digits, the ball and the paddle move randomly when controlled by the pseudorandom sequence. Once the identification has converged to the only system that the AI directly controls -the paddle, neither the ball nor the digits -, the algorithm looks for friends and enemies. It also detects that the ball is a friend, as it sometimes increases the score (when it breaks a brick). The best scores are in the range of 200 points which, after analysis, corresponds to partially destroyed rows of bricks. It never destroys all the bricks, as it sometimes misses the ball, especially when it looses the "me", for instance when the paddle's size changes or when the paddle disappears according to the basic image processing algorithm. Moreover, we noticed that the movement of the "me" under control of the algorithm sometimes never reaches a steady-state: it oscillates by a few pixels at a frequency of 5.4 Hz. Our explanation from a control perspective is as follows: the "me" can be approximated by a second-order system, and the control strategy is almost equivalent to a proportional controller. This, in the context of Linear-Time-Invariant (LTI) systems, would already explain the oscillations. Moreover, the strong non-linearities present both in the control input and the non-LTI "me" also explain in part these oscillations. The input being quantized to only three values (left, nothing, right), the closed-loop system generates a signal similar to limit-cycles. These oscillations, in turn, are responsible for many of the balls missed by the algorithm. To summarize the on the Atari games, we implemented a few of the concepts presented in 3.1: the notion of entities rather than samples or pixels, and the "me" with the behavior introduced in 3.2. This led to a learning time of a few thousand frames to get a better than human score on Breakout, however it still does not match the best score reached by the DQN after millions of frames on Pong. Continental philosophy lead us to formalize a mathematical concept to control an agent evolving in a world, whether it is simulated or real. The power of this framework was illustrated by the theoretical example of the slot car on unknown circuits. Results from experiments with a real slot car, using real analog signals confirmed our expectations, even though it only used a basic survival approach. Moreover, the same basic survival strategy was applied to two Atari 2600 games and showed the same trend: even though not as skilled as, for instance, DQN-based agents trained with two hundred million frames, our AI reached in less than ten thousand frames scores that DQN met after learning with a few million frames. The next steps are to apply the transposition properties to the Atari games, as we did for the slot car, which should further decrease the learning time when playing a new game. Moreover, going beyond the basic survival strategy will be mandatory to reach higher scores: approaches based on Monte-Carlo Tree Search will be investigated.

Represent the TLDR for finding the corresponding scientific article

Continental-philosophy-inspired approach to learn with few data.

By taking inspiration from linguistics, specifically the Universal Grammar hypothesis, we learn language agnostic universal representations which we can utilize to do zero-shot learning across languages.