sequence processing with convnets sequence processing with convnets in you learned about convolutional neural networks (convnetsand how they perform particularly well on computer vision problemsdue to their ability to operate convolutionallyextracting features from local input patches and allowing for representation modularity and data efficiency the same properties that make convnets excel at computer vision also make them highly relevant to sequence processing time can be treated as spatial dimensionlike the height or width of image such convnets can be competitive with rnns on certain sequence-processing problemsusually at considerably cheaper computational cost recently convnetstypically used with dilated kernelshave been used with great success for audio generation and machine translation in addition to these specific successesit has long been known that small convnets can offer fast alternative to rnns for simple tasks such as text classification and timeseries forecasting understanding convolution for sequence data the convolution layers introduced previously were convolutionsextracting patches from image tensors and applying an identical transformation to every patch in the same wayyou can use convolutionsextracting local patches (subsequencesfrom sequences (see figure window of size input features input time extracted patch dot product with weights output output features figure how convolut ion workseach output timest ep is obtained from emporal patch in the input sequence such convolution layers can recognize local patterns in sequence because the same input transformation is performed on every patcha pattern learned at certain position in sentence can later be recognized at different positionmaking convnets translation invariant (for temporal translationsfor instancea convnet processing sequences of characters using convolution windows of size should be able to learn words or word fragments of length or lessand it should be able to recognize licensed to