reduction of data sets by using tensors. Tensors are related in this way to principal component analysis. Tensors came to be used directly in convolutional neural networks to represent images and video. Panagakis et al. survey the use of tensors in computer vision. Definition Let be a field such as the real numbers or the complex numbers . A tensor is an array over : Here, and are positive integers, and is the number of dimensions, number of ways, or mode of the tensor. Various data types are embedded into tensors. For example, a grayscale image with resolution is embedded in a mode-2 tensor as A color image with 3 channels for RGB might be embedded in a mode-3 tensor with three elements in an additional dimension: In natural language processing, a word might be expressed as a vector via the Word2vec algorithm. Thus becomes a mode-1 tensor The embedding of subject-object-verb semantics requires embedding relationships among three words. Because a word is itself a vector, subject-object-verb semantics could be expressed using mode-3 tensors In practice the neural network designer is primarily concerned with the specification of embeddings, the connection of tensor layers, and the operations performed on them in a network. Modern machine learning frameworks manage the optimization, tensor factorization and backpropagation automatically. As unit values Tensors may be used as the unit values of neural networks which extend the concept of scalar, vector and matrix values to multiple dimensions. The output value of single layer unit is the sum-product of its input units and the connection weights filtered through the activation function : where If each output element of is a scalar, then we have the classical definition of an artificial neural network. By replacing each unit component with a tensor, the network is able to express higher dimensional data such as images or videos: This use of tensors to replace unit values is common in convolutional neural networks where each unit might be an image processed through multiple layers. By embedding the data in tensors such network structures enable learning of complex data types. In fully connected layers Tensors may also be used to compute the layers of a fully connected neural network, where the tensor is applied to the entire layer instead of individual unit values. The output value of single layer unit is the sum-product of its input units and the connection weights filtered through