In machine learning, a tensor is a way of embedding high-dimensional data into a multi-dimensional array (data type) suitable for artificial neural networks. Tensors are able to represent images, movies, volumes, sounds, and relationships among words and concepts in a unified way. Operations on tensors can be expressed in terms of matrix multiplication and the Kronecker product. With tensor decomposition, tensors can be factorized into smaller tensors, which enable networks with many fewer parameters. The computation of gradients, an important aspect of the backpropagation algorithm, can be performed through tensors in a consistent way that enables unified frameworks such as PyTorch and TensorFlow. Computations are often performed on graphics processing units (GPUs) using CUDA and on dedicated hardware such as Google's Tensor Processing Unit or Nvidia's Tensor core. These developments have greatly accelerated neural network architectures and increased the size and complexity of models that can be trained. Etymology In mathematics and physics, tensors and tensor fields are multi-linear maps, which require the choice of a basis to be represented as multi-dimensional arrays of numbers and functions, respectively. In machine learning, tensors are multi-dimensional arrays of numbers representing embedded data that are not interpreted as multi-linear maps. The term tensor was adopted in machine learning through early work that identified the value of tensor decomposition as a way to reduce the number of learned parameters in optimization problems. History The history of tensors in machine learning builds on the use of vectors and matrices for optimization problems. Tensors appeared in several areas as a form of data reduction. One of the first use of tensors for machine learning appeared in natural language processing, where early networks attempted to learn relationships between concepts in text. A single word can be expressed as a vector via Word2vec. Thus a relationship between two words can be encoded in a matrix. However, for more complex relationships such as subject-object-verb, it is necessary to build higher-dimensional networks. In 2009, the work of Sutsekver introduced Bayesian Clustered Tensor Factorization to model relational concepts while reducing the parameter space. In the area of computer vision, work by Kunihiko Fukushima in 1980 built the concept of a convolutional neural network with two-dimensional images stored in matrices. To work with video it is necessary to embed three or more dimensions in neural networks. In the 2000s, the field of multilinear subspace learning was established to focus specifically on dimensionality