Oseledts, the author observes that Tucker decomposition is "suitable for small dimensions, especially for the three-dimensional case. For large d it is not suitable." Thus tensor-trains can be used to factorize larger tensors in higher dimensions. Tensor graphs The unified data architecture and automatic differentiation of tensors has enabled higher-level designs of machine learning in the form of tensor graphs. This leads to new architectures, such as tensor-graph convolutional networks (TGCN), which identify highly non-linear associations in data, combine multiple relations, and scale gracefully, while remaining robust and performant. These development are impacting all areas of machine learning, such as text mining and clustering, time varying data, and neural networks where the input data is a social graph in and the data changes dynamically. Hardware Tensors provide a unified way to train neural networks for more complex data sets. However, training is expensive to compute on classical CPU hardware. In 2014, Nvidia developed cuDNN, CUDA Deep Neural Network, a library for a set of optimized primitives written in the parallel CUDA language. CUDA and thus cuDNN run on dedicated GPUs that implement unified massive parallelism in hardware. These GPUs were not yet dedicated chips for tensors, but rather existing hardware adapted for parallel computation in machine learning. From 2015-2017, Google invented the Tensor Processing Unit (TPU). TPUs are dedicated, fixed function hardware units that specialize in the matrix multiplications needed for tensor products. Specifically, they implement an array of 65,536 multiply units that can perform a 256x256 matrix sum-product in just one global instruction cycle. Later in 2017, Nvidia released its own Tensor Core with the Volta GPU architecture. Each Tensor Core is a microunit that can perform a 4x4 matrix sum-product. There are eight tensor cores for each shared memory (SM) block. The first GV100 GPU card has 108 SMs resulting in 672 tensor cores. This device accelerated machine learning by 12x over the previous Tesla GPUs. The number of tensor cores scales as the number of cores and SM units continue to grow in each new generation of cards. The development of GPU hardware, combined with the unified architecture of tensor cores, has enabled the training of much larger neural networks. As of 2022, the largest neural network was GPT-3, a language model with over 175 billion learned parameters (network weights) that produces human-like text. The widely popular chatbot ChatGPT is built on top of GPT-3 using supervised