if i was giving systolic arrays thumbs like siskyl and ebert (RIP) i would give them 0
ADD
fpga/TPUNOTES
+SuperScalar is hard
+VLIW is scam but maybe okay if we remove V? (322-bit is only 41 bytes)
+Google TPU is nice. Admit it!!!!! U just jealous U don't have one
+
+TPUv1:
+256x256 = 64K MACs
+
+TPUv2:
+MXU (128x128) supports BF16
+compiler-controlled memory hierarchy
+322-bit VLIW instructions from local memory
+
+TPUv3:
+2x 128x128 MXU
+2x HBM capacity, 900 GB/s
+"midlife kicker"
+32MB on chip
+Same 322-bit wide as TPUv2
+
+TPUv4i:
+7 nm
+4x 128x128 MXU
+Is it inference only just because they removed the interconnect?
+144MB on chip
+Still HBM (SRAM is 20x more energy efficient than DRAM)
+four-dimensional (triple-strided) tensor DMA architecture (DRAM -> SRAM)
+ This lessens the stride burden on the SRAM
+Custom on-chip interconnect (OCI)
+512B native access size (64 uint8)
+The VLIW instruction needed extra fields to handle the four MXUs and the CMEM scratchpad memory, which were easy to add given no need for binary compatibility. The TPUv4i instruction is 25% wider than TPUv3
+https://www.researchgate.net/figure/Architecture-of-a-systolic-array-based-DNN-accelerator-that-serves-as-a-baseline-for_fig1_325861610
+TPUv4i first sums groups of four multiplication results together
+custom four-input floating point adder that eliminates the rounding and normalization logic for the intermediate results.
+
+https://www.hotchips.org/assets/program/conference/day2/HotChips2020_ML_Training_Google_Norrie_Patil.v01.pdf
+322b VLIW bundle
+* 2 scalar slots
+* 4 vector slots (2 for load/store)
+ * 128 wide vector
+* 2 matrix slots (push, pop)
+ * bfloat16 multiply {1,8,7}
+ * float32 accumulation
+* 1 misc slot
+* 6 immediates
+
+

ADD
fpga/matmul.py
+#!/usr/bin/env python3
+import numpy as np
+np.set_printoptions(linewidth=200)
+
+# https://www.youtube.com/watch?v=cmy7LBaWuZ8&t=74s
+
+D = 8
+
+N = D
+A = np.random.rand(N,N)
+B = np.random.rand(N,N)
+#A = np.arange(1,N*N+1).reshape(N,N)
+#B = np.ones((N,N))*2
+#B = np.arange(1,17).reshape(4,4) + 10
+C = A @ B
+
+def reset():
+ global acc, acache, wcache
+ acc = np.zeros((D,D))
+ acache = np.zeros((D,D))
+ wcache = np.zeros((D,D))
+
+def mxu(a):
+ global acc, acache, wcache
+ assert a.shape == (D,)
+ acache[:, 1:] = acache[:, :-1]
+ acache[:, 0] = a
+ ret = np.copy(acc[0])
+ acc[0:-1] = acc[1:]
+ acc[-1] = 0
+ acc += acache * wcache
+ """
+ print("****")
+ print(acache)
+ print(acc)
+ print(ret)
+ """
+ return ret
+
+def apad(a):
+ ret = np.zeros((a.shape[0], a.shape[1]+a.shape[1]-1))
+ for i in range(0, a.shape[0]):
+ ret[i, i:i+a.shape[1]] = a[i]
+ return ret
+
+def unapad(a):
+ ret = np.zeros(((a.shape[0]+1)//2, a.shape[1]))
+ #print(a.shape, ret.shape)
+ for i in range(0, a.shape[1]):
+ ret[:, i] = a[ret.shape[0]-i-1:a.shape[0]-i, i]
+ return ret
+
+
+
+AA = apad(A.T)
+print(AA)
+print(A)
+print(B)
+print(C)
+print("**************************")
+
+reset()
+wcache = B
+out = []
+for n in range(N+N-1):
+ r = mxu(AA[:, -1-n])
+ if n >= N:
+ out.append(r)
+for n in range(N):
+ r = mxu(np.zeros(N))
+ out.append(r)
+ret = unapad(np.array(out)[::-1])
+assert np.allclose(C, ret)
+print(ret)
+

if i was giving systolic arrays thumbs like siskyl and ebert (RIP) i would give them 0