more tpu notes
MODIFY
fpga/TPUNOTES
fpga/TPUNOTES
@@ -34,8 +34,9 @@ custom four-input floating point adder that eliminates the rounding and normaliz
https://www.hotchips.org/assets/program/conference/day2/HotChips2020_ML_Training_Google_Norrie_Patil.v01.pdf
322b VLIW bundle
+
* 2 scalar slots
-* 4 vector slots (2 for load/store)
+* 4 vector slots (2 for load/store, 2 for ALU?)
* 128 wide vector
* 2 matrix slots (push, pop)
* bfloat16 multiply {1,8,7}
@@ -43,4 +44,26 @@ https://www.hotchips.org/assets/program/conference/day2/HotChips2020_ML_Training
* 1 misc slot
* 6 immediates
+Scalar Unit:
+4Ki x 32b Scalar Memory
+32 x 32b Scalar Reg File
+2x ALUs
+
+128x Vector Unit:
+ 32Ki x 32b Vector (Lane) Memory (8 ports)
+ 8x:
+ 32 x 32b Vector (Lane) Reg File
+ 2x ALUs
+ Connections to matrix unix
+2048 Vector ALUs
+
+2x Matrix Multiply Unit
+ 128 x 128 systolic array
+ Streaming LHS and results
+ Stationary RHS (w/ optional transpose)
+
+Transpose / Permute Unit
+ Transpose, Reduction, Permutation
+ allow reshuffling of data across vector lanes
+

more tpu notes