benchmarking 512x512 GEMM
MODIFY
ane/2_compile/simple/gemm.plist
ane/2_compile/simple/gemm.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<!-- dd if=/dev/zero of=/tmp/zero bs=1024 count=16384 -->
+<!-- 1024x512 is the max that gets compiled into one op -->
<plist version="1.0">
<dict>
<key>Networks</key>
<key>image</key>
<dict>
<key>InputChannels</key>
- <integer>1024</integer>
+ <integer>512</integer>
<key>InputHeight</key>
<integer>1</integer>
<key>InputWidth</key>
<key>Name</key>
<string>probs</string>
<key>OutputChannels</key>
- <integer>1024</integer>
+ <integer>512</integer>
<key>OutputType</key>
<string>Float16</string>
<key>Params</key>

MODIFY
ane/README.md
ane/README.md
@@ -28,10 +28,10 @@ The ops have several parts
* TileDMADst -- Output DMA engine
It can work with 8 base addresses for the DMA streams per OP
-* 2x Read
+* 2x Read, both used for things like sum
* 1x Write
* 1x T?
-* 4x Kernel, I assume each is the base for 4 of the engines
+* 4x Kernel, though only the first one seems used
## Normal Flow for ANE Usage

MODIFY
ane/lib/testconv.py
ane/lib/testconv.py
#!/usr/bin/env python3
+import time
from ane import ANE, ANETensor
+def benchmark(ane):
+ tin = ANETensor(512*0x20)
+ tout = ANETensor(512*0x20)
+ dat = open("../ops/gemm.hwx", "rb").read()
+ for k,v in ane.debug(dat[0x4000:0x4300], 16).items():
+ print(k,v)
+ comp = ane.compile(dat)
+
+ st = time.time()
+ for i in range(1000):
+ ret = ane.run(comp, tin, tout)
+ et = time.time()
+ ts = (et-st)
+ ops = 1000*512*512*512*2
+
+ print("%.2f ms, %.2f gigaops/sec" % (ts*1000, ops*1e-9/ts))
+
+
if __name__ == "__main__":
ane = ANE()
@@ -30,12 +49,15 @@ if __name__ == "__main__":
twd[0x21] = 5
twd[0x22] = 5
- twd[0x40] = 6
+ twd[0x40] = 12
print("** before **")
print(tind)
print(toutd)
+ #benchmark(ane)
+ #exit(0)
+
"""
dat = list(open("../ops/sum.hwx", "rb").read())
dat = bytes(dat)

MODIFY
ane/ops/gemm.hwx
ane/ops/gemm.hwx
Binary files a/ane/ops/gemm.hwx and b/ane/ops/gemm.hwx differ

benchmarking 512x512 GEMM