title: Why is Numpy with Ryzen Threadripper so much slower than Xeon?
body: <p>I know that Numpy can use different backends like OpenBLAS or MKL. I have also read that MKL is heavily optimized for Intel, so usually people suggest to use OpenBLAS on AMD, right?</p>
  <p>I use the following test code:</p>
  <pre class="lang-py prettyprint-override"><code>import numpy as np
  
  def testfunc(x):
      np.random.seed(x)
      X = np.random.randn(2000, 4000)
      np.linalg.eigh(X @ X.T)
  
  %timeit testfunc(0)
  </code></pre>
  <p>I have tested this code using different CPUs:</p>
  <ul>
  <li>On <strong>Intel Xeon E5-1650 v3</strong>, this code performs in <strong>0.7s</strong> using <strong>6 out of 12 cores</strong>.</li>
  <li>On <strong>AMD Ryzen 5 2600</strong>, this code performs in <strong>1.45s</strong> using <strong>all 12 cores</strong>.</li>
  <li>On <strong>AMD Ryzen Threadripper 3970X</strong>, this code performs in <strong>1.55s</strong> using <strong>all 64 cores</strong>.</li>
  </ul>
  <p>I am using the same Conda environment on all three systems. According to <code>np.show_config()</code>, the Intel system uses the MKL backend for Numpy (<code>libraries = ['mkl_rt', 'pthread']</code>), whereas the AMD systems use OpenBLAS (<code>libraries = ['openblas', 'openblas']</code>). The CPU core usage was determined by observing <code>top</code> in a Linux shell:</p>
  <ul>
  <li>For the <strong>Intel Xeon E5-1650 v3</strong> CPU (6 physical cores), it shows 12 cores (6 idling).</li>
  <li>For the <strong>AMD Ryzen 5 2600</strong> CPU (6 physical cores), it shows 12 cores (none idling).</li>
  <li>For the <strong>AMD Ryzen Threadripper 3970X</strong> CPU (32 physical cores), it shows 64 cores (none idling).</li>
  </ul>
  <p>The above observations give rise to the following questions:</p>
  <ol>
  <li>Is that normal, that linear algebra on up-to-date AMD CPUs using OpenBLAS is <em>that much slower</em> than on a six-year-old Intel Xeon? <em>(also addressed in Update 3)</em></li>
  <li>Judging by the observations of the CPU load, it looks like Numpy utilizes the multi-core environment in all three cases. How can it be that the Threadripper is even slower than the Ryzen 5, even though it has almost six times as many physical cores? <em>(also see Update 3)</em></li>
  <li>Is there anything that can be done to speed up the computations on the Threadripper? <em>(partially answered in Update 2)</em></li>
  </ol>
  <hr />
  <p><strong>Update 1:</strong> The OpenBLAS version is 0.3.6. I read somewhere, that upgrading to a newer version might help, however, with OpenBLAS updated to 0.3.10, the performance for <code>testfunc</code> is still 1.55s on AMD Ryzen Threadripper 3970X.</p>
  <hr />
  <p><strong>Update 2:</strong> Using the MKL backend for Numpy in conjunction with the environment variable <code>MKL_DEBUG_CPU_TYPE=5</code> (as described <a href="https://www.pugetsystems.com/labs/hpc/How-To-Use-MKL-with-AMD-Ryzen-and-Threadripper-CPU-s-Effectively-for-Python-Numpy-And-Other-Applications-1637/#TestsystemsAMDThreadripper3960x,Ryzen3900XandIntelXeon2175W" rel="noreferrer">here</a>) reduces the run time for <code>testfunc</code> on AMD Ryzen Threadripper 3970X to only 0.52s, which is actually more or less satisfying. FTR, setting this variable via <code>~/.profile</code> did not work for me on Ubuntu 20.04. Also, setting the variable from within Jupyter did not work. So instead I put it into <code>~/.bashrc</code> which works now. Anyways, performing 35% faster than an old Intel Xeon, is this all we get, or can we get more out of it?</p>
  <hr />
  <p><strong>Update 3:</strong> I play around with the number of threads used by MKL/OpenBLAS:</p>
  <p><a href="https://i.stack.imgur.com/uVJV8.png" rel="noreferrer"><img src="https://i.stack.imgur.com/uVJV8.png" alt="run time by the number of threads and library" /></a></p>
  <p>The run times are reported in seconds. The best result of each column is underlined. I used OpenBLAS 0.3.6 for this test. The conclusions from this test:</p>
  <ul>
  <li><strong>The single-core performance of the Threadripper using OpenBLAS is a bit better</strong> than the single-core performance of the Xeon (11% faster), however, its single-core performance is even better when using MKL (34% faster).</li>
  <li><strong>The multi-core performance of the Threadripper using OpenBLAS is ridiculously worse</strong> than the multi-core performance of the Xeon. What is going on here?</li>
  <li><strong>The Threadripper performs overall better than the Xeon</strong>, when MKL is used (26% to 38% faster than Xeon). The overall best performance is achieved by the Threadripper using 16 threads and MKL (36% faster than Xeon).</li>
  </ul>
  <hr />
  <p><strong>Update 4:</strong> Just for clarification. No, I do not think that (a) <a href="https://www.pugetsystems.com/labs/hpc/How-To-Use-MKL-with-AMD-Ryzen-and-Threadripper-CPU-s-Effectively-for-Python-Numpy-And-Other-Applications-1637/#TestsystemsAMDThreadripper3960x,Ryzen3900XandIntelXeon2175W" rel="noreferrer">this</a> or (b) <a href="https://stackoverflow.com/questions/63174453/when-you-have-an-amd-cpu-can-you-speed-up-code-that-uses-the-intel-mkl/63174454#63174454">that</a> answers this question. (a) suggests that <em>&quot;OpenBLAS does nearly as well as MKL&quot;</em>, which is a strong contradiction to the numbers I observed. According to my numbers, OpenBLAS performs ridiculously worse than MKL. The question is why. (a) and (b) both suggest using <code>MKL_DEBUG_CPU_TYPE=5</code> in conjunction with MKL to achieve maximum performance. This might be right, but it does neither explain <em>why</em> OpenBLAS is <em>that dead</em> slow. Neither it explains, why even with MKL and <code>MKL_DEBUG_CPU_TYPE=5</code> the <strong>32-core Threadripper is only 36% faster than the six-year-old 6-core Xeon</strong>.</p>