title: Understanding the impact of lfence on a loop with two long dependency chains, for increasing lengths
body: <p>I was playing with the code in <a href="https://stackoverflow.com/a/50496379/5801661">this answer</a>, slightly modifying it:</p>
  
  <pre><code>BITS 64
  
  GLOBAL _start
  
  SECTION .text
  
  _start:
   mov ecx, 1000000
  
  .loop:
  
   ;T is a symbol defined with the CLI (-DT=...)
  
   TIMES T imul eax, eax
   lfence
   TIMES T imul edx, edx
  
  
   dec ecx
  jnz .loop
  
   mov eax, 60           ;sys_exit
   xor edi, edi
   syscall
  </code></pre>
  
  <p>Without the <code>lfence</code> I the results I get are consistent with the static analysis in that answer.  </p>
  
  <p>When I introduce a <strong>single</strong> <code>lfence</code> I'd expect the CPU to execute the <code>imul edx, edx</code> sequence of the <em>k-th</em> iteration in parallel with the <code>imul eax, eax</code> sequence of the next (<em>k+1-th</em>) iteration.<br>
  Something like this (calling <em>A</em> the <code>imul eax, eax</code> sequence and <em>D</em> the <code>imul edx, edx</code> one):</p>
  
  <pre><code>|
  | A
  | D A
  | D A
  | D A
  | ...
  | D A
  | D
  |
  V time
  </code></pre>
  
  <p>Taking more or less the same number of cycles but for one unpaired parallel execution.  </p>
  
  <p>When I measure the number of cycles, for the original and modified version, with <code>taskset -c 2 ocperf.py stat -r 5 -e cycles:u '-x ' ./main-$T</code> for <code>T</code> in the range below I get</p>
  
  <pre><code>T   Cycles:u    Cycles:u    Delta
      lfence      no lfence
  
  10  42047564    30039060    12008504
  15  58561018    45058832    13502186
  20  75096403    60078056    15018347
  25  91397069    75116661    16280408
  30  108032041   90103844    17928197
  35  124663013   105155678   19507335
  40  140145764   120146110   19999654
  45  156721111   135158434   21562677
  50  172001996   150181473   21820523
  55  191229173   165196260   26032913
  60  221881438   180170249   41711189
  65  250983063   195306576   55676487
  70  281102683   210255704   70846979
  75  312319626   225314892   87004734
  80  339836648   240320162   99516486
  85  372344426   255358484   116985942
  90  401630332   270320076   131310256
  95  431465386   285955731   145509655
  100 460786274   305050719   155735555
  </code></pre>
  
  <p><a href="https://i.stack.imgur.com/3VYZq.png" rel="noreferrer"><img src="https://i.stack.imgur.com/3VYZq.png" alt="Plotted data of above"></a></p>
  
  <p>How can the values of <code>Cycles:u lfence</code> be explained?<br>
  I would have expected them to be similar to those of <code>Cycles:u no lfence</code> since a single <code>lfence</code> should prevent only the first iteration from being executed in parallel for the two blocks.<br>
  I don't think it's due to the <code>lfence</code> overhead as I believe that should be constant for all <code>T</code>s.  </p>
  
  <p>I'd like to fix what's wrong with my <em>forma mentis</em> when dealing with the static analysis of code. </p>
  
  <hr>
  
  <p><a href="https://github.com/margaretbloom/interleaved-cycle/tree/master" rel="noreferrer">Supporting repository with source files</a>.</p>