title: PERF_TYPE_HARDWARE and PERF_TYPE_HW_CACHE concurrent monitoring
body: <p>I'm working on a custom implementation on top of <code>perf_event_open</code> syscall.</p>
  
  <p>The implementation aims to support various of <code>PERF_TYPE_HARDWARE</code>, <code>PERF_TYPE_SOFTWARE</code> and <code>PERF_TYPE_HW_CACHE</code> events for <strong>specific threads on any core</strong>.</p>
  
  <p>In <em>Intel® 64 and IA-32 Architectures Software Developer’s Manual vol 3B</em> I see the following for my testing CPU (Kaby Lake):</p>
  
  <p><a href="https://i.stack.imgur.com/FDrAE.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/FDrAE.png" alt="enter image description here"></a></p>
  
  <p>To my understanding so far, one can monitor (theoretically) unlimited <code>PERF_TYPE_SOFTWARE</code> events concurrently but limited (without multiplexing) <code>PERF_TYPE_HARDWARE</code> and <code>PERF_TYPE_HW_CACHE</code> events concurrently since each event is measured by the limited (as can be seen on the manual above) number of counters of the CPU's PMU.</p>
  
  <p>So for a quad-core Kaby Lake CPU with HyperThreading enabled I assume that up to 4 <code>PERF_TYPE_HARDWARE</code>/<code>PERF_TYPE_HW_CACHE</code> events can be monitored concurrently (or up to 8 if only 4 threads are used).</p>
  
  <p>Experimenting with the above assumptions I found out that while I can successfully monitor up to 4 <code>PERF_TYPE_HARDWARE</code> events (for 8 threads) this is not the case for <code>PERF_TYPE_HW_CACHE</code> events where only up to 2 events can be monitored concurrently!</p>
  
  <p>I also tried to use only 4 threads but the upper limit of concurrently monitored 'PERF_TYPE_HARDWARE' events remains 4. The same is happening with HyperThreading <strong>disabled</strong>!</p>
  
  <p>One could ask: why do you need to avoid multiplexing. First of all, the implementation needs to be as much accurate as possible by avoiding the potential blind spots of multiplexing and secondly when the "upper limit" is exceeded all event values are 0...</p>
  
  <p>The <code>PERF_TYPE_HW_CACHE</code> events I'm targeting are:</p>
  
  <pre><code>CACHE_LLC_READ(PERF_HW_CACHE_TYPE_ID.PERF_COUNT_HW_CACHE_LL.value  | PERF_HW_CACHE_OP_ID.PERF_COUNT_HW_CACHE_OP_READ.value &lt;&lt; 8 | PERF_HW_CACHE_OP_RESULT_ID.PERF_COUNT_HW_CACHE_RESULT_ACCESS.value &lt;&lt; 16),
  CACHE_LLC_WRITE(PERF_HW_CACHE_TYPE_ID.PERF_COUNT_HW_CACHE_LL.value  | PERF_HW_CACHE_OP_ID.PERF_COUNT_HW_CACHE_OP_WRITE.value &lt;&lt; 8 | PERF_HW_CACHE_OP_RESULT_ID.PERF_COUNT_HW_CACHE_RESULT_ACCESS.value &lt;&lt; 16),
  CACHE_LLC_READ_MISS(PERF_HW_CACHE_TYPE_ID.PERF_COUNT_HW_CACHE_LL.value  | PERF_HW_CACHE_OP_ID.PERF_COUNT_HW_CACHE_OP_READ.value &lt;&lt; 8 | PERF_HW_CACHE_OP_RESULT_ID.PERF_COUNT_HW_CACHE_RESULT_MISS.value &lt;&lt; 16),
  CACHE_LLC_WRITE_MISS(PERF_HW_CACHE_TYPE_ID.PERF_COUNT_HW_CACHE_LL.value  | PERF_HW_CACHE_OP_ID.PERF_COUNT_HW_CACHE_OP_WRITE.value &lt;&lt; 8 | PERF_HW_CACHE_OP_RESULT_ID.PERF_COUNT_HW_CACHE_RESULT_MISS.value &lt;&lt; 16),
  </code></pre>
  
  <p>all are implemented with the provided formula:</p>
  
  <pre><code>(perf_hw_cache_id) | (perf_hw_cache_op_id &lt;&lt; 8) |
  (perf_hw_cache_op_result_id &lt;&lt; 16)
  </code></pre>
  
  <p>and are manipulated as a group (the first is the group leader etc).</p>
  
  <p>So, my questions are the following:</p>
  
  <ol>
  <li>Which counters of the PMU are used for <code>PERF_TYPE_HARDWARE</code> and which for <code>PERF_TYPE_HW_CACHE</code> events and where can I find this information?</li>
  <li>What is the difference between the <code>PERF_TYPE_HARDWARE</code> pre-defined events (such as <code>PERF_COUNT_HW_CACHE_MISSES</code>) and the <code>PERF_TYPE_HW_CACHE</code> events?</li>
  <li>Any suggestions on how to monitor without multiplexing all listed <code>PERF_TYPE_HW_CACHE</code> events?</li>
  <li>Any suggestions on how to monitor without multiplexing up to 8 <code>PERF_TYPE_HARDWARE</code> or/and <code>PERF_TYPE_HW_CACHE</code> events?</li>
  </ol>
  
  <p>Thanks in advance!</p>