title: End-to-end correlation for API Management, Functions and ServiceBus
body: <p>I have a problem regarding the end-to-end correlation of a process chain on Azure. After seeing this great <a href="http://mlogdberg.com/2019/02/functions/understandhowlogsinai" rel="nofollow noreferrer">article</a> from Mattias Lögdberg, I wanted to achieve a correlation chain as depicted here:
  <a href="https://i.stack.imgur.com/CEKrF.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/CEKrF.png" alt="Desired state"></a>
  By following Mattias steps, I've managed to achieve this, BUT when slightly modifying the chain by removing the second step and directly posting messages into the queue, the chain gets broken.</p>
  
  <p><strong>New Setup</strong></p>
  
  <ol>
  <li><strong>User</strong> calls <strong>API Management</strong></li>
  <li><strong>API Management</strong> calls <strong>ServiceBus</strong> (via REST) </li>
  <li><strong>ServiceBus</strong> triggers <strong>Function</strong></li>
  <li><strong>Function</strong> calls <strong>API Management</strong> (Functions v3)</li>
  </ol>
  
  <p><a href="https://i.stack.imgur.com/G0JtH.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/G0JtH.png" alt="Call chain"></a></p>
  
  <p>In the first call, the <em>Diagnostic-Id</em> header is set to the w3c-traceparent <a href="https://www.w3.org/TR/trace-context/#examples-of-http-traceparent-headers" rel="nofollow noreferrer">standard</a>, so the ServiceBus message can correlate. This happens by executing following policy in the inbound section:</p>
  
  <pre><code>&lt;set-header name="Diagnostic-Id" exists-action="override"&gt;
      &lt;value&gt;@($"00-{context.RequestId.ToString("N")}-{String.Format("{0:x16}", new Random().Next(0x1000000))}-00")&lt;/value&gt;
  &lt;/set-header&gt;
  </code></pre>
  
  <p>I've tried out several variations of this header, but this one at least got me to following flow:
  <a href="https://i.stack.imgur.com/BJNkt.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/BJNkt.png" alt="Current flow"></a></p>
  
  <p>Is there something I am missing to "summarize" the logs properly?
  All operations have the same operation_Id and different operation_parentId's - exactly as in the process with the function.</p>
  
  <p><strong>+++ Update 19/11 +++</strong></p>
  
  <p>A new finding I made is, that the correlation depends on the Request-Id header. But as the ServiceBus expects a "Diagnostic-Id", the APIM generated Request-Id is ignored. Nevertheless, the best workaround to "catch em all" I found is as follows:</p>
  
  <pre><code>&lt;set-header name="Diagnostic-Id" exists-action="override"&gt;
      &lt;value&gt;@(context.Request.Headers.GetValueOrDefault("Request-Id", context.RequestId.ToString("N")}&lt;/value&gt;
  &lt;/set-header&gt;
  </code></pre>
  
  <p>This makes sure, that the caller id is taken into respect. If there's no caller Request-Id, the apim generated request id is defaulted. It's still not the view with the "Time Spent in Queue"-feature, but at least it's a comprehensible solution.</p>