title: Application Insights on Azure Cloud Service - Generating high CPU usage
body: <p>I added Application Insights into an existing Azure Cloud Services application consisted of 1 WebRole and 1 Worker Role. I did it by following the instructions from the <a href="https://docs.microsoft.com/en-us/azure/application-insights/app-insights-cloudservices#sdk" rel="nofollow noreferrer">docs here</a> which is basically installing the NuGet Packages and setting the <code>InstrumentationKey</code> manually. </p>
  
  <p>I didn't customize nor change anything on the <code>ApplicationInsights.config</code> file that was auto-generated when I installed the SDK. This is where all the collectors and initializers are defined. I don't configure anything else regarding App Insights in my c# code.</p>
  
  <p><strong>The problem:</strong></p>
  
  <p>When we deployed the changes to production, we started to notice some really high response times and high CPU load/spikes that definitely wasn't there before.</p>
  
  <p><strong>What we did so far to narrow the issue:</strong></p>
  
  <p>We set up a small stress test in a dev environment. We deployed a previous version of the API without Application Insights enabled and ran the tests. Although the CPU was under high load (of course due to the stress tests) the response time was on average 1 sec.</p>
  
  <p>Then, we did the same but with the code version that contains the Application Insight enabled. CPU was high, the same, but the response time was much more slow, going up to 5s.</p>
  
  <p>We are more or less lost on what would be the culprit here. We had also installed the "Application Insight Agent", or Profiler as explained in <a href="https://github.com/Microsoft/ApplicationInsights-dotnet/issues/427" rel="nofollow noreferrer">this issue</a>  in the Cloud Service to be able to see the SQL Queries executed in the Dependencies. We thought that it would be the cause.. but even disabling it didn't change anything in Performance. So, we discarded the Profiler already.</p>
  
  <p>We just started with App Insights so it's kind of hard to guess what's wrong. Our best bet is that in the <code>ApplicationInsights.config</code> we maybe need to remove some Modules or stop collecting some data but at this point is just guessing work.</p>
  
  <p><strong>Something related that may/may not be relevant</strong></p>
  
  <p>Together with App Insights, I also introduced <code>Serilog</code> and the <code>Serilog.Sinks.ApplicationInsights</code> that basically just send my logs to AppInsights as Traces. But I really don't think that has anything to do with it.</p>
  
  <p><strong>Edit:</strong> Adding more details and package versions</p>
  
  <p><strong>Web Role:</strong> ASP.NET Web API 2 An API that basically makes calls to SQL Azure Database</p>
  
  <p><strong>NET Framework:</strong> 4.6.2</p>
  
  <p><strong>Hosted on:</strong> Azure Cloud Services osFamily=5</p>
  
  <p><strong>VM Size</strong>: Two instances of A1 v2 for the Web Role.</p>
  
  <p><strong>Requests:</strong> The version without AI has an average response time of 1 sec. The version with AI has an average response of 3.5. Some requests even take 5 seconds.</p>
  
  <p>Basically, when AI is enabled, I see <strong>spikes in CPU usage</strong>. See image below from the Cloud Service Metrics View:
  <a href="https://i.stack.imgur.com/KnJW7.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/KnJW7.png" alt="enter image description here"></a></p>
  
  <p><strong>AI Packages/Versions:</strong></p>
  
  <pre><code>&lt;package id="Microsoft.ApplicationInsights" version="2.7.2" targetFramework="net462" /&gt;
    &lt;package id="Microsoft.ApplicationInsights.Agent.Intercept" version="2.4.0" targetFramework="net462" /&gt;
    &lt;package id="Microsoft.ApplicationInsights.DependencyCollector" version="2.7.2" targetFramework="net462" /&gt;
    &lt;package id="Microsoft.ApplicationInsights.PerfCounterCollector" version="2.7.2" targetFramework="net462" /&gt;
    &lt;package id="Microsoft.ApplicationInsights.Web" version="2.7.2" targetFramework="net462" /&gt;
    &lt;package id="Microsoft.ApplicationInsights.WindowsServer" version="2.7.2" targetFramework="net462" /&gt;
    &lt;package id="Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel" version="2.7.2" targetFramework="net462" /&gt;
    &lt;package id="Serilog" version="2.7.1" targetFramework="net462" /&gt;
    &lt;package id="Serilog.Sinks.ApplicationInsights" version="2.6.0" targetFramework="net462" /&gt;
  </code></pre>
  
  <p>Here is the content of the <code>ApplicationInsights.config</code> file on the WebApi project:</p>
  
  <pre><code>&lt;?xml version="1.0" encoding="utf-8"?&gt;
  &lt;ApplicationInsights xmlns="http://schemas.microsoft.com/ApplicationInsights/2013/Settings"&gt;
  
    &lt;TelemetryInitializers&gt;
      &lt;Add Type="Microsoft.ApplicationInsights.DependencyCollector.HttpDependenciesParsingTelemetryInitializer, Microsoft.AI.DependencyCollector"/&gt;
      &lt;Add Type="Microsoft.ApplicationInsights.WindowsServer.AzureRoleEnvironmentTelemetryInitializer, Microsoft.AI.WindowsServer"/&gt;
      &lt;Add Type="Microsoft.ApplicationInsights.WindowsServer.AzureWebAppRoleEnvironmentTelemetryInitializer, Microsoft.AI.WindowsServer"/&gt;
      &lt;Add Type="Microsoft.ApplicationInsights.WindowsServer.BuildInfoConfigComponentVersionTelemetryInitializer, Microsoft.AI.WindowsServer"/&gt;
      &lt;Add Type="Microsoft.ApplicationInsights.Web.WebTestTelemetryInitializer, Microsoft.AI.Web"/&gt;
      &lt;Add Type="Microsoft.ApplicationInsights.Web.SyntheticUserAgentTelemetryInitializer, Microsoft.AI.Web"&gt;
        &lt;Filters&gt;search|spider|crawl|Bot|Monitor|AlwaysOn&lt;/Filters&gt;
      &lt;/Add&gt;
      &lt;Add Type="Microsoft.ApplicationInsights.Web.ClientIpHeaderTelemetryInitializer, Microsoft.AI.Web"/&gt;
      &lt;Add Type="Microsoft.ApplicationInsights.Web.OperationNameTelemetryInitializer, Microsoft.AI.Web"/&gt;
      &lt;Add Type="Microsoft.ApplicationInsights.Web.OperationCorrelationTelemetryInitializer, Microsoft.AI.Web"/&gt;
      &lt;Add Type="Microsoft.ApplicationInsights.Web.UserTelemetryInitializer, Microsoft.AI.Web"/&gt;
      &lt;Add Type="Microsoft.ApplicationInsights.Web.AuthenticatedUserIdTelemetryInitializer, Microsoft.AI.Web"/&gt;
      &lt;Add Type="Microsoft.ApplicationInsights.Web.AccountIdTelemetryInitializer, Microsoft.AI.Web"/&gt;
      &lt;Add Type="Microsoft.ApplicationInsights.Web.SessionTelemetryInitializer, Microsoft.AI.Web"/&gt;
    &lt;/TelemetryInitializers&gt;
    &lt;TelemetryModules&gt;
      &lt;Add Type="Microsoft.ApplicationInsights.DependencyCollector.DependencyTrackingTelemetryModule, Microsoft.AI.DependencyCollector"&gt;
        &lt;ExcludeComponentCorrelationHttpHeadersOnDomains&gt;
          &lt;Add&gt;core.windows.net&lt;/Add&gt;
          &lt;Add&gt;core.chinacloudapi.cn&lt;/Add&gt;
          &lt;Add&gt;core.cloudapi.de&lt;/Add&gt;
          &lt;Add&gt;core.usgovcloudapi.net&lt;/Add&gt;
          &lt;Add&gt;localhost&lt;/Add&gt;
          &lt;Add&gt;127.0.0.1&lt;/Add&gt;
        &lt;/ExcludeComponentCorrelationHttpHeadersOnDomains&gt;
        &lt;IncludeDiagnosticSourceActivities&gt;
          &lt;Add&gt;Microsoft.Azure.EventHubs&lt;/Add&gt;
          &lt;Add&gt;Microsoft.Azure.ServiceBus&lt;/Add&gt;
        &lt;/IncludeDiagnosticSourceActivities&gt;
      &lt;/Add&gt;
      &lt;Add Type="Microsoft.ApplicationInsights.Extensibility.PerfCounterCollector.PerformanceCollectorModule, Microsoft.AI.PerfCounterCollector"&gt;
      &lt;/Add&gt;
      &lt;Add Type="Microsoft.ApplicationInsights.Extensibility.PerfCounterCollector.QuickPulse.QuickPulseTelemetryModule, Microsoft.AI.PerfCounterCollector"/&gt;
      &lt;Add Type="Microsoft.ApplicationInsights.WindowsServer.AppServicesHeartbeatTelemetryModule, Microsoft.AI.WindowsServer"/&gt;
      &lt;Add Type="Microsoft.ApplicationInsights.WindowsServer.AzureInstanceMetadataTelemetryModule, Microsoft.AI.WindowsServer"&gt;
  
      &lt;/Add&gt;
      &lt;Add Type="Microsoft.ApplicationInsights.WindowsServer.DeveloperModeWithDebuggerAttachedTelemetryModule, Microsoft.AI.WindowsServer"/&gt;
      &lt;Add Type="Microsoft.ApplicationInsights.WindowsServer.UnhandledExceptionTelemetryModule, Microsoft.AI.WindowsServer"/&gt;
      &lt;Add Type="Microsoft.ApplicationInsights.WindowsServer.UnobservedExceptionTelemetryModule, Microsoft.AI.WindowsServer"&gt;
      &lt;/Add&gt;
      &lt;Add Type="Microsoft.ApplicationInsights.Web.RequestTrackingTelemetryModule, Microsoft.AI.Web"&gt;
        &lt;Handlers&gt;
          &lt;Add&gt;Microsoft.VisualStudio.Web.PageInspector.Runtime.Tracing.RequestDataHttpHandler&lt;/Add&gt;
          &lt;Add&gt;System.Web.StaticFileHandler&lt;/Add&gt;
          &lt;Add&gt;System.Web.Handlers.AssemblyResourceLoader&lt;/Add&gt;
          &lt;Add&gt;System.Web.Optimization.BundleHandler&lt;/Add&gt;
          &lt;Add&gt;System.Web.Script.Services.ScriptHandlerFactory&lt;/Add&gt;
          &lt;Add&gt;System.Web.Handlers.TraceHandler&lt;/Add&gt;
          &lt;Add&gt;System.Web.Services.Discovery.DiscoveryRequestHandler&lt;/Add&gt;
          &lt;Add&gt;System.Web.HttpDebugHandler&lt;/Add&gt;
        &lt;/Handlers&gt;
      &lt;/Add&gt;
      &lt;Add Type="Microsoft.ApplicationInsights.Web.ExceptionTrackingTelemetryModule, Microsoft.AI.Web"/&gt;
      &lt;Add Type="Microsoft.ApplicationInsights.Web.AspNetDiagnosticTelemetryModule, Microsoft.AI.Web"/&gt;
    &lt;/TelemetryModules&gt;
    &lt;ApplicationIdProvider Type="Microsoft.ApplicationInsights.Extensibility.Implementation.ApplicationId.ApplicationInsightsApplicationIdProvider, Microsoft.ApplicationInsights"/&gt;
    &lt;TelemetryProcessors&gt;
      &lt;Add Type="Microsoft.ApplicationInsights.Extensibility.PerfCounterCollector.QuickPulse.QuickPulseTelemetryProcessor, Microsoft.AI.PerfCounterCollector"/&gt;
      &lt;Add Type="Microsoft.ApplicationInsights.Extensibility.AutocollectedMetricsExtractor, Microsoft.ApplicationInsights"/&gt;
      &lt;Add Type="Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel.AdaptiveSamplingTelemetryProcessor, Microsoft.AI.ServerTelemetryChannel"&gt;
        &lt;MaxTelemetryItemsPerSecond&gt;5&lt;/MaxTelemetryItemsPerSecond&gt;
        &lt;ExcludedTypes&gt;Event&lt;/ExcludedTypes&gt;
      &lt;/Add&gt;
      &lt;Add Type="Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel.AdaptiveSamplingTelemetryProcessor, Microsoft.AI.ServerTelemetryChannel"&gt;
        &lt;MaxTelemetryItemsPerSecond&gt;5&lt;/MaxTelemetryItemsPerSecond&gt;
        &lt;IncludedTypes&gt;Event&lt;/IncludedTypes&gt;
      &lt;/Add&gt;
    &lt;/TelemetryProcessors&gt;
    &lt;TelemetryChannel Type="Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel.ServerTelemetryChannel, Microsoft.AI.ServerTelemetryChannel"/&gt;
  &lt;/ApplicationInsights&gt;
  </code></pre>