title: ASP.NET Core Integration Test works locally but throws null reference exception when running in production environment
body: <p>I have an ASP.NET Core 2.2 Razor Pages Web App that I have written some integration tests for following <a href="https://docs.microsoft.com/en-us/aspnet/core/test/integration-tests?view=aspnetcore-2.2" rel="noreferrer">the official guide</a>.</p>
  
  <p>I can get the tests to run locally using <code>dotnet test</code> or the test runners built into Visual Studio. However, on the build server (Azure DevOps Hosted 2017 agent) the tests will return a 500 error. I thought it might be related to user secrets as stated on <a href="https://www.hanselman.com/blog/EasierFunctionalAndIntegrationTestingOfASPNETCoreApplications.aspx" rel="noreferrer">Scott Hanselman's guide</a> but I am still getting the same error, even after implementing some of his suggested fixes (I don't believe I need all of them) :</p>
  
  <ul>
  <li>Added <code>builder.AddUserSecrets&lt;Startup&gt;();</code> to Startup.</li>
  <li>Implemented a <code>CustomWebApplicationFactory</code> to set the environment to "Development" - code below has this as "Production" to reproduce the failure.</li>
  </ul>
  
  <p>I also sanity checked against <a href="https://fullstackmark.com/post/20/painless-integration-testing-with-aspnet-core-web-api" rel="noreferrer">this guide</a> which is more controller focused but since I only care about the response codes at this stage it serves my purpose. I have downloaded the verbose logs and they don't shed any light on the issue.</p>
  
  <p>My code is below:</p>
  
  <p>CustomWebApplicationFactory:</p>
  
  <pre><code>using Microsoft.AspNetCore.Authentication.AzureAD.UI;
  using Microsoft.AspNetCore.Authentication.OpenIdConnect;
  using Microsoft.AspNetCore.Hosting;
  using Microsoft.AspNetCore.Mvc.Testing;
  using Microsoft.AspNetCore.TestHost;
  using Microsoft.Extensions.DependencyInjection;
  
  namespace WebPortal.Int.Tests
  {
      /// &lt;summary&gt;
      /// Based on https://fullstackmark.com/post/20/painless-integration-testing-with-aspnet-core-web-api
      /// &lt;/summary&gt;
      public class CustomWebApplicationFactory&lt;TStartup&gt; : WebApplicationFactory&lt;Startup&gt; where TStartup : class
      {
          public CustomWebApplicationFactory() { }
  
          protected override void ConfigureWebHost(IWebHostBuilder builder)
          {
              builder
                  .ConfigureTestServices(
                      services =&gt;
                      {
                          services.Configure(AzureADDefaults.OpenIdScheme, (System.Action&lt;OpenIdConnectOptions&gt;)(o =&gt;
                          {
                              // CookieContainer doesn't allow cookies from other paths
                              o.CorrelationCookie.Path = "/";
                              o.NonceCookie.Path = "/";
                          }));
                      }
                  )
                  .UseEnvironment("Production")
                  .UseStartup&lt;Startup&gt;();
  
          }
      }
  }
  </code></pre>
  
  <p>AuthenticationTests:</p>
  
  <pre><code>using Microsoft.AspNetCore.Mvc.Testing;
  using System;
  using System.Net.Http;
  using System.Threading.Tasks;
  using Xunit;
  
  namespace WebPortal.Int.Tests
  {
      public class AuthenticationTests : IClassFixture&lt;CustomWebApplicationFactory&lt;Startup&gt;&gt;
      {
          private HttpClient _httpClient { get; }
  
          public AuthenticationTests(CustomWebApplicationFactory&lt;Startup&gt; fixture)
          {
              WebApplicationFactoryClientOptions webAppFactoryClientOptions = new WebApplicationFactoryClientOptions
              {
                  // Disallow redirect so that we can check the following: Status code is redirect and redirect url is login url
                  // As per https://docs.microsoft.com/en-us/aspnet/core/test/integration-tests?view=aspnetcore-2.2#test-a-secure-endpoint
                  AllowAutoRedirect = false
              };
  
              _httpClient = fixture.CreateClient(webAppFactoryClientOptions);
          }
  
          [Theory]
          [InlineData("/")]
          [InlineData("/Index")]
          [InlineData("/Error")]
          public async Task Get_PagesNotRequiringAuthenticationWithoutAuthentication_ReturnsSuccessCode(string url)
          {
              // Act
              HttpResponseMessage response = await _httpClient.GetAsync(url);
  
              // Assert
              try
              {
                  response.EnsureSuccessStatusCode();
              }
              catch (HttpRequestException ex)
              {
                  Console.WriteLine(ex.Message, ex.InnerException.Message);
              }
          }
      }
  }
  </code></pre>
  
  <p>Startup:</p>
  
  <pre><code>using Microsoft.AspNetCore.Builder;
  using Microsoft.AspNetCore.Hosting;
  using Microsoft.AspNetCore.Mvc;
  using Microsoft.Extensions.Configuration;
  using Microsoft.Extensions.DependencyInjection;
  using WebPortal.Authentication;
  using WebPortal.Common.ConfigurationOptions;
  using WebPortal.DataAccess;
  using Microsoft.AspNetCore.Authentication;
  using Microsoft.AspNetCore.Authentication.AzureAD.UI;
  using Microsoft.AspNetCore.Authentication.OpenIdConnect;
  using Microsoft.AspNetCore.Http;
  
  namespace WebPortal
  {
      public class Startup
      {
          public Startup(IConfiguration configuration)
          {
              Configuration = configuration;
          }
  
          public IConfiguration Configuration { get; }
  
          // This method gets called by the runtime. Use this method to add services to the container.
          public void ConfigureServices(IServiceCollection services)
          {
              services.Configure&lt;CookiePolicyOptions&gt;(options =&gt;
              {
                  // This lambda determines whether user consent for non-essential cookies is needed for a given request.
                  options.CheckConsentNeeded = context =&gt; false;
                  options.MinimumSameSitePolicy = SameSiteMode.None;
              });
  
              services.AddOptions&lt;PowerBiSettings&gt;()
                  .Bind(Configuration.GetSection("PowerBI"))
                  .ValidateDataAnnotations()
                  .Validate(o =&gt; o.AreSettingsValid());
  
              services.AddOptions&lt;AzureActiveDirectorySettings&gt;()
                  .Bind(Configuration.GetSection("AzureAd"))
                  .ValidateDataAnnotations()
                  .Validate(o =&gt; o.AreSettingsValid());
  
              services.AddAuthentication(AzureADDefaults.AuthenticationScheme)
                  .AddAzureAD(options =&gt; Configuration.Bind("AzureAd", options))
                  .AddCookie();
  
              services.Configure&lt;OpenIdConnectOptions&gt;(AzureADDefaults.OpenIdScheme, options =&gt;
              {
                  options.Authority = options.Authority + "/v2.0/";
                  options.TokenValidationParameters.ValidateIssuer = false;
              });
  
              services.AddTransient&lt;Authentication.IAuthenticationHandler, AuthenticationHandler&gt;();
              services.AddTransient&lt;IReportRepository, ReportRepository&gt;();
  
              services.AddHttpContextAccessor();
  
              services
                  .AddMvc()
                  .SetCompatibilityVersion(CompatibilityVersion.Version_2_2)
                  .AddRazorPagesOptions(options =&gt;
                  {
                      options.Conventions.AuthorizePage("/Reports");
                  });
          }
  
          // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
          public void Configure(IApplicationBuilder app, IHostingEnvironment env)
          {
              var builder = new ConfigurationBuilder();
  
              if (env.IsDevelopment())
              {
                  app.UseDeveloperExceptionPage();
  
                  builder.AddUserSecrets&lt;Startup&gt;();
              }
              else
              {
                  app.UseExceptionHandler("/Error");
                  // The default HSTS value is 30 days. You may want to change this for production scenarios, see https://aka.ms/aspnetcore-hsts.
                  app.UseHsts();
              }
  
              app.UseHttpsRedirection();
              app.UseStaticFiles();
  
              app.UseAuthentication();
  
              app.UseMvc();
          }
      }
  }
  </code></pre>
  
  <p>Error output:</p>
  
  <pre><code>[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.4.1 (64-bit .NET Core 4.6.27317.07)
  [xUnit.net 00:00:01.30]   Discovering: WebPortal.Int.Tests
  [xUnit.net 00:00:01.40]   Discovered:  WebPortal.Int.Tests
  [xUnit.net 00:00:01.41]   Starting:    WebPortal.Int.Tests
  info: Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[0]
      User profile is available. Using 'C:\Users\VssAdministrator\AppData\Local\ASP.NET\DataProtection-Keys' as key repository and Windows DPAPI to encrypt keys at rest.
  info: Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[58]
      Creating key {dd820f09-8139-4d7d-954a-399923660f42} with creation date 2019-03-18 22:13:27Z, activation date 2019-03-18 22:13:27Z, and expiration date 2019-06-16 22:13:27Z.
  info: Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository[39]
      Writing data to file 'C:\Users\VssAdministrator\AppData\Local\ASP.NET\DataProtection-Keys\key-dd820f09-8139-4d7d-954a-399923660f42.xml'.
  info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
      Request starting HTTP/2.0 GET http://localhost/Index  
  warn: Microsoft.AspNetCore.HttpsPolicy.HttpsRedirectionMiddleware[3]
      Failed to determine the https port for redirect.
  fail: Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware[1]
      An unhandled exception has occurred while executing the request.
  System.ArgumentNullException: Value cannot be null.
  Parameter name: uriString
  at System.Uri..ctor(String uriString)
  at Microsoft.AspNetCore.Authentication.AzureAD.UI.OpenIdConnectOptionsConfiguration.Configure(String name, OpenIdConnectOptions options)
  at Microsoft.Extensions.Options.OptionsFactory`1.Create(String name)
  at Microsoft.Extensions.Options.OptionsMonitor`1.&lt;&gt;c__DisplayClass10_0.&lt;Get&gt;b__0()
  at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
  at System.Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
  at System.Lazy`1.CreateValue()
  at Microsoft.Extensions.Options.OptionsCache`1.GetOrAdd(String name, Func`1 createOptions)
  at Microsoft.Extensions.Options.OptionsMonitor`1.Get(String name)
  at Microsoft.AspNetCore.Authentication.AuthenticationHandler`1.InitializeAsync(AuthenticationScheme scheme, HttpContext context)
  at Microsoft.AspNetCore.Authentication.AuthenticationHandlerProvider.GetHandlerAsync(HttpContext context, String authenticationScheme)
  at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
  at Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
  at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.Invoke(HttpContext context)
  fail: Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware[3]
      An exception was thrown attempting to execute the error handler.
  System.ArgumentNullException: Value cannot be null.
  Parameter name: uriString
  at System.Uri..ctor(String uriString)
  at Microsoft.AspNetCore.Authentication.AzureAD.UI.OpenIdConnectOptionsConfiguration.Configure(String name, OpenIdConnectOptions options)
  at Microsoft.Extensions.Options.OptionsFactory`1.Create(String name)
  at Microsoft.Extensions.Options.OptionsMonitor`1.&lt;&gt;c__DisplayClass10_0.&lt;Get&gt;b__0()
  at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
  --- End of stack trace from previous location where exception was thrown ---
  at System.Lazy`1.CreateValue()
  at Microsoft.Extensions.Options.OptionsCache`1.GetOrAdd(String name, Func`1 createOptions)
  at Microsoft.Extensions.Options.OptionsMonitor`1.Get(String name)
  at Microsoft.AspNetCore.Authentication.AuthenticationHandler`1.InitializeAsync(AuthenticationScheme scheme, HttpContext context)
  at Microsoft.AspNetCore.Authentication.AuthenticationHandlerProvider.GetHandlerAsync(HttpContext context, String authenticationScheme)
  at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
  at Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
  at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.Invoke(HttpContext context)
  info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2]
      Request finished in 449.9633ms 500 
  info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
      Request starting HTTP/2.0 GET http://localhost/Error  
  fail: Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware[1]
      An unhandled exception has occurred while executing the request.
  System.ArgumentNullException: Value cannot be null.
  Parameter name: uriString
  at System.Uri..ctor(String uriString)
  at Microsoft.AspNetCore.Authentication.AzureAD.UI.OpenIdConnectOptionsConfiguration.Configure(String name, OpenIdConnectOptions options)
  at Microsoft.Extensions.Options.OptionsFactory`1.Create(String name)
  at Microsoft.Extensions.Options.OptionsMonitor`1.&lt;&gt;c__DisplayClass10_0.&lt;Get&gt;b__0()
  at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
  --- End of stack trace from previous location where exception was thrown ---
  at System.Lazy`1.CreateValue()
  at Microsoft.Extensions.Options.OptionsCache`1.GetOrAdd(String name, Func`1 createOptions)
  at Microsoft.Extensions.Options.OptionsMonitor`1.Get(String name)
  at Microsoft.AspNetCore.Authentication.AuthenticationHandler`1.InitializeAsync(AuthenticationScheme scheme, HttpContext context)
  at Microsoft.AspNetCore.Authentication.AuthenticationHandlerProvider.GetHandlerAsync(HttpContext context, String authenticationScheme)
  at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
  at Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
  at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.Invoke(HttpContext context)
  [xUnit.net 00:00:02.61]     WebPortal.Int.Tests.AuthenticationTests.Get_PagesNotRequiringAuthenticationWithoutAuthentication_ReturnsSuccessCode(url: "/Index") [FAIL]
  [xUnit.net 00:00:02.61]       System.ArgumentNullException : Value cannot be null.
  [xUnit.net 00:00:02.61]       Parameter name: uriString
  [xUnit.net 00:00:02.61]       Stack Trace:
  [xUnit.net 00:00:02.61]            at System.Uri..ctor(String uriString)
  [xUnit.net 00:00:02.61]            at Microsoft.AspNetCore.Authentication.AzureAD.UI.OpenIdConnectOptionsConfiguration.Configure(String name, OpenIdConnectOptions options)
  [xUnit.net 00:00:02.61]            at Microsoft.Extensions.Options.OptionsFactory`1.Create(String name)
  [xUnit.net 00:00:02.61]            at Microsoft.Extensions.Options.OptionsMonitor`1.&lt;&gt;c__DisplayClass10_0.&lt;Get&gt;b__0()
  [xUnit.net 00:00:02.61]            at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
  [xUnit.net 00:00:02.61]         --- End of stack trace from previous location where exception was thrown ---
  [xUnit.net 00:00:02.61]            at System.Lazy`1.CreateValue()
  [xUnit.net 00:00:02.61]            at Microsoft.Extensions.Options.OptionsCache`1.GetOrAdd(String name, Func`1 createOptions)
  [xUnit.net 00:00:02.61]            at Microsoft.Extensions.Options.OptionsMonitor`1.Get(String name)
  [xUnit.net 00:00:02.61]            at Microsoft.AspNetCore.Authentication.AuthenticationHandler`1.InitializeAsync(AuthenticationScheme scheme, HttpContext context)
  [xUnit.net 00:00:02.61]            at Microsoft.AspNetCore.Authentication.AuthenticationHandlerProvider.GetHandlerAsync(HttpContext context, String authenticationScheme)
  [xUnit.net 00:00:02.61]            at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
  [xUnit.net 00:00:02.61]            at Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(HttpContext context)
  [xUnit.net 00:00:02.61]            at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.Invoke(HttpContext context)
  [xUnit.net 00:00:02.61]            at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.Invoke(HttpContext context)
  [xUnit.net 00:00:02.61]            at Microsoft.AspNetCore.TestHost.HttpContextBuilder.&lt;&gt;c__DisplayClass10_0.&lt;&lt;SendAsync&gt;b__0&gt;d.MoveNext()
  [xUnit.net 00:00:02.61]         --- End of stack trace from previous location where exception was thrown ---
  [xUnit.net 00:00:02.61]            at Microsoft.AspNetCore.TestHost.ClientHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
  [xUnit.net 00:00:02.61]            at Microsoft.AspNetCore.Mvc.Testing.Handlers.CookieContainerHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
  [xUnit.net 00:00:02.61]            at Microsoft.AspNetCore.Mvc.Testing.Handlers.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)[xUnit.net 00:00:02.63]     WebPortal.Int.Tests.AuthenticationTests.Get_PagesNotRequiringAuthenticationWithoutAuthentication_ReturnsSuccessCode(url: "/Error") [FAIL]
  [xUnit.net 00:00:02.64]     WebPortal.Int.Tests.AuthenticationTests.Get_PagesNotRequiringAuthenticationWithoutAuthentication_ReturnsSuccessCode(url: "") [FAIL]
  
  [xUnit.net 00:00:02.61]            at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
  [xUnit.net 00:00:02.61]         D:\a\1\s\WebPortal.Int.Tests\AuthenticationTests.cs(24,0): at WebPortal.Int.Tests.AuthenticationTests.Get_PagesNotRequiringAuthenticationWithoutAuthentication_ReturnsSuccessCode(String url)
  [xUnit.net 00:00:02.61]         --- End of stack trace from previous location where exception was thrown ---
  </code></pre>
  
  <p><strong>edit</strong></p>
  
  <p>It isn't very clear to me where/why I am getting a null reference because as far as I can tell my <code>OpenIdConnectOptions</code> configuration is correct (and it works with AAD SSO).</p>