title: Null reference exception when generating a url with UrlHelper.Action method
body: <p>For some reason, when certain bots visit the site, generating a url with the <code>UrlHelper.Action</code> method raises a null exception from <code>System.Web.HttpServerVarsCollection.Get</code>. I've done some debugging and the call stack originiates with an attempt to get the 'HTTP_X_ORIGINAL_URL' from the <code>HttpContextBase.Request.ServerVariables</code> collection.</p>
  
  <p>If I visit the same address directly from a browser - no problem. The page is server and no error is logged. It only seems to occur when visited by a bot.</p>
  
  <p>Not sure if it's relevant or not, but the site was just migrated to IIS 7.5. Still using .NET 2.0 in Integrated mode.</p>
  
  <p>Looking at the code as reversed by Reflector the only place where a null exception can occur directly in the <code>Get</code> method is the call to <code>this._request.FetchServerVariables</code>. As if the complete request was not set up properly.</p>
  
  <p>Has anyone else faced this problem or discovered a workaround? Why would the request be set up differently when visited by a bot?</p>
  
  <p><strong>UPDATE</strong>: Some additional debugging has shown that the HttpServerVarsCollection was Disposed, along with it's parent HttpRequest object. The question now is - how can the Request object returned by HttpContext.Current be exposed before the request is complete?</p>
  
  <p><strong>HttpServerVarsCollection.Get Method</strong></p>
  
  <pre><code>public override string Get(string name)
  {
      if (!this._populated)
      {
          string simpleServerVar = this.GetSimpleServerVar(name);
          if (simpleServerVar != null)
          {
              return simpleServerVar;
          }
          this.Populate();
      }
      if (this._iis7workerRequest == null)
      {
          return this.GetServerVar(base.BaseGet(name));
      }
      string serverVar = this.GetServerVar(base.BaseGet(name));
      if (string.IsNullOrEmpty(serverVar))
      {
          // Only place null reference can happen
          serverVar = this._request.FetchServerVariable(name);
      }
      return serverVar;
  }
  </code></pre>
  
  <p><strong>Full stack trace</strong></p>
  
  <pre><code>NullReferenceException: Object reference not set to an instance of an object.]
     System.Web.HttpServerVarsCollection.Get(String name) +8645730
     System.Collections.Specialized.NameValueCollection.get_Item(String name) +7
     System.Web.Mvc.PathHelpers.GenerateClientUrlInternal(HttpContextBase httpContext, String contentPath) in C:\Dev\Site\MVC\Microsoft\src\SystemWebMvc\Mvc\PathHelpers.cs:39
     System.Web.Mvc.PathHelpers.GenerateClientUrl(HttpContextBase httpContext, String contentPath) in C:\Dev\Site\MVC\Microsoft\src\SystemWebMvc\Mvc\PathHelpers.cs:21
     System.Web.Mvc.UrlHelper.GenerateUrl(String routeName, String actionName, String controllerName, RouteValueDictionary routeValues, RouteCollection routeCollection, RequestContext requestContext, Boolean includeImplicitMvcValues) in C:\Dev\Site\MVC\Microsoft\src\SystemWebMvc\Mvc\UrlHelper.cs:136
     System.Web.Mvc.UrlHelper.GenerateUrl(String routeName, String actionName, String controllerName, RouteValueDictionary routeValues) in C:\Dev\Site\MVC\Microsoft\src\SystemWebMvc\Mvc\UrlHelper.cs:101
     System.Web.Mvc.UrlHelper.Action(String actionName, String controllerName, Object routeValues) in C:\Dev\Site\MVC\Microsoft\src\SystemWebMvc\Mvc\UrlHelper.cs:51
     www.CmsExtensions.Document(UrlHelper urlHelper, String path) in C:\Dev\Site\www\Code\CmsExtensions.cs:33
     www.CmsExtensions.Document(UrlHelper urlHelper, Document document) in C:\Dev\Site\www\Code\CmsExtensions.cs:20
     www.&lt;&gt;c__DisplayClass17.&lt;Load&gt;b__c(Document d) in C:\Dev\Site\www\Global.asax.cs:251
     Fringine.Cms.DocumentContentParser.ReplaceDocumentRefs(IResolvingDocumentCache cache, Match match) +258
     Fringine.Cms.&lt;&gt;c__DisplayClass4.&lt;ParseContent&gt;b__2(Match m) +17
     System.Text.RegularExpressions.RegexReplacement.Replace(MatchEvaluator evaluator, Regex regex, String input, Int32 count, Int32 startat) +234
     System.Text.RegularExpressions.Regex.Replace(String input, MatchEvaluator evaluator, Int32 count, Int32 startat) +28
     System.Text.RegularExpressions.Regex.Replace(String input, MatchEvaluator evaluator) +38
     System.Text.RegularExpressions.Regex.Replace(String input, String pattern, MatchEvaluator evaluator, RegexOptions options) +47
     Fringine.Cms.DocumentContentParser.ParseContent(String content, IResolvingDocumentCache cache) +83
     Fringine.Cms.ResolvingDocumentCache.&lt;Parse&gt;b__0(String d) +21
     Fringine.Cms.DocumentCache.GetParsedData(String id, String content, IDocumentService documentService, Func`2 parser) +216
     Fringine.Cms.ResolvingDocumentCache.Parse(String id, String content) +67
     Fringine.Cms.CachedDocument.GetSummary() +966
     Fringine.Cms.CachedDocument.get_Summary() +19
     ASP.views_document_widget_feeddocumentsummary_ascx.__Render__control1(HtmlTextWriter __w, Control parameterContainer) +841
     System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children) +256
     System.Web.UI.Control.RenderChildren(HtmlTextWriter writer) +19
     System.Web.UI.Control.Render(HtmlTextWriter writer) +10
     System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter) +27
     System.Web.UI.Control.RenderControl(HtmlTextWriter writer, ControlAdapter adapter) +99
     System.Web.UI.Control.RenderControl(HtmlTextWriter writer) +25
     System.Web.UI.Control.RenderChildrenInternal(HtmlTextWriter writer, ICollection children) +134
     System.Web.UI.Control.RenderChildren(HtmlTextWriter writer) +19
     System.Web.UI.Page.Render(HtmlTextWriter writer) +29
     System.Web.Mvc.ViewPage.Render(HtmlTextWriter writer) in C:\Dev\Site\MVC\Microsoft\src\SystemWebMvc\Mvc\ViewPage.cs:107
     System.Web.UI.Control.RenderControlInternal(HtmlTextWriter writer, ControlAdapter adapter) +27
     System.Web.UI.Control.RenderControl(HtmlTextWriter writer, ControlAdapter adapter) +99
     System.Web.UI.Control.RenderControl(HtmlTextWriter writer) +25
     System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint) +1266
  </code></pre>