Should I use Request.UrlReferrer when determining referrals
<p>I came upon an interesting discussion with my team around the use of HttpRequest.UrlReferrer and wanted to solicit feedback from the community.  According to the <a href="http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.36" rel="nofollow">W3C spec</a>:</p>

<blockquote>
  <p>The Referer[sic] request-header field
  allows the client to specify, for the
  server's benefit, the address (URI) of
  the resource from which the
  Request-URI was obtained (the
  "referrer", although the header field
  is misspelled.) The Referer
  request-header allows a server to
  generate lists of back-links to
  resources for interest, logging,
  optimized caching, etc. It also allows
  obsolete or mistyped links to be
  traced for maintenance. The Referer
  field MUST NOT be sent if the
  Request-URI was obtained from a source
  that does not have its own URI, such
  as input from the user keyboard.as input from the user keyboard.</p>
</blockquote>

<p>The Request.UrlReferrer object does the work of converting referral strings that contain well formed URIs to an object with properties on every request.  </p>

<p>According to our logs there are requests that come in that contain invalid data in the referral such as:</p>

<pre><code>localhost
app:/BeamBackTest.swf
app:/multtiple.swf
app:/AFriendFeed.swf
ALToolBar
app:/index.html
mhtml:file://C:\Documents+and+Settings\User\Desktop\oracle\What+is+a+View+in+Oracle+-+Stack+Overflow.mht
</code></pre>

<p>Using Request.UrlReferrer would mean the above cases would be NULL.  Is it better to discard the invalid data based on the W3C spec by using Request.UrlReferrer or preserve it by using Request.ServerVariables["HTTP_REFERER"] even though the data may be interesting, but potentially useless. </p>
