Method for returning valid URLs from a sitemap URL
Just a couple of thougths...




if (string.IsNullOrWhiteSpace(sitemapText))
  yield break;



This is a very unusual handling of null/empty argument values and should be documented. A more meaniningful name like GetSitemapUrlsOrDefault would also help. Without it the expected way is to throw an ArgumentException.




foreach (Match m in urlRegex.Matches(sitemapText))
  urls.Add(CleanUriString(m.Value));

foreach (var url in urls)
{
  var cleanedUriString = CleanUriString(url);

  if (Uri.IsWellFormedUriString(cleanedUriString, UriKind.RelativeOrAbsolute))
      yield return new Uri(cleanedUriString);
}



You don't need all this. Instead you can build a nice query that does everything in one go and in a single run:

return 
    urlRegex
        .Matches(sitemapText)
        .Cast&lt;Match&gt;()
        .Select(m =&gt; CleanUriString(m.Value))
        .Where(cleanedUriString =&gt; Uri.IsWellFormedUriString(cleanedUriString, UriKind.RelativeOrAbsolute))
        .Select(cleanedUriString =&gt; new Uri(cleanedUriString));


or alternatively:

return
    from m in urlRegex.Matches(sitemapText).Cast&lt;Match&gt;()
    let cleanedUriString = CleanUriString(m.Value)
    where Uri.IsWellFormedUriString(cleanedUriString, UriKind.RelativeOrAbsolute)
    select new Uri(cleanedUriString);




You don't need the .ToCharArray(); on the legalCharacters string. A string is already a char-array.



Oh, and of course the WebClient needs to be disposed. Wrapping it with a using would be good.