A faster way to look up for a url in a WebRequest/WebResponse html What I'm trying to do here is get an RSS feed and append an Enclosure XML node to each item, which has a link to a video file (wmv).

Try the below code with

url = "http://www.microsoft.com/events/series/digitalblackbelt.aspx?tab=rss"


for eg to get the point

The performance hit here consists of two parts: the LONG "foreach" loop, along with the HUGE number of requests done to retrieve the media link (marked in the code by the comment "This part needs attention"). Any advice concerning how to get the media link a much faster manner would really be appreciated!

You can get a glimpse of what that code do, by comparing this feed (the URL):

http://www.microsoft.com/events/series/digitalblackbelt.aspx?tab=rss

to this feed (created by the code below), note that it will be slow and might give error, if it did just refresh the page:

http://mshady.apphb.com/feeds/index?url=http://www.microsoft.com/events/series/digitalblackbelt.aspx?tab=rss

public class FeedsController : Controller
{
    private XmlDocument _Xml;

    [OutputCache(VaryByParam = "url", Duration=86400)]    
    public string Index(string url)
    {
        string feed = "";

        _Xml = new XmlDocument();
        _Xml.Load(url);

        foreach (XmlNode node in _Xml.ChildNodes[0].ChildNodes[0].ChildNodes)
        {
            if (node.Name == "item")
            {
                XmlNode childnode = GetEnclosureXmlNode(node);
                node.AppendChild(childnode);
            }
        }

        feed =  _Xml.OuterXml;

        return feed;
    }

    private string GetMediaUrl(string articleUrl)
    {
        string mediaUrl = "";

        ////////////////////////////////////////////    
        // This part needs attention
        ////////////////////////////////////////////    
        Regex regex = new Regex(@"&lt;a href=""(.*?)""&gt;WMV Download&lt;/a&gt;");
        var request = HttpWebRequest.Create(articleUrl);
        var response = request.GetResponse();
        string responseHtml = new StreamReader(response.GetResponseStream()).ReadToEnd();
        var match = regex.Match(responseHtml);
        ////////////////////////////////////////////    
        // This part needs attention
        ////////////////////////////////////////////    


        if (match.Success)
            mediaUrl = match.Groups[1].Value;

        return mediaUrl;
    }

    private XmlNode GetEnclosureXmlNode(XmlNode node)
    {
        string articleUrl = node["link"].InnerText;
        string mediaUrl = GetMediaUrl(articleUrl);

        XmlElement childnode = CreateEnclosureXmlNode(mediaUrl);
        return childnode;
    }

    private XmlElement CreateEnclosureXmlNode(string mediaUrl)
    {
        XmlElement childnode = _Xml.CreateElement("enclosure");
        childnode.SetAttribute("url", mediaUrl);
        childnode.SetAttribute("type", "video/wmv");
        return childnode;
    }
}
I would bet that the regex.Match is the performance problem, as Jeff Atwood describes here.
One thing that I would do is move regex definition to a static variable outside of the function defined as below:
static Regex regex = new Regex(@&quot;&lt;a href=&quot;&quot;(.*?)&quot;&quot;&gt;WMV Download&lt;/a&gt;&quot;, RegexOptions.Compiled);

That's the only real thing that jumps out at me. Try taking a look at this blog post for more ideas and information.
Should give you something to try until guys with more experience glance their eyes over it :)