How to properly extract content from a blog article?
<p>I am trying to extract content from a blog article like this:</p>

<pre><code>static void GetBlogData (string blogPostUrl)
{
    string blogPostContent = null;

    WebClient client = new WebClient ();
    //client.Headers.Add (HttpRequestHeader.Referer, "http://www.stackoverflow.com");

    TextWriter writer = new StreamWriter ("/home/nanda/projects/mono/common/article");

    try
    {
        blogPostContent = client.DownloadString (blogPostUrl);
    }

    catch (Exception ex)
    {
        Term.PrintLn ("Unable to download\n{0}", ex.Message);
    }

    if (blogPostContent != null) 
    {
        writer.WriteLine (blogPostContent);
    } 

    else
    {
        Term.PrintLn ("No content found");
    }
}
</code></pre>

<p>I am aware that this is too simple of an approach, but I want to know why I am unable to extract content from some URLs like they have a block or something. How can I detect if a website/blog is blocking me from downloading its content?</p>
