title: Python - Newspaper Library - Why is it missing sizable portions of articles?
body: <p>I'm using the newspaper library, V. 2.7 found <a href="http://newspaper.readthedocs.io/en/latest/" rel="nofollow">here</a>. </p>
  
  <p>When I download, parse, and print the text, it gives me a much smaller portion of the article than exists in reality. Why is this? Is there any way to fix this? </p>
  
  <p>Here is my code:</p>
  
  <pre><code>from newspaper import Article
  
  url = "http://www.nytimes.com/2010/10/07/us/politics/07manchin.html"
  
  article_object = Article(url, language = 'en')
  article_object.download()
  article_object.parse()
  article_object.text
  </code></pre>
  
  <p>If you look at what it prints out, and <strong><a href="http://www.nytimes.com/2010/10/07/us/politics/07manchin.html" rel="nofollow">compares with the actual article online</a></strong>, one finds that the article.text skips over the first 7-8 paragraphs of the actual article. Any way to make sure the "full" article is captured?</p>
  
  <p>Here's the output of article_object.text:</p>
  
  <blockquote>
    <p>u'The national Republican Party has smelled blood and is spending
    millions on television ads here as part of its drive to retake the
    Senate. One new ad shows men in baseball caps predicting that Mr.
    Manchin will transform into \u201cWashington Joe,\u201d declaring,
    \u201cWe\u2019ve got to stop Obama.\u201d\n\nPhoto\n\nMr. Manchin
    thought he had punched all the right buttons in this state with a
    Democratic majority but conservative values. He won his second term as
    governor in 2008 with 70 percent of the vote and has garnered praise
    for fiscal responsibility; even Mr. Raese calls him a good governor.
    He has been endorsed by the Chamber of Commerce, the National Rifle
    Association and both the coal industry and the mine workers.\n\nMr.
    Manchin is highlighting his conservative credentials and record of
    independence and he has accused his opponent of waging a campaign of
    \u201cfear and smear.\u201d But the anti-Obama message appears to be
    resonating with some residents \u2014 perhaps the same voters who gave
    the state to Senator John McCain and President Bush in the last three
    presidential elections.\n\n\u201cManchin has been a great governor,
    but I\u2019m going to vote for Raese,\u201d said Jeff Whittington, 41,
    a tire distributor in Charleston. \u201cI think Manchin will get to
    the Senate and be a rubber stamp for Obama,\u201d he said, repeating
    Mr. Raese\u2019s message almost word for word.\n\nIn a state where
    coal is seen as the fount of prosperity, Mr. Raese has seized on what
    he calls ambiguous statements by Mr. Manchin on proposals for a
    cap-and-trade system to reduce greenhouse gases. Mr. Manchin now
    speaks clearly on the issue: he is firmly against cap-and-trade and a
    carbon tax, he emphasized in the interview.\n\nOn Wednesday, Mr.
    Manchin had an opportunity to underscore his support for coal and to
    distance himself from the president. With the head of the coal
    producers\u2019 association at his side in the state Capitol, he
    announced that the state was suing two federal agencies, seeking to
    reverse the stricter controls on mountain-top coal mining adopted in
    2009 by the Obama administration.\n\nAdvertisement Continue reading
    the main story\n\nDiscussing the suit against the Environmental
    Protection Agency and the Army Corps of Engineers, Mr. Manchin said
    that the more stringent procedures were unlawful and had harmed the
    state by slowing new mining projects to a trickle, and he accused the
    Obama administration of trying \u201cto destroy our coal industry and
    way of life.\u201d\n\nThe E.P.A. responded that its actions were
    legally and scientifically sound.\n\nMr. Manchin, independent experts
    and even the West Virginia Coal Association have taken exception to a
    Raese ad accusing Mr. Manchin of passing a state law that
    \u201celiminates 25 percent of coal usage in our power
    plants.\u201d\n\nIn fact, the law calls for progress in new energy
    technologies that include cleaner coal, and it had the support of the
    industry.\n\nMr. Raese has also lambasted the governor for past
    statements welcoming the Obama health plan; Mr. Manchin says that some
    core elements of the plan, like protecting children\u2019s coverage,
    are good but that others should be repealed. Mr. Raese calls the
    health plan \u201cpure, unadulterated socialism\u201d that should be
    discarded entirely, a message that appeals to many small-business
    owners.\n\nPhoto\n\nMr. Raese is a brawny and self-confident man who
    has run unsuccessfully for the Senate and for governor before and now
    feels currents flowing his way. \u201cHow do you put together a
    business plan when you have Obamacare facing you, when you\u2019ve got
    cap-and-trade facing you?\u201d he asked in an interview at the
    Republican office in Morgantown, his home base.\n\nMr. Raese runs
    limestone mines and a steel-fabrication company and is part owner of a
    radio network and a newspaper. He has at least three homes around the
    country, and his wife lives in their home in Palm Beach, Fla., while
    he has kept his residence in West Virginia.\n\nWhile Mr. Raese\u2019s
    anti-Washington message has wide appeal, his unequivocal support for
    Nafta and free trade and his opposition to unions may not help him in
    a state with a blue-collar heritage. Mr. Manchin\u2019s campaign has
    run ads featuring Mr. Raese boasting about his inherited wealth and
    highlighting his opposition to minimum wages and other worker
    protections.\n\nIn interviews across the state, some residents said
    that while Mr. Raese can seem arrogant, they admired his record of
    creating jobs and were ready to send a new face to Washington rather
    than elect a man they see as a career politician.\n\nAdvertisement
    Continue reading the main story\n\nThe only polls published so far
    were automated \u2014 done without personal interviews \u2014 and are
    not widely accepted as reliable. But private polling by both camps
    indicates an unexpectedly close race.\n\nFord Frances, 53, a lawyer in
    Charleston, said he likes Mr. Manchin as governor \u2014 Mr. Manchin
    will retain his job if he loses the Senate race \u2014 but that he is
    leaning toward voting for Mr. Raese because he worries about a
    skyrocketing federal deficit and thinks government has too big a role
    in the economy.\n\nOthers, including Cheryl Bonner, 64, of Fairmont,
    said they thought it was unfair to paint Mr. Manchin with the excesses
    of Washington. \u201cHe\u2019s been for West Virginia, and I\u2019m
    leaning toward voting for him,\u201d said Ms. Bonner, who had to start
    working again, as a store clerk, because her and her husband\u2019s
    Social Security and pension checks did not cover the bills. She hopes
    that retirees will be protected, whoever wins, she said.\n\nSome
    residents said Mr. Manchin was just unlucky to be running for national
    office this year. \u201cProbably at any other time he\u2019d have
    gotten my vote,\u201d said a 51-year-old credit analyst in Charleston,
    an economic conservative who voted for Mr. Manchin in 2008 but is
    troubled by the administration\u2019s economic
    policies.\n\n\u201cPutting him in that Senate seat would enhance the
    Democrats\u2019 position,\u201d said the man, who declined to give his
    name because his company has dealings with the governor\u2019s office.
    \u201cWe\u2019d rather keep him here as governor.\u201d'</p>
  </blockquote>