title: how to extract particular data using pig script from web crawled data (nutch)
body: <p>This sample crawl data using nutch 2.3.1 where i need to take title and urls internal links and external links attached with website any suggestion most welcome.</p>
  
  <p>I used this command to import data from hbase to pig</p>
  
  <pre><code>`data9 = load 'hbase://htest15_webpage' using org.apache.pig.backend.hadoop.hbase.HBaseStorage('f:cnt', '-loadKey true');
  </code></pre>
  
  <p>`</p>
  
  <pre><code>column=f:cnt, timestamp=1487743991250, value=&lt;!DOCTYPE htm
                    l&gt;\x0D\x0A&lt;!--[if IE 7]&gt;\x0D\x0A&lt;html class="ie ie7" lang=
                    "en-US" prefix="og: http://ogp.me/ns# fb: http://ogp.me/ns
                    /fb#"&gt;\x0D\x0A&lt;![endif]--&gt;\x0D\x0A&lt;!--[if IE 8]&gt;\x0D\x0A&lt;h
                    tml class="ie ie8" lang="en-US" prefix="og: http://ogp.me/
                    ns# fb: http://ogp.me/ns/fb#"&gt;\x0D\x0A&lt;![endif]--&gt;\x0D\x0A
                    &lt;!--[if !(IE 7) | !(IE 8)  ]&gt;&lt;!--&gt;\x0D\x0A&lt;html lang="en-U
                    S" prefix="og: http://ogp.me/ns# fb: http://ogp.me/ns/fb#"
                    &gt;\x0D\x0A&lt;!--&lt;![endif]--&gt;\x0D\x0A&lt;head&gt;\x0D\x0A  &lt;meta cha
                    rset="UTF-8" /&gt; \x0D\x0A  &lt;meta name="viewport" content="w
                    idth=device-width" /&gt;  \x0D\x0A  \x0D\x0A&lt;title&gt;www.hardhe
                    ro.com&lt;/title&gt;\x0A\x0A&lt;!-- SEO Ultimate (http://www.seodes
                    ignsolutions.com/wordpress-seo/) --&gt;\x0A\x09&lt;meta name="de
                    scription" content="hardhero.com blog that provides intere
                    sting articles on general topics and ideas." /&gt;\x0A\x09&lt;me
                    ta name="keywords" content="Internet Blog,SEO Blog,Interne
                    t Marketing Guide" /&gt;\x0A\x09&lt;meta property="og:type" cont
                    ent="blog" /&gt;\x0A\x09&lt;meta property="og:title" content="ww
                    w.hardhero.com" /&gt;\x0A\x09&lt;meta property="og:url" content=
                    "http://hardhero.com/" /&gt;\x0A\x09&lt;meta property="og:site_n
                    ame" content="www.hardhero.com" /&gt;\x0A\x09&lt;meta name="twit
                    ter:card" content="summary" /&gt;\x0A&lt;!-- /SEO Ultimate --&gt;\x
                    0A\x0A&lt;link rel="alternate" type="application/rss+xml" tit
                    le="www.hardhero.com &amp;raquo; Feed" href="http://hardhero.c
                    om/feed/" /&gt;\x0A&lt;link rel="alternate" type="application/rs
                    s+xml" title="www.hardhero.com &amp;raquo; Comments Feed" href
                    ="http://hardhero.com/comments/feed/" /&gt;\x0A\x09\x09&lt;scrip
                    t type="text/javascript"&gt;\x0A\x09\x09\x09window._wpemojiSe
                    ttings = {"baseUrl":"http:\x5C/\x5C/s.w.org\x5C/images\x5C
                    /core\x5C/emoji\x5C/72x72\x5C/","ext":".png","source":{"co
                    ncatemoji":"http:\x5C/\x5C/hardhero.com\x5C/wp-includes\x5
                    C/js\x5C/wp-emoji-release.min.js?ver=4.2.11"}};\x0A\x09\x0
                    9\x09!function(a,b,c){function d(a){var c=b.createElement(
                    "canvas"),d=c.getContext&amp;&amp;c.getContext("2d");return d&amp;&amp;d.f
                    illText?(d.textBaseline="top",d.font="600 32px Arial","fla
                    g"===a?(d.fillText(String.fromCharCode(55356,56812,55356,5
                    6807),0,0),c.toDataURL().length&gt;3e3):(d.fillText(String.fr
                    omCharCode(55357,56835),0,0),0!==d.getImageData(16,16,1,1)
                    .data[0])):!1}function e(a){var c=b.createElement("script"
                    );c.src=a,c.type="text/javascript",b.getElementsByTagName(
                    "head")[0].appendChild(c)}var f,g;c.supports={simple:d("si
                    mple"),flag:d("flag")},c.DOMReady=!1,c.readyCallback=funct
                    ion(){c.DOMReady=!0},c.supports.simple&amp;&amp;c.supports.flag||(
                    g=function(){c.readyCallback()},b.addEventListener?(b.addE
                    ventListener("DOMContentLoaded",g,!1),a.addEventListener("
                    load",g,!1)):(a.attachEvent("onload",g),b.attachEvent("onr
                    eadystatechange",function(){"complete"===b.readyState&amp;&amp;c.r
                    eadyCallback()})),f=c.source||{},f.concatemoji?e(f.concate
                    moji):f.wpemoji&amp;&amp;f.twemoji&amp;&amp;(e(f.twemoji),e(f.wpemoji)))}(
                    window,document,window._wpemojiSettings);\x0A\x09\x09&lt;/scr
                    ipt&gt;\x0A\x09\x09&lt;style type="text/css"&gt;\x0Aimg.wp-smiley,\
                    x0Aimg.emoji {\x0A\x09display: inline !important;\x0A\x09b
                    order: none !important;\x0A\x09box-shadow: none !important
                    ;\x0A\x09height: 1em !important;\x0A\x09width: 1em !import
                    ant;\x0A\x09margin: 0 .07em !important;\x0A\x09vertical-al
                    ign: -0.1em !important;\x0A\x09background: none !important
                    ;\x0A\x09padding: 0 !important;\x0A}\x0A&lt;/style&gt;\x0A&lt;link 
                    rel='stylesheet' id='es-widget-css-css'  href='http://hard
                    hero.com/wp-content/plugins/email-subscribers/widget/es-wi
                    dget.css?ver=4.2.11' type='text/css' media='all' /&gt;\x0A&lt;li
                    nk rel='stylesheet' id='shootingstar-style-css'  href='htt
                    p://hardhero.com/wp-content/themes/shootingstar/style.css?
                    ver=4.2.11' type='text/css' media='all' /&gt;\x0A&lt;link rel='s
                    tylesheet' id='shootingstar-elegantfont-css'  href='http:/
                    /hardhero.com/wp-content/themes/shootingstar/css/elegantfo
  </code></pre>
  
  <p>></p>