title: How to read weblog file using URLExtract Get unique URL using python
body: <p>Beginner Python.. I have a large log file</p>
  
  <p>how to read and get the log file URL only and I am trying to get only certain URLs like (https, http, :443, :80)
  Because some log file indexes are different, how to read certain fields correctly from different lines</p>
  
  <p>How to get unique URL results? I tried a unique URL, but can not get a unique URL</p>
  
  <p>Sample log file format:</p>
  
  <pre><code>60.80.94.184    anonymous   Moz/2.0 (iPhone; CPU iPhone OS 9_0 like Mac OS X) Apple/65.1.90 (HTML, like Gecko) Version/12.0 Mobile/15E Safari/604.1 2012-06-22  03:43:51    -   60.80.17.54 8090    0   781 9843    SSL-tunnel  -   qs.rtoas.zp:80  Upstream    0   0x3 Allowed
  180.81.82.170   anonymous   iPad1,3/09.1.1 (16q0)   2012-06-24  04:53:57    -   90.80.97.54 8070    47  217 8440    http    GET http://init-p0.pu.apple.com/bag?v=9 Upstream    200 0x400   Allowed
  109.13.61.195   anonymous   clo/76.119 Network/95.0.3 Dain/1.2.0    2012-06-25  09:43:54    -   190.22.19.94    8220    0   517 5057    SSL-tunnel  -   eree-022.opt-2.icloud-content.com:443   Upstream    0   0x8 Allowed
  20.81.82.110    anonymous   iPad1,1/09.1.1 (46q5)   2012-06-27  14:53:57    -   40.10.27.54 8070    47  217 8440    http    GET https://qwe-pu.uoras.com/bag?v=19   Upstream    200 x00 Allowed
  </code></pre>
  
  <p>code:</p>
  
  <pre><code>from urlextract import URLExtract
  files = "WEB_000.w3c"
  with open(files, 'r', encoding='utf-8') as f:
      for i in f:
          lines = i.strip()
          extractor = URLExtract()
          urls = extractor.find_urls(lines, only_unique=True)
          li = list(set())
          for wa in urls:
              li.append(wa)
          for se in li:
              print(se)
  </code></pre>