title: file_get_html failed to open stream: Connection refused in
body: <p>I got this function to get some info from a external URL.
  Problems is, if the website got robots no-index this function crashes which crashes the foreach loop afterwards.</p>
  
  <p><strong>Error msg:</strong></p>
  
  <blockquote>
    <p>Warning: file_get_contents(<a href="http://webontwerp-arnhem.nl/contact" rel="nofollow noreferrer">http://webontwerp-arnhem.nl/contact</a>): failed to open stream: Connection refused in /var/www/vhosts/free-sitemap-generator.com/httpdocs/includes/cra/simple_html_dom.php on line 79</p>
    
    <p>Fatal error: Uncaught Error: Call to a member function find() on boolean in /var/www/vhosts/free-sitemap-generator.com/httpdocs/includes/cra/xml-functions.php:60 Stack trace: #0 /var/www/vhosts/free-sitemap-generator.com/httpdocs/crawler.php(44): crawl_site('<a href="http://webontwe" rel="nofollow noreferrer">http://webontwe</a>...') #1 {main} thrown in /var/www/vhosts/free-sitemap-generator.com/httpdocs/includes/cra/xml-functions.php on line 60</p>
  </blockquote>
  
  <p><strong>Function:</strong></p>
  
  <pre><code>function file_get_html($url, $use_include_path = false, $context=null, $offset = -1, $maxLen=-1, $lowercase = true, $forceTagsClosed=true, $target_charset = DEFAULT_TARGET_CHARSET, $stripRN=true, $defaultBRText=DEFAULT_BR_TEXT, $defaultSpanText=DEFAULT_SPAN_TEXT)
  {
      $dom = new simple_html_dom(null, $lowercase, $forceTagsClosed, $target_charset, $stripRN, $defaultBRText, $defaultSpanText);
      $contents = file_get_contents($url, $use_include_path, $context, $offset);
  
      if (empty($contents) || strlen($contents) &gt; MAX_FILE_SIZE)
      {
          return false;
      }
  
      $dom-&gt;load($contents, $lowercase, $stripRN);
      return $dom;
  }
  </code></pre>
  
  <p><strong>Calling the function with loop:</strong></p>
  
  <pre><code>function crawl_site($u) {
      $urlList = array();
      global $crawled_urls, $found_urls;
      $uen = urlencode($u);
      if ((array_key_exists($uen, $crawled_urls) == 0 || $crawled_urls[$uen] &lt; date("YmdHis", strtotime('-25 seconds', time())))) {
          $html = file_get_html($u);
  
          $crawled_urls[$uen] = date("YmdHis");
          foreach($html -&gt; find("a")as $li) {
              $url = perfect_url($li -&gt; href, $u);
              $enurl = urlencode($url);
              $str = basename($url);
              $dirn = dirname($url);
              if ($url != '' &amp;&amp; substr($url, 0, 4) != "mail" &amp;&amp; substr($url, 0, 3) != "tel" &amp;&amp; substr($url, 0, 5) != "phone" &amp;&amp; substr($url, 0, 5) != "skype" &amp;&amp; substr($url, 0, 4) != "java" &amp;&amp; array_key_exists($enurl, $found_urls) == 0) {
                  $found_urls[$enurl] = 1;
                  $pos = strpos($str[0], '#');
                  $ext = strpos($url, $u);
                  if ($ext != = false &amp;&amp; $pos == = false) {
                      echo "&lt;li&gt;&lt;div class='url-row'&gt;$dirn/&lt;span class='strong'&gt;$str&lt;/span&gt;&lt;/div&gt;&lt;/li&gt;";
  
                      array_push($urlList, $url);
                  }
              }
          }
      }
  }
  </code></pre>