title: How can I use the Google Custom Search API to check if a list of website's is currently indexed by Google?
body: <p>Currently, you can import a spreadsheet (contains a list of websites) into a database. You are able to click a button and it retrieves the number of pages that Google has indexed for each domain in the database.</p>
  
  <p>It works fine up until about ~400 requests and then my requests come back with nothing. Not sure what I am doing incorrectly or how to make this work - any ideas?</p>
  
  <pre><code>try {
          require('db.php');
          $conn = new PDO ( "mysql:host=localhost;dbname=" . $database, $user, $pass );
          $conn-&gt;setAttribute( PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION );
          $stmt = $conn-&gt;query('SELECT `url` FROM domains');
          $stmt-&gt;setFetchMode(PDO::FETCH_ASSOC);
          while($row = $stmt-&gt;fetch()){
              $id = $row['id'];
              $url = $row['url'];
              $pages = getIndexedPageCount($url);
              if ( $pages &gt; 0 ) {
                  $status = 1;
              } else {
                  $status = 0;
              }
              $prep = $conn-&gt;prepare("
              UPDATE `domains`
                  SET
                      url = :url,
                      pages = :pages,
                      status = :status
                  WHERE url = :url
              ");
              $prep-&gt;execute(array(
                  ':url'      =&gt;  $url,
                  ':pages'    =&gt;  $pages,
                  ':status'   =&gt;  $status
              ));
          }
      } catch ( PDOException $e ) {
          echo 'Error: ' . $e-&gt;getMessage();
          exit;
      }
  
  
      function getIndexedPageCount($domain) {
  
      // remove http:// and https://
      if ( strpos( $domain, 'http:' ) == 0 ) { $domain = str_replace('http://', '', $domain); }
      if ( strpos( $domain, 'https:' ) == 0 ) { $domain = str_replace('https://', '', $domain); }
  
          $content = file_get_contents('https://www.googleapis.com/customsearch/v1?key={{Removed API Key}}&amp;q=site:' . $domain);
  
          if (strlen($content) &gt; 1) {
              $data = json_decode($content);
              return intval($data-&gt;searchInformation-&gt;totalResults);
          }
  
          else {
              echo "Error: URL does not exist.";
          }
      }
  </code></pre>
  
  <p>Any help would be greatly appreciated; if anyone has an idea I'll try and make it work!</p>