URL checker is very slow I have +-645 IRL links I need to validate. The code below is my attempt, but my problem is that it takes forever to check all links and sometimes times out. Is there a better way of validating so many links? 

index.php

&lt;?php 
error_reporting(E_ALL);
ini_set('display_errors', 1);

$notValid = [];

// This array contains +-645 url in this array, http://www.example.com is just for example purposes.
$companyData = ['http://www.example.com','http://www.example.com', 'http://www.example.com', 'http://www.example.com'];
// Basic structure of $companyData array

Array
(
    [0] =&gt; Array
        (
            [Name] =&gt; Name-Ltd
            [Code] =&gt; XYZ
            [Url] =&gt; http://www.example.co.za/example-click-a-company/Name-Ltd
        )

    [1] =&gt; Array
        (
            [[Name] =&gt; Name-Ltd
            [Code] =&gt; ZXY
            [Url] =&gt; http://www.example.co.za/example-click-a-company/Name-Ltd
        )

    [2] =&gt; Array
        (
            [Name] =&gt; Name-Ltd
            [Code] =&gt; ZYX
            [Url] =&gt; http://www.example.co.za/example-click-a-company/Name-Ltd
        )

    [3] =&gt; Array
        (
            [Name] =&gt; Name-Ltd
            [Code] =&gt; YZX
            [Url] =&gt; http://www.example.co.za/example-click-a-company/Name-Ltd
        )

    [4] =&gt; Array
        (
            [Name] =&gt; Name-Ltd
            [Code] =&gt; XZY
            [Url] =&gt; http://www.example.co.za/example-click-a-company/Name-Ltd

)

// This is to loop through all url's
for ($i=0; $i &lt; count($companyData); $i++) 
{ 
    $url = $companyData[$i]['Url'];

    $headers = get_headers($url, 1);

    if(@$headers['Content-Length'] == 1153) 
    {
        $exists = 'not valid';

        $notValid[$i] = $companyData[$i]['Url'];
    }
    else 
    {
        $exists = 'valid';
    }
}

foreach ($notValid as $not) 
{
    echo "The following URL's need your attention: &lt;br&gt;";
    echo $not;
}

?&gt;
Your script is slow because you are doing a sequence of HTTP REQUESTS. Most of the time your server / script is just waiting for the RESPONSE of the requested URL. 

small things you can do


as mentioned yourself in the comments you can change your max execution time to prevent reaching it. (which does not make your script any faster ; -)
move away from get headers and use curl or similar to have the ability to change the connection timeout. You could set it to 1 sec and flag every url which fails to manual review (assuming all healthy urls will respond in time , and some slow false positives). Small benefit: your script should be finished in max url*sec time 


big things

so the problem is: in practice is your script waiting for responses, in theory it does not have to , as the checking of an url does not depend on each other. Classic case for parallelization. Downside: php is not the language you want to do parallelization in. 

upside: you are not the first with this problem


https://stackoverflow.com/questions/9308779/php-parallel-curl-requests
http://reactphp.org/


or use golang or something with build in parallelization