title: Fixing a 500 Internal Server Error on large image process
body: <p><strong>Information:</strong> I have already looked around for answers on SO, Google and other places with no avail. Please do not redirect me to other answers or websites unless it involves new information.</p>
  
  <p>I am building a huge website which converts thousands of images in one go and uploads details of these images to the Database (All images are already uploaded via FTP). The process only creates thumbnails and creates image links in the tables.</p>
  
  <p><strong>The problem:</strong> The process takes on average 30 minutes to complete. After roughly 15 minutes, the site displays a <code>500 Internal Server Error</code> and the website will not reload until the process has completed. The process will still complete as it should in the background? (No idea how this works, but it does). No errors are recorded on the server as I have checked. </p>
  
  <p><strong>Attempted Solutions:</strong> I have already attempted a couple things...</p>
  
  <p>Adding this to the <code>.htaccess</code> file</p>
  
  <pre><code>&lt;IfModule mod_php5.c&gt;
  php_value max_execution_time 0
  php_value max_input_time 0
  php_value memory_limit 5000M
  php_value post_max_size 5000M
  php_value upload_max_filesize 5000M
  &lt;/IfModule&gt;
  </code></pre>
  
  <p>Using <code>set_time_limit(0);</code> within the function and the script itself.</p>
  
  <p>I have looked at the option to update the <code>httpd.conf</code> file to change the <code>MaxRequestLen</code> but I'm not sure if that would be the correct thing to do in this circumstance? </p>
  
  <p><strong>Scripts:</strong></p>
  
  <p>Batch Upload Script - </p>
  
  <pre><code>if(isset($_GET['create_all_albums'])){
      $length2 = 8;
      $randomString2 = substr(str_shuffle("0123456789"), 0, $length2);
      setcookie('album_error_log', $randomString2, time() + (3600), "/");
      $starting_folder = $_GET['create_all_albums'];
  
      $date_time = date("Y-m-d H:i:s");
      $useridnu = $_SESSION['userid'];
      $useremail = $_SESSION['email_address'];
      mysqli_query($conn,"INSERT INTO new_recent_batch_uses (use_id,user_id,email_address,error_ref,batched_folder,batch_started,batch_completed) VALUES ('','$useridnu','$useremail','$randomString2','$starting_folder','$date_time','$date_time')");
  
      $i = 0;
      $c = 0;
  
      $paths = scandir($starting_folder);
      foreach($paths as $path){
  
          if($path == '.' || $path == '..'){
  
          } else {
              $path = $starting_folder.$path;
              $folders = scandir($path);
              $count_requests = count($folders);
               $arr_content2 = array();
               $com = 0;
              foreach($folders as $folder){
  
                  if($folder == '.' || $folder == '..'){
  
                  } else {
                      ++$i;
                      $file_name = $path.'/'.$folder;
                      if (substr($file_name, -1, 1) == '/')
                         {
                         $file_name = substr($file_name, 0, -1);
                         }
  
  
                         $string_1 = $file_name;
  
                         $string_1 = substr($string_1, 0, strrpos( $string_1, '/'));
  
                         $club_name = substr(strrchr($file_name, "/"), 1);
                         if($club_name == 'xots'){
                         $club_name = 'ots';
                         } else {
                         $club_name = $club_name;
                         }
  
                         $club_date = substr(strrchr($string_1, "/"), 1);
                         $club_date = chunk_split($club_date, 2, '/');
                         $club_date = rtrim($club_date, "/");
                         $club_date = substr_replace($club_date, '20', 6, 0);
                         $club_date = str_replace('/', '-', $club_date);
                         $club_date = date('Y-m-d', strtotime($club_date));
  
                         $cutyear = substr($string_1, 0, strrpos( $string_1, '/'));
                         $year = substr(strrchr($cutyear, "/"), 1);
  
                         $club_area = substr($cutyear, 0, strrpos( $cutyear, '/'));
                         $club_area = substr(strrchr($club_area, "/"), 1);
  
  
                         $getvenueinfo = mysqli_fetch_assoc(mysqli_query($conn, "SELECT * FROM `new_categories` WHERE `short_code` = '$club_name' AND `cat_city` = '$club_area' AND `allow_albums` = '1'"));
  
                              $cat_id_2 = $getvenueinfo['cat_id'];
                              $cat_name_2 = $getvenueinfo['cat_name'];
                              $cat_down_lvl_2 = $getvenueinfo['cat_down_lvl'];
                              $album_name = date("D dS M Y", strtotime($club_date));
                              $album_date = date("Y-m-d", strtotime($club_date));
                              $todays_date = date("Y-m-d");
                       if(!$cat_id_2 == '' || !$cat_name_2 == '' || !$cat_down_lvl_2 == ''){
                        $length = 25;
                        $randomString = substr(str_shuffle("0123456789"), 0, $length);
  
  
                if($album_date == '1970-01-01'){
                                   $date_time = date("Y-m-d H:i:s");
                                   mysqli_query($conn,"INSERT INTO new_upload_error_log (error_id,error_ref,error_type,error_image,error_album,error_date,success) VALUES ('','$randomString2','Album Creation was unsuccessful (Album date does not seem correct)...','$file_name','','$date_time','4')");
                              } else {
  
  
                        $search_ID = true;
  
                        while($search_ID == true) {
                              $sqla = "SELECT * FROM `new_albums` WHERE `album_ref` = '$randomString'";
                              $resulta = $conn-&gt;query($sqla);
  
                              if (!$resulta-&gt;num_rows &gt; 0) {
  
  
                              $sqlc = "SELECT * FROM `new_albums` WHERE `album_name` = '$album_name' AND `venue_name` = '$cat_name_2' AND `city_name` = '$club_area'";
                              $resultc = $conn-&gt;query($sqlc);
  
                                     if (!$resultc-&gt;num_rows &gt; 0) {
  
                                     mysqli_query($conn,"INSERT INTO new_albums (album_id,album_name,album_desc,album_password,album_date,venue_name,venue_short_code,city_name,cat_id,album_ref,date_added,amount_of_files,viewed_count) 
               VALUES ('','$album_name','','','$album_date','$cat_name_2','$club_name','$club_area','$cat_down_lvl_2','$randomString','$todays_date','0','0')");
  
                 mysqli_query($conn, "UPDATE new_categories SET `last_updated` = '$todays_date' WHERE `cat_down_lvl` = '$cat_id'");
                 ++$c;
  
                 if($image_convert_on_album_create == 1){
  
                      $com++; 
  
                      $percent = intval($com/$count_requests * 100);
                      $arr_content2['percent'] = $percent;
                      $arr_content2['message'] = $com . " folders(s) processed.";
                      create_thumbs($conn, $randomString, $file_name, $randomString2);
                      file_put_contents("tmp2/" . session_id() . ".txt", json_encode($arr_content2));
  
  
          } else {
  
          }
  
                           } else {
      $date_time = date("Y-m-d H:i:s");
  
       mysqli_query($conn,"INSERT INTO new_upload_error_log (error_id,error_ref,error_type,error_image,error_album,error_date,success) VALUES ('','$randomString2','Album Creation was unsuccessful (Album already exists)...','$file_name','','$date_time','4')");
                           }
  
  
  
               $search_ID = false;
  
              } 
            } 
                              }
                        } else {
                            $date_time = date("Y-m-d H:i:s");
  
       mysqli_query($conn,"INSERT INTO new_upload_error_log (error_id,error_ref,error_type,error_image,error_album,error_date,success) VALUES ('','$randomString2','Album Creation was unsuccessful (Category could not be located)...','$file_name','','$date_time','3')");
                        }
  
                  }
  
              } // 2nd foreach
          }
  
      } // 1st foreach
      $date_time = date("Y-m-d H:i:s");
      mysqli_query($conn,"UPDATE new_recent_batch_uses SET `batch_completed` = '$date_time' WHERE `error_ref` = '$randomString2'");
  
      header("Location: admin_batch_upload.php?album_create_completed=1&amp;completed=".$c."&amp;folders=".$i."#images");
  }
  </code></pre>
  
  <p>create_thumbs function - </p>
  
  <pre><code>function create_thumbs($conn, $albums_to_add, $choosen_files, $error_ref) { 
  set_time_limit(0);
  $get_quality = mysqli_fetch_assoc(mysqli_query($conn, "SELECT `value` FROM `new_system_settings` WHERE `setting_id` = '31'"));
  $img_quality = $get_quality['value'];
  
  $get_resize = mysqli_fetch_assoc(mysqli_query($conn, "SELECT `value` FROM `new_system_settings` WHERE `setting_id` = '32'"));
  $img_resize = $get_resize['value'];
  
  
  session_start();
  $dirname = $choosen_files;
  chmod($dirname, 0777);
  $todays_date = date("Y-m-d");
  $date_time = date("Y-m-d H:i:s");
  
  if($albums_to_add == ''){
       mysqli_query($conn,"INSERT INTO new_upload_error_log (error_id,error_ref,error_type,error_image,error_album,error_date,success) VALUES ('','$error_ref','There was no album selected for this file','$choosen_files','$albums_to_add','$date_time','3')");
  } else {
  
  $check_end = substr($dirname, -1);
  if($check_end == '/'){
  $dirname = $choosen_files;  
  } else {
  $dirname = $choosen_files.'/';
  }
  
  $images = glob($dirname."*.{JPG,jpg,PNG,png,gif}", GLOB_BRACE);
  $filecount = count($images);
  
  $arr_content = array();
  
  
  $counti = 0;
  foreach($images as $image) {
  $counti++;  
  $percent = intval($counti/$filecount * 100);
    $arr_content['percent'] = $percent;
    $arr_content['message'] = $counti . " image(s) processed.";
  
  $file = $image;
  
  $data = $image;    
  $cut_name = substr($data, strpos($data, "/") + 1);  
  $cut_name = explode('/',$cut_name); 
  $cut_name = end($cut_name); 
  
  
  if (0 === strpos($cut_name, 'thb_')) {
     mysqli_query($conn,"INSERT INTO new_upload_error_log (error_id,error_ref,error_type,error_image,error_album,error_date,success) VALUES ('','$error_ref','Compressed Image already exists in this file...','$newfile','$albums_to_add','$date_time','4')");
  } else {
  
  if (strpos($cut_name, 'thumb_') == false &amp;&amp; strpos($cut_name, 'normal_') == false) {
  
      if (strpos($cut_name, 'mini_') == false) {
  
  
      $newfile = $dirname.'thb_'.$cut_name;
      ///unlink($newfile);
      if (file_exists($newfile)) {
  
      mysqli_query($conn,"INSERT INTO new_upload_error_log (error_id,error_ref,error_type,error_image,error_album,error_date,success) VALUES ('','$error_ref','Compressed Image already exists in this file...','$newfile','$albums_to_add','$date_time','4')");
  
  } else {
  
      $info = getimagesize($file);
  
  list($width, $height) = getimagesize($file);
  
      $new_width = $width * $img_resize / 100;
      $new_height = $height * $img_resize / 100;
  
  if ($info['mime'] == 'image/jpeg') $image = imagecreatefromjpeg($file);
      elseif ($info['mime'] == 'image/gif') $image = imagecreatefromgif($file);
      elseif ($info['mime'] == 'image/png') $image = imagecreatefrompng($file);
  
  $image = imagecreatetruecolor($new_width, $new_height);
  $photo = imagecreatefromjpeg($file);
  imagecopyresampled($image, $photo, 0, 0, 0, 0, $new_width, $new_height, $width, $height);
  
  
      imagejpeg($image, $newfile, $img_quality);
  
  
  
      $origional_image = $file;
      $compressed_image = $newfile;
  
  if (file_exists($compressed_image)) {   
  
  $query = mysqli_query($conn, "SELECT * FROM new_images WHERE alb_ref = '".$albums_to_add."' AND full_link = '".$origional_image."'");
  if(mysqli_num_rows($query) &gt; 0){
  
      mysqli_query($conn,"INSERT INTO new_upload_error_log (error_id,error_ref,error_type,error_image,error_album,error_date,success) VALUES ('','$error_ref','This image already exists within this Album...','$compressed_image','$albums_to_add','$date_time','4')");
  
  } else {    
      mysqli_query($conn,"INSERT INTO new_images (img_id,alb_ref,thumb_link,full_link,viewed_count,date_added) VALUES ('','$albums_to_add','$compressed_image','$origional_image','0','$todays_date')");
  
      mysqli_query($conn,"UPDATE new_albums SET amount_of_files = amount_of_files + 1 WHERE album_ref = '$albums_to_add'");
  }
  
  } else {
  
      mysqli_query($conn,"INSERT INTO new_upload_error_log (error_id,error_ref,error_type,error_image,error_album,error_date,success) VALUES ('','$error_ref','This image failed to compress...','$origional_image','$albums_to_add','$date_time','3')");
  
  }
      file_put_contents("tmp/" . session_id() . ".txt", json_encode($arr_content));  
  }
  
  
  } else {
  
  }   
  } else {
  
  }
  
  }
  }   
  }
  }
  </code></pre>
  
  <p>If anybody has any other solutions for me, that would be great? Even some advice on how I can work out the actual type of error I am getting. I am pretty sure it is a timeout error, because it happens on every attempt although on smaller folder structures with less files, the system works flawlessly.</p>