Compressing large jpeg images I'm working with thousands of large image files in a regularly updated library. The following script does the job (on average reduces my file size ~95%) but costs me around 25 seconds to compress one image. Obviously, I can just let the script run overnight, but it would be cool if I can shave some time off this process. I'm mostly looking for any unnecessary redundancies or overhead in the script that can be trimmed out to speed up the process. I'm still new to Python, so go easy on me.

from PIL import Image
from pathlib import Path
import os, sys
import glob

root_dir = "/.../"

basewidth = 3500

for filename in glob.iglob(root_dir + '*.jpg', recursive=True):
    p = Path(filename)
    img = p.relative_to(root_dir)
    new_name = (root_dir + 'compressed/' + str(img))
    print(new_name)
    im = Image.open(filename)
    wpercent = (basewidth/float(im.size[0]))
    hsize = int((float(im.size[1])*float(wpercent)))
    im = im.resize((basewidth,hsize), Image.ANTIALIAS)
    im.save(new_name, 'JPEG', quality=40)
I think what greybeard is getting at in the comments is that you could get away with squishing these images much more than you presently are. It sounds like you're basically using the reduced versions as thumbnails, but these "thumbnails" are almost twice the width (over three times the area) of a standard HD monitor. 

Dropping basewidth to 1920 (or possibly much lower still) seems like a good idea. Contrary to greybeard, I think your JPEG "quality" setting is fine, but you could play around with different variations of smaller-vs-larger and crisp-vs-compressed.

A minute of googling suggests that PIL is in fact the normal choice for image handling. Maybe there's something better, but I'll suppose not. Given that the originals are large (50MB? more?), it may simply be that there's a lot of work to be done. That said, there are some things you can try.


General hardware: Given that this is a heavy task, you may have some luck just running the same code on a fancier computer. This is an expensive option; even if you feel like throwing money at the problem still do plenty of research first.
Storage hardware: One possible bottleneck is the hard drive you're reading from and writing to. If you can easily try the task against a different (faster or slower) harddrive while keeping everything else the same, that may be informative.
Memory: Does you computer have a lot of RAM? Is that memory available for this python script to use? If you have a way of making half the currently available memory unavaliable, check if that makes the process twice as slow.
Memory leaks: I see the line im = Image.open(filename), and looking at the docs suggests that you should probably have a call to load or more likely close someplace. 
Parallelism: Your computer probably has multiple processor cores. It's hard to say if or how well this script is using all of those cores. If you try running your script twice at the same time (against non-overlapping target sets), how much slower is each concurrent process?
It's not clear if ANTIALIAS is a valid option to the resize method in the 3.x version. Try NEAREST.
Oh hey there's a thumbnail function that affects the way the file is read from disk!. Playing around with the underlying draft function may also be helpful.


And because this is code review:


Move your format and quality constants up to the same place as basewidth.