Efficiently generating thumbnails with ImageMagick and convert - imagemagick

Question

I'm looking to efficiently generate various sized thumbnails with ImageMagick's convert utility in Python. Some of my image files are quite large (~15MB JPGs).
One way I could do it would be to take the full-sized image, and to generate the various thumbnails from the full-sized image, as follows:
convert sample_image.jpg -resize 1024x768 sample_image-1024x768.jpg
convert sample_image.jpg -resize 800x600 sample_image-800x600.jpg
convert sample_image.jpg -resize 400x300 sample_image-400x300.jpg
convert sample_image.jpg -resize 200x150 sample_image-200x150.jpg
But another way would be to resize the images from each other:
convert sample_image.jpg -resize 1024x768 sample_image-1024x768.jpg
convert sample_image-1024x768.jpg -resize 800x600 sample_image-800x600.jpg
convert sample_image-800x600.jpg -resize 400x300 sample_image-400x300.jpg
convert sample_image-400x300.jpg -resize 200x150 sample_image-200x150.jpg
Is there any downside to doing this, or perhaps a better way? It seems like this would be a lot more efficient.
As a corollary, are there any flags or "tricks" convert uses to speed up the process?

Answer 1

ImageMagick has a few tricks up its sleeves which help you to optimize for speed when you want to process large images and when you want to create different output from the same original:
Make use of ImageMagick's mpr:{name} feature, which makes it temporarily save the input image into a named memory program register, from which you can later (while processing) read the data much faster than you could do from harddisk.
Do all resize operations in one single process writing out the different output sizes you require.
And the even better news is you can combine both these into one single command.
So you do not need to run multiple processes with all their context-switching overhead -- do it all in one go.
The following example also crops two separate areas from the original image and creates re-sized thumbnails from them, just to show how many different operations IM can do in one commandline. It also, of course outputs the sizes you requested. (You'll need, of course, a really large-dimensioned input image for the cropping parameters to work).
convert \
huge-original.jpg \
-quality 80 \
-colorspace rgb \
+profile '*' \
-filter Lanczos \
-write mpr:copy-of-huge-original \
+delete \
mpr:copy-of-huge-original -crop '3000x2000+0+480' -resize '200x125!>' -write thumb1-extract.jpg +delete \
mpr:copy-of-huge-original -crop '2000x1500+280+220' -resize '75x75!>' -write thumb2-extract.jpg +delete \
mpr:copy-of-huge-original -resize '1024x768' -write sample-1024x768.jpg +delete \
mpr:copy-of-huge-original -resize '800x600' -write sample-800x600.jpg +delete \
mpr:copy-of-huge-original -resize '400x300' -write sample-400x300.jpg +delete \
mpr:copy-of-huge-original -resize '200x150' -write sample-200x150.jpg +delete \
mpr:copy-of-huge-original -resize '163x163!>' -write sample-163x163.jpg
Update
I only now saw the question asked by #JonathanOng: How to stream the output to <stdout>?
Assuming, you want the format going to stdout also be JPEG, you can try this:
convert \
huge-original.jpg \
-quality 80 \
-colorspace rgb \
+profile '*' \
-filter Lanczos \
+write mpr:copy-of-huge-original \
mpr:copy-of-huge-original -crop '3000x2000+0+480' -resize '200x125!>' +write thumb1-extract.jpg \
mpr:copy-of-huge-original -crop '2000x1500+280+220' -resize '75x75!>' +write thumb2-extract.jpg \
mpr:copy-of-huge-original -resize '1024x768' +write jpeg:- \
mpr:copy-of-huge-original -resize '800x600' +write jpeg:- \
mpr:copy-of-huge-original -resize '400x300' +write jpeg:- \
mpr:copy-of-huge-original -resize '200x150' +write jpeg:- \
mpr:copy-of-huge-original -resize '163x163!>' +write jpeg:-
This way each variant will go to stdout. How you deal with this stream of consecutive images then, is up to you...
Note, instead of writing -write filename +delete you can use +write filename. It amounts to the same effect.

Answer 2

I tried timing vipsthumbnail against #KurtPfeifle's excellent answer. I ran this with a 10k x 10k pixel RGB JPG image (about 15MB):
convert \
/data/john/pics/wtc.jpg \
-quality 80 \
-colorspace rgb \
+profile '*' \
-filter Lanczos \
-write mpr:copy-of-huge-original \
+delete \
mpr:copy-of-huge-original -resize '1024x768' -write sample-1024x768.jpg +delete \
mpr:copy-of-huge-original -resize '800x600' -write sample-800x600.jpg +delete \
mpr:copy-of-huge-original -resize '400x300' -write sample-400x300.jpg +delete \
mpr:copy-of-huge-original -resize '200x150' -write sample-200x150.jpg +delete \
mpr:copy-of-huge-original -resize '163x163!>' -write sample-163x163.jpg x.jpg
I found I needed the extra x.jpg at the end, I'm not sure why. On this machine (E5-1650 3.2 GHz, IM 6.8.9-9) I see:
$ time ./m.sh
real 0m6.560s
user 0m31.908s
sys 0m0.264s
peak RES 711MB
This is (I think) the equivalent with vipsthumbnail:
img=/data/john/pics/wtc.jpg
icc=/usr/share/color/argyll/ref/sRGB.icm
for size in 1024x768 800x600 400x300 200x150 163x163; do
vipsthumbnail $img --size $size --eprofile $icc -o vips-sample-$size.jpg[Q=80]
done
vipsthumbnail defaults to Lanczos3. Timing it with vips-8.4.4 I see:
$ time ./n.sh
real 0m2.376s
user 0m2.412s
sys 0m0.108s
peak RES 70MB
So a useful speedup, and a large drop in memory use.
Because memory use is low, you can run many vipsthumbnail in parallel without killing your server. If I change the script to be:
img=/data/john/pics/wtc.jpg
icc=/usr/share/color/argyll/ref/sRGB.icm
parallel vipsthumbnail $img \
--size {} --eprofile $icc -o vips-sample-{}.jpg[Q=80] ::: \
1024x768 800x600 400x300 200x150 163x163
I now see:
$ time ./n.sh
real 0m0.593s
user 0m1.960s
sys 0m0.100s
peak RES 280MB
More than 10x faster than ImageMagick.

Answer 3

For my point of view, and after testing, resizing 1024x768 to 800x600 is bad for rescaling algorithm.
The next resize are more easier, because of multiple of integer (2).
So, for quality reason, I personnaly thing, this is better :
convert sample_image.jpg -resize 1024x768 sample_image-1024x768.jpg
convert sample_image.jpg -resize 800x600 sample_image-800x600.jpg
convert sample_image-800x600.jpg -resize 400x300 sample_image-400x300.jpg
convert sample_image-400x300.jpg -resize 200x150 sample_image-200x150.jpg

Answer 4

15MB JPGs are really large. I would first resize it to reasonable size (say 2500x2500) with fastest "-sample" parameter and this smaller image then resize to different thumbnails.
You can make intelligent decision based on image size and choose the right way of resize.
I would recommend to focus on thumbnail quality instead of conversion speed so please look at different filters (-filter), sharping (-unsharp) and recommended downsampling methods

Answer 5

I'm thumbnailing ~50MB JPG files. The one option which made the biggest difference (~5x speedup) was "-define jpeg:size 128x128" before the input filename. The example given here:
http://www.imagemagick.org/Usage/formats/#jpg_read
...made a huge difference:
convert -define jpeg:size=128x128 jpeg_large.jpg -thumbnail 64x64 jpeg_thumbnail.jpg
-define jpeg:size allows ImageMagick to read in only as much data as it needs from disk, which greatly reduces load time for very large images.
As the linked page suggests, use a jpeg:size= of twice your final thumbnail size to avoid aliasing.
The -thumbnail option, described here:
http://www.imagemagick.org/Usage/resize/#thumbnail
...samples and strips the image, further speeding up the process.