ImageMagick - How can I calculate disk cache and memory usage? - imagemagick

I am trying to resize ten 21 GiB PSB-images on Windows 10 using ImageQuick 7:
Version: ImageMagick 7.0.7-31 Q16 x64 2018-05-07 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2018 ImageMagick Studio LLC
License: http://www.imagemagick.org/script/license.php
Visual C++: 180040629
Features: Cipher DPC Modules OpenMP
Delegates (built-in): bzlib cairo flif freetype gslib heic jng jp2 jpeg lcms lqr lzma openexr pangocairo png ps raw rsvg tiff webp xml zlib
Image details:
d:\magick identify -verbose extreme.psb
Image: extreme.psb
Format: PSB (Adobe Large Document Format)
Class: DirectClass
Geometry: 70866x53291+0+0
Resolution: 6000x6000
Print size: 11.811x8.88183
Units: PixelsPerInch
Colorspace: sRGB
Type: TrueColor
Base type: Undefined
Endianess: MSB
Depth: 16-bit
Channel depth:
Red: 16-bit
Green: 16-bit
Blue: 16-bit
Channel statistics:
Pixels: 3776520006
[...]
I did some math using ImageMagick's formula (under "Cache Storage and Resource Requirements"):
width * height * bit-depth / 8 * channels
which would make the cache file 30,212,160,048 bytes large.
Then trying to resize the image:
d:\magick extreme.psb -resize 95% extreme.tiff
But, it still halts (after 15 minutes) with a message saying:
magick: unable to extend cache 'extreme.psb': No space left on device # error/cache.c/OpenPixelCache/3673.
The SSD-disk has 48 GiB available for cache - 60 GiB including the original image. Even if the cache file is 30 GiB, it would still have 18 GiB left.
How can I foresee the total size necessary to process (any) images?
On a side note: As you can see, I was able to run identify -verbose on it, which also created a 21 GiB temporary file. So I am thinking that there might be more temporary files than meets the eye.

Related

Why did ImageMagick reduce my JPG quality severely when increasing saturation?

My command was convert imagename.jpeg -quality 100 -modulate 100,200 imagename-ImageMagick.jpeg and if you look at the Imgur gallery here:
https://imgur.com/a/V8zr4Lu
you can see that it has tons of JPG artefacts, to the point where the text is almost illegible. All I wanted to do was increase colour saturation.
Version info:
Version: ImageMagick 6.9.11-21 Q16 x86_64 2020-07-25 https://imagemagick.org Copyright: © 1999-2020 ImageMagick Studio LLC License: https://imagemagick.org/script/license.php Features: Cipher DPC Modules Delegates (built-in): bzlib djvu fftw fontconfig freetype gslib heic jbig jng jp2 jpeg lcms ltdl lzma openexr png ps raw tiff webp x xml zlib
Because when you increase saturation that much you make the existing JPEG artifacts more visible (in fact you make all the image problems more visible...). Adding saturation is increasing the tone differences, while the JPEG compression is based on the idea that the tone differences it creates are still below your perception, so increased saturation makes the existing and barely noticeable artifacts suddenly very visible.
-modulate 100,200 is a very aggressive setting, try something lighter
-quality 100 is pointless, you are just saving existing artifacts with a high accuracy.
maybe try a very light blur before increasing saturation.
Using Imagemagick 6.9.12.77, it looks fine to me
convert galaxy.jpg -modulate 100,200,100 -quality 100 x.jpg
Perhaps you need to upgrade either or both Imagemagick or libjpg
convert -version
Version: ImageMagick 6.9.12-77 Q16 aarch64 17704 https://legacy.imagemagick.org
Copyright: (C) 1999 ImageMagick Studio LLC
License: https://imagemagick.org/script/license.php
Features: Cipher DPC Modules
Delegates (built-in): bzlib cairo djvu fftw fontconfig freetype gslib gvc heic jbig jng jp2 jpeg lcms lqr ltdl lzma openexr png ps raw rsvg tiff webp x xml zlib

LZW compression creating huge files in ImageMagick

I'm using a recent version of ImageMagick on Windows 10 to resize and compress .tif files with mogrify *.tif -resize 20% -compress lzw. The input files are all the same dimensions and weigh a few hundred to a thousand KB, but the output files are all the same size, around 138 MB. This suggests to me that the mogrified files are being stored uncompressed. Using mogrify *.tif -compress zip leaves the files at their inflated size.
Why is the compression step failing?
Output of magick -version:
Version: ImageMagick 7.1.0-35 Q16-HDRI x64 ff6dbbb:20220516 https://imagemagick.org
Copyright: (C) 1999 ImageMagick Studio LLC
License: https://imagemagick.org/script/license.php
Features: Cipher DPC HDRI Modules OpenCL OpenMP(2.0)
Delegates (built-in): bzlib cairo flif freetype gslib heic jng jp2 jpeg jxl lcms lqr lzma openexr pangocairo png ps raqm raw rsvg tiff webp xml zip zlib
Compiler: Visual Studio 2022 (193231328)
The sequence of commands is important: success was obtained with mogrify -resize 20% -compress lzw *.tif.

Why would converting CMYK image to sRGB for display on web work locally, but not on prod server?

Our graphic artist is delivering image ready for print in CMYK colorspace JPG format, which is correct for printing. I am converting these for display on the web. I understand the "best" thing to do is convert the src.jpg to sRGB colorspace. I've tried both command line using convert and Imagick approach using transformImageColorspace() and find it works fine on my local machine (Fedora 32 Linux) but doesn't work on the production server (CentOS 7.3 Linux).
We've had this process working for a couple of years on production using the command line convert but we recently tightened security and disabled exec from PHP, so I'm trying to make the same process work via the Imagick bindings, but the resultant image doesn't display correctly. I don't know if the ICC color profile isn't being applied properly, or the conversion to sRGB is going wrong, or whatever else may be happening.
Local machine:
Version: ImageMagick 6.9.10-86 Q16 x86_64 2020-01-31 https://imagemagick.org
Prod server:
Version: ImageMagick 6.9.11-19 Q16 x86_64 2020-06-15 https://imagemagick.org
Both machines show 'lcms' is in the Delegates (built-in) list, which is one thing that may cause trouble but looks fine.
Source image: src.jpg in CMYK format, a mostly grey/white image.
Loading this image in Gnome's Image Viewer shows incorrect colours, a blueish tinge, but I think this is because Gnome doesn't handle the CMYK properly. Loading the same image into GIMP auto-converts it to sRGB and it shows grey/white properly. My aim is to produce output file src_srgb.jpg that displays grey/white when loaded in Image Viewer (so it clearly is a simple RGB profile image).
Source code:
class Logger {
public function log ($str) {
echo $str . "\n";
}
}
$l = new Logger();
$master_filename_abs = 'srgb/src.jpg';
$icc_profile_file = 'AdobeRGB1998.icc';
$pathinfo = pathinfo($master_filename_abs);
// e.g. turn mickey.jpg into mickey_srgb.jpg
$preview_filename = $pathinfo['filename'] . '_srgb.' . $pathinfo['extension'];
$preview_filename_abs = $pathinfo['dirname'] . '/' . $preview_filename;
$useCmdLine = false; // use cmd line 'convert', or Imagick?
try {
if ($useCmdLine) {
$cmd = 'convert "' . $master_filename_abs . '" -verbose' .
' -profile "' . $icc_profile_file . '"' .
' -colorspace srgb' .
' -resize 800 "' . $preview_filename_abs . '" 2>&1';
$l->log('executing "' . $cmd . '"...');
exec($cmd, $output, $return_var);
$l->log("exec returned $return_var with " . count($output) . ' lines of output:' .
join("\n", $output));
} else {
$master_img = new Imagick();
$master_img->readImage($master_filename_abs);
$icc_profile_contents = file_get_contents($icc_profile_file);
$l->log('About to profileImage()...');
$r = $master_img->profileImage('icc', $icc_profile_contents);
$l->log("profileImage returned $r " . ($r == TRUE ? 'true' : 'false'));
$space = $master_img->getImageColorspace();
$l->log("Colorspace starts as $space, CMYK is " . Imagick::COLORSPACE_CMYK);
$r = $master_img->transformImageColorspace(imagick::COLORSPACE_SRGB);
$l->log("transformImageColorspace returned $r " . ($r == TRUE ? 'true' : 'false'));
$space = $master_img->getImageColorspace();
$l->log("Colorspace after conversion is $space, SRGB is " . Imagick::COLORSPACE_SRGB);
$master_img->scaleImage(800, 0);
$l->log('Saving RGB profile version to ' . $preview_filename_abs);
$master_img->writeImage($preview_filename_abs);
}
} catch (Exception $ex) {
$l->log('Caught ' . $ex);
}
Prerequisites: the src.jpg should be in 'srgb' directory, and AdobeRGB1998.icc profile should be in current directory.
As you can see there's both the command line attempt, and the Imagick attempt, in the script, with a $useCmdLine boolean that toggles between them.
Result on local machine: src_srgb.jpg is created and looks OK. Colours match the src.jpg image.
$ /opt/remi/php71/root/bin/php imagick_srgb.php
About to profileImage()...
profileImage returned 1 true
Colorspace starts as 13, CMYK is 12
transformImageColorspace returned 1 true
Colorspace after conversion is 13, SRGB is 13
Saving RGB profile version to srgb/src_srgb.jpg
identify -verbose srgb/src_srgb.jpg shows:
Colorspace: sRGB
Type: TrueColor
...
Profiles:
Profile-8bim: 6694 bytes
Profile-exif: 4617 bytes
Profile-icc: 560 bytes
Profile-iptc: 17 bytes
unknown[2,0]:
Image Name[2,5]: Print
Profile-xmp: 16695 bytes
Result on prod server: src_srgb.jpg is created but looks blueish.
$ /opt/cpanel/ea-php73/root/usr/bin/php imagick_srgb.php
About to profileImage() with 1 bytes of profile data...
profileImage returned 1 true
Colorspace starts as 12, CMYK is 12
transformImageColorspace returned 1 true
Colorspace after conversion is 13, SRGB is 13
Saving RGB profile version to srgb/src_srgb.jpg
identify -verbose srgb/src_srgb.jpg shows:
Colorspace: sRGB
Type: TrueColor
...
Profiles:
Profile-8bim: 6694 bytes
Profile-exif: 4617 bytes
Profile-iptc: 17 bytes
unknown[2,0]:
Image Name[2,5]: Print
Profile-xmp: 16695 bytes
I see the 'good' version on local machine has Profile-icc in the list of profiles, whereas the 'bad' version from prod does not. Could this be connected to the root cause of the problem, and if so, what can I do about it? I believe exif, iptc and xmp profiles are metadata and can be safely ignored for the purpose of this test.
From a little trial and error I think I've got the transformImageColorspace and profileImage calls in the right order.
Advice would be hugely appreciated.
I cannot share the full src.jpg but here is a cropped part of it (cropped with imagemagick convert to retain its colour profile etc)
Edit Reading PHP Imagick won't add ICC color profile I thought I'd demonstrate production server's lcms presence:
$ convert -list configure | grep DELEGATES
DELEGATES bzlib djvu mpeg fftw fontconfig freetype gslib heic jbig jng jpeg lcms lzma openexr openjp2 pango png raqm raw rsvg tiff webp wmf x xml zlib zstd
DELEGATES bzlib cairo djvu fftw fontconfig freetype gslib gvc heic jbig jng jp2 jpeg lcms ltdl lzma openexr pangocairo png ps raqm raw rsvg tiff webp wmf x xml zlib
$ rpm -qa | grep cms
lcms2-devel-2.6-3.el7.x86_64
lcms2-2.6-3.el7.x86_64
Edit 2 There is something weird about imagemagick on the two systems. My local machine reports an icc profile on src.jpg, but production does not, for exactly the same image. I tried downloading my own attachment to this post as SO.jpg to check, copied it to prod, and diffed the output of identify -verbose SO.jpg:
$ diff so_local so_prod
1c1,2
< Image: SO.jpg
---
> Image:
> Filename: srgb/SO.jpg
12c13
< Endianess: Undefined
---
> Endianness: Undefined
83,84d83
< Profiles:
< Profile-icc: 557168 bytes
86,89c85,86
< date:create: 2020-06-17T06:14:10+00:00
< date:modify: 2020-06-17T06:14:10+00:00
< icc:copyright: Copyright 2000 Adobe Systems, Inc.
< icc:description: U.S. Web Coated (SWOP) v2
---
> date:create: 2020-06-17T06:14:59+00:00
> date:modify: 2020-06-17T06:14:59+00:00
94c91
< filename: SO.jpg
---
> filename: srgb/SO.jpg
99c96
< Pixels per second: 25.3889MB
---
> Pixels per second: 27.3785MB
102c99
< Version: ImageMagick 6.9.10-86 Q16 x86_64 2020-01-31 https://imagemagick.org
---
> Version: ImageMagick 6.9.11-19 Q16 x86_64 2020-06-15 https://imagemagick.org
This really stinks for me, with little understanding of imagemagick. I will look into downgrading ImageMagick packages on production to match local but we're getting into daytime operating hours on a production server so I may not be able to.
Edit 3 Going back over our historical jobs, it seems to have stopped working on June 1 2020, and on that day I see that ImageMagick-6.9.11.14-1.el7.remi.x86_64 was upgraded to ImageMagick-6.9.11.16-1.el7.remi.x86_64 (and the -libs and -devel packages too). So, it looks like this breaking change was introduced between 6.9.11.14-1 and 6.9.11.16-1. The remi repo we get them from only seems to hold two most recent, 6.9.11.18-1 and 6.9.11.19-1 and downgrading to 6.9.11.18-1 does not fix it for me.
Edit 4 May have got PHP Imagick bindings to do the conversion successfully. From a comment at https://www.imagemagick.org/discourse-server/viewtopic.php?p=67114#p67114 "you should define the colorspace CMYK before , apply whatever operation you need and then define the RGB colorspace" with the example convert -colorspace CMYK mona_lisa.cmyk.jpg -modulate 110 -colorspace RGB monalisa_rgb.jpg .. so I tried it in a desperate attempt (first setting profile to null to remove any existing ones):
$r = $master_img->profileImage('icc', null);
$r = $master_img->profileImage('icc', file_get_contents($cmyk_icc_profile_file));
$r = $master_img->profileImage('icc', file_get_contents($srgb_icc_profile_file));
$r = $master_img->transformImageColorspace(imagick::COLORSPACE_SRGB);
To my great surprise, with my local version 6.9.10-86 and old version 6.9.11-14 on prod, the output file appears correct in Gnome Image Viewer and web browser, so I think that's one problem solved. I've still not figured out why the latest version of ImageMagick 6.9.11-19 doesn't seem to work, will keep at it.
Edit 5 The ImageMagick guys accepted and fixed my bug report :) https://github.com/ImageMagick/ImageMagick6/issues/83

How to prevent a dark edge with transparency using montage?

I'm trying to montage some semi-transparent icons into one image using imagemagicks montage tool.
Unfortunately, after every operation with alpha a bit of black is showing everywhere on anti-aliased edges.
For example, these two icons:
and
After issuing
montage 1.png 2.png -background none montage.png
Result in:
I've tried numerous combinations of alpha, background and changing option order and haven't found a way to prevent this. Any idea?
My system is windows 8.1+cygwin, montage version is:
$ montage -version
Version: ImageMagick 6.9.1-3 Q16 x86_64 2015-07-01
http://www.imagemagick.org Copyright: Copyright (C) 1999-2015
ImageMagick Studio LLC License:
http://www.imagemagick.org/script/license.php Features: DPC OpenMP
Delegates (built-in): autotrace bzlib cairo fftw fontconfig freetype
fpx gslib jbig jng jpeg lcms lzma pangocairo png ps rsvg tiff webp x xml zlib
This looks like a version-specific bug, as the given command works fine on ImageMagick 6.9.1-10 and also using:
convert 1.png 2.png +append 3.png

lcms error using ImageMagick compare on JPEGs with colour profile

When running compare
compare -metric MSE some-image-1.jpg some-image-2.jpg /dev/null
1) When images are identical (OK)
0 (0)
This is what I would expect to see.
2) When images differ (OK)
Error: Command failed: 13.1266 (0.000200299)
I can get the data that I want, but I'm not sure why it's presented as an error.
3) When image is a JPEG with a Photoshop colour profile (ISSUE)
Error: Command failed: lcms: Error #12288; Corrupted memory profile
This is the real issue, and only happens on JPEG images with a Photoshop colour profile - can anyone point me to why this would happen?
Environment
OS X 10.9.4
GhostScript 9.07 at /opt/Ghostscript
compare at /opt/ImageMagick/bin/compare
Version: ImageMagick 6.8.8-6 Q16 x86_64 2014-02-17 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2014 ImageMagick Studio LLC
Features: DPC
Delegates: bzlib fftw jng jpeg lcms lzma png tiff webp xml zlib
Thank you for your time.

Resources