ImageMagick convert OOMing inside Singularity container - imagemagick

When I run a specific ImageMagick convert command (to produce an animated GIF) inside a Singularity container it is consistently giving an memory error:
convert-im6.q16: cache resources exhausted `foo.gif' # error/cache.c/OpenPixelCache/4083.
convert-im6.q16: memory allocation failed `foo.gif' # error/quantize.c/AssignImageColors/496.
How can I debug this?
I'm not getting any memory issues outside of Singularity which gives me the impression that either Singularity is artificially limiting my available memory or ImageMagick is configured poorly inside of Singularity. I am not seeing memory issues for any of my other applications running inside Singularity which makes me think it's an ImageMagick issue.
Here are some details about my system:
$ singularity --version
singularity version 3.7.1-1.el8
Inside the container:
Singularity> convert --version
Version: ImageMagick 6.9.10-23 Q16 x86_64 20190101 https://imagemagick.org
Copyright: © 1999-2019 ImageMagick Studio LLC
License: https://imagemagick.org/script/license.php
Features: Cipher DPC Modules OpenMP
Delegates (built-in): bzlib djvu fftw fontconfig freetype jbig jng jpeg lcms lqr ltdl lzma openexr pangocairo png tiff webp wmf x xml zlib
and
Singularity> convert -list resource
Resource limits:
Width: 16KP
Height: 16KP
List length: 18.446744EP
Area: 128MP
Memory: 256MiB
Map: 512MiB
Disk: 1GiB
File: 768
Thread: 256
Throttle: 0
Time: unlimited
The Memory: 256MiB stands out as especially low. I should have at least 2GiB. How does ImageMagick configure these limits and how can I change them?

The problem turned out to be with ImageMagick's system-wide policy.xml that was installed in my container. Updating that file with more generous "memory" and "disk" values fixed this problem.
You can find the location of your system's policy.xml file using the command convert -list policy (Hat tip to Kurt Pfeifle's answer which clued me in on this). For me it was at /etc/ImageMagick-6/policy.xml. You can edit that file
(if you have root access). In the end I decided just to delete the file since I don't want my system from restricting my use at all inside the container.
You can set limits via the command line, say convert -limit memory 2GiB ... or environment variables (See Kurt Pfeifle's answer for details). However, this method does not allow expanding larger than system-wide limits set in policy.xml because this policy file is meant to be a way for system administrators to forcibly limit users. Therefore, the only way to remedy this is to update/remove the system-wide policy.

Related

Why does ImageMagick fail with "no decode delegate for this image format `PNG'" even when the png delegate is installed?

I'm working on openSUSE Tumbleweed.
I have ImageMagick installed.
$ convert --version
Version: ImageMagick 7.1.0-52 Q16-HDRI x86_64 20549 https://imagemagick.org
Copyright: (C) 1999 ImageMagick Studio LLC
License: https://imagemagick.org/script/license.php
Features: Cipher DPC HDRI Modules OpenMP(4.5)
Delegates (built-in): bzlib djvu fontconfig freetype gslib jng jpeg lcms ltdl lzma png ps raw tiff x xml zlib
Compiler: gcc (12.2)
Notice that the PNG delegate is installed, at least according to --version.
Despite this, converting fails.
$ convert covalent_platform.svg covalent_platform.png
convert: no decode delegate for this image format `PNG' # error/constitute.c/ReadImage/776.
convert: no images defined `covalent_platform.png' # error/convert.c/ConvertImageCommand/3342.
Even identifying a non-PNG file fails, with the same PNG delegate problem.
$ identify covalent_platform.svg
identify: no decode delegate for this image format `PNG' # error/constitute.c/ReadImage/776.
Any ideas for what might be wrong here?
I reinstalled libpng-devel and reinstalled IM, but the problem remains.
I figured out the problem, thanks to #fmw42 pointing out that if magick -list format has no output then the installation is bad.
I had installed IM from https://imagemagick.org/script/download.php
These downloads are not compatible with openSUSE.
After uninstalling and reinstalling IM from https://software.opensuse.org/package/ImageMagick, everything works as expected.

imagemagick wmf support on Alpine Linux

I am installing imagemagick on Alpine, which picks up 7.0.10 version of imagemagick.
My primary use is to covert WMF to PNG.
But convert sample.wmf sample.png gives error
convert: no decode delegate for this image format `WMF' # error/constitute.c/ReadImage/572.
convert: no images defined `sample.png' # error/convert.c/ConvertImageCommand/3322.
As per https://www.imagemagick.org/script/formats.php, I also installed libwmf, which does not resolve the issue.
identify -list format | grep WMF does not return any result.
Updated Answer
You are in luck! GraphicsMagick can do it on alpine:latest:
apk add graphicsmagick
gm identify -version
GraphicsMagick 1.3.36 20201226 Q16 http://www.GraphicsMagick.org/
Copyright (C) 2002-2020 GraphicsMagick Group.
Additional copyrights and licenses apply to this software.
See http://www.GraphicsMagick.org/www/Copyright.html for details.
Feature Support:
Native Thread Safe yes
Large Files (> 32 bit) yes
Large Memory (> 32 bit) yes
BZIP no
DPS no
FlashPix no
FreeType yes
Ghostscript (Library) no
JBIG no
JPEG-2000 no
JPEG yes
Little CMS no
Loadable Modules yes
Solaris mtmalloc no
Google perftools tcmalloc no
OpenMP no
PNG yes
TIFF yes
TRIO no
Solaris umem no
WebP yes
WMF yes <--- HERE IT IS
X11 no
XML yes
ZLIB yes
Now do the conversion from WMF to PNG:
gm convert sample.wmf result.png
Original Answer
I don't think you'll be able to do that, with ImageMagick at least...
I tried installing libwmf on alpine:latest and then installing ImageMagick from source and it declined to use libwmf v0.2.12
So I checked what ImageMagick requires and it wants libwmf v0.2.8.2.
So I tried alpine:3.8 which can install libwmf v0.2.8.4 but ImageMagick still wouldn't accept that (xxx/ipa.h is missing).
So I looked back to alpine:3.3 and alpine:3.4 and they have no libwmf.
So I tried alpine:3.5 and that was the same libwmf version as alpine:3.8
TLDR; alpine:3.5's libwmf is too new for ImageMagick and alpine:3.4 doesn't have libwmf at all.
Note: I found the packages and versions of libwmf on this website.

Why would converting CMYK image to sRGB for display on web work locally, but not on prod server?

Our graphic artist is delivering image ready for print in CMYK colorspace JPG format, which is correct for printing. I am converting these for display on the web. I understand the "best" thing to do is convert the src.jpg to sRGB colorspace. I've tried both command line using convert and Imagick approach using transformImageColorspace() and find it works fine on my local machine (Fedora 32 Linux) but doesn't work on the production server (CentOS 7.3 Linux).
We've had this process working for a couple of years on production using the command line convert but we recently tightened security and disabled exec from PHP, so I'm trying to make the same process work via the Imagick bindings, but the resultant image doesn't display correctly. I don't know if the ICC color profile isn't being applied properly, or the conversion to sRGB is going wrong, or whatever else may be happening.
Local machine:
Version: ImageMagick 6.9.10-86 Q16 x86_64 2020-01-31 https://imagemagick.org
Prod server:
Version: ImageMagick 6.9.11-19 Q16 x86_64 2020-06-15 https://imagemagick.org
Both machines show 'lcms' is in the Delegates (built-in) list, which is one thing that may cause trouble but looks fine.
Source image: src.jpg in CMYK format, a mostly grey/white image.
Loading this image in Gnome's Image Viewer shows incorrect colours, a blueish tinge, but I think this is because Gnome doesn't handle the CMYK properly. Loading the same image into GIMP auto-converts it to sRGB and it shows grey/white properly. My aim is to produce output file src_srgb.jpg that displays grey/white when loaded in Image Viewer (so it clearly is a simple RGB profile image).
Source code:
class Logger {
public function log ($str) {
echo $str . "\n";
}
}
$l = new Logger();
$master_filename_abs = 'srgb/src.jpg';
$icc_profile_file = 'AdobeRGB1998.icc';
$pathinfo = pathinfo($master_filename_abs);
// e.g. turn mickey.jpg into mickey_srgb.jpg
$preview_filename = $pathinfo['filename'] . '_srgb.' . $pathinfo['extension'];
$preview_filename_abs = $pathinfo['dirname'] . '/' . $preview_filename;
$useCmdLine = false; // use cmd line 'convert', or Imagick?
try {
if ($useCmdLine) {
$cmd = 'convert "' . $master_filename_abs . '" -verbose' .
' -profile "' . $icc_profile_file . '"' .
' -colorspace srgb' .
' -resize 800 "' . $preview_filename_abs . '" 2>&1';
$l->log('executing "' . $cmd . '"...');
exec($cmd, $output, $return_var);
$l->log("exec returned $return_var with " . count($output) . ' lines of output:' .
join("\n", $output));
} else {
$master_img = new Imagick();
$master_img->readImage($master_filename_abs);
$icc_profile_contents = file_get_contents($icc_profile_file);
$l->log('About to profileImage()...');
$r = $master_img->profileImage('icc', $icc_profile_contents);
$l->log("profileImage returned $r " . ($r == TRUE ? 'true' : 'false'));
$space = $master_img->getImageColorspace();
$l->log("Colorspace starts as $space, CMYK is " . Imagick::COLORSPACE_CMYK);
$r = $master_img->transformImageColorspace(imagick::COLORSPACE_SRGB);
$l->log("transformImageColorspace returned $r " . ($r == TRUE ? 'true' : 'false'));
$space = $master_img->getImageColorspace();
$l->log("Colorspace after conversion is $space, SRGB is " . Imagick::COLORSPACE_SRGB);
$master_img->scaleImage(800, 0);
$l->log('Saving RGB profile version to ' . $preview_filename_abs);
$master_img->writeImage($preview_filename_abs);
}
} catch (Exception $ex) {
$l->log('Caught ' . $ex);
}
Prerequisites: the src.jpg should be in 'srgb' directory, and AdobeRGB1998.icc profile should be in current directory.
As you can see there's both the command line attempt, and the Imagick attempt, in the script, with a $useCmdLine boolean that toggles between them.
Result on local machine: src_srgb.jpg is created and looks OK. Colours match the src.jpg image.
$ /opt/remi/php71/root/bin/php imagick_srgb.php
About to profileImage()...
profileImage returned 1 true
Colorspace starts as 13, CMYK is 12
transformImageColorspace returned 1 true
Colorspace after conversion is 13, SRGB is 13
Saving RGB profile version to srgb/src_srgb.jpg
identify -verbose srgb/src_srgb.jpg shows:
Colorspace: sRGB
Type: TrueColor
...
Profiles:
Profile-8bim: 6694 bytes
Profile-exif: 4617 bytes
Profile-icc: 560 bytes
Profile-iptc: 17 bytes
unknown[2,0]:
Image Name[2,5]: Print
Profile-xmp: 16695 bytes
Result on prod server: src_srgb.jpg is created but looks blueish.
$ /opt/cpanel/ea-php73/root/usr/bin/php imagick_srgb.php
About to profileImage() with 1 bytes of profile data...
profileImage returned 1 true
Colorspace starts as 12, CMYK is 12
transformImageColorspace returned 1 true
Colorspace after conversion is 13, SRGB is 13
Saving RGB profile version to srgb/src_srgb.jpg
identify -verbose srgb/src_srgb.jpg shows:
Colorspace: sRGB
Type: TrueColor
...
Profiles:
Profile-8bim: 6694 bytes
Profile-exif: 4617 bytes
Profile-iptc: 17 bytes
unknown[2,0]:
Image Name[2,5]: Print
Profile-xmp: 16695 bytes
I see the 'good' version on local machine has Profile-icc in the list of profiles, whereas the 'bad' version from prod does not. Could this be connected to the root cause of the problem, and if so, what can I do about it? I believe exif, iptc and xmp profiles are metadata and can be safely ignored for the purpose of this test.
From a little trial and error I think I've got the transformImageColorspace and profileImage calls in the right order.
Advice would be hugely appreciated.
I cannot share the full src.jpg but here is a cropped part of it (cropped with imagemagick convert to retain its colour profile etc)
Edit Reading PHP Imagick won't add ICC color profile I thought I'd demonstrate production server's lcms presence:
$ convert -list configure | grep DELEGATES
DELEGATES bzlib djvu mpeg fftw fontconfig freetype gslib heic jbig jng jpeg lcms lzma openexr openjp2 pango png raqm raw rsvg tiff webp wmf x xml zlib zstd
DELEGATES bzlib cairo djvu fftw fontconfig freetype gslib gvc heic jbig jng jp2 jpeg lcms ltdl lzma openexr pangocairo png ps raqm raw rsvg tiff webp wmf x xml zlib
$ rpm -qa | grep cms
lcms2-devel-2.6-3.el7.x86_64
lcms2-2.6-3.el7.x86_64
Edit 2 There is something weird about imagemagick on the two systems. My local machine reports an icc profile on src.jpg, but production does not, for exactly the same image. I tried downloading my own attachment to this post as SO.jpg to check, copied it to prod, and diffed the output of identify -verbose SO.jpg:
$ diff so_local so_prod
1c1,2
< Image: SO.jpg
---
> Image:
> Filename: srgb/SO.jpg
12c13
< Endianess: Undefined
---
> Endianness: Undefined
83,84d83
< Profiles:
< Profile-icc: 557168 bytes
86,89c85,86
< date:create: 2020-06-17T06:14:10+00:00
< date:modify: 2020-06-17T06:14:10+00:00
< icc:copyright: Copyright 2000 Adobe Systems, Inc.
< icc:description: U.S. Web Coated (SWOP) v2
---
> date:create: 2020-06-17T06:14:59+00:00
> date:modify: 2020-06-17T06:14:59+00:00
94c91
< filename: SO.jpg
---
> filename: srgb/SO.jpg
99c96
< Pixels per second: 25.3889MB
---
> Pixels per second: 27.3785MB
102c99
< Version: ImageMagick 6.9.10-86 Q16 x86_64 2020-01-31 https://imagemagick.org
---
> Version: ImageMagick 6.9.11-19 Q16 x86_64 2020-06-15 https://imagemagick.org
This really stinks for me, with little understanding of imagemagick. I will look into downgrading ImageMagick packages on production to match local but we're getting into daytime operating hours on a production server so I may not be able to.
Edit 3 Going back over our historical jobs, it seems to have stopped working on June 1 2020, and on that day I see that ImageMagick-6.9.11.14-1.el7.remi.x86_64 was upgraded to ImageMagick-6.9.11.16-1.el7.remi.x86_64 (and the -libs and -devel packages too). So, it looks like this breaking change was introduced between 6.9.11.14-1 and 6.9.11.16-1. The remi repo we get them from only seems to hold two most recent, 6.9.11.18-1 and 6.9.11.19-1 and downgrading to 6.9.11.18-1 does not fix it for me.
Edit 4 May have got PHP Imagick bindings to do the conversion successfully. From a comment at https://www.imagemagick.org/discourse-server/viewtopic.php?p=67114#p67114 "you should define the colorspace CMYK before , apply whatever operation you need and then define the RGB colorspace" with the example convert -colorspace CMYK mona_lisa.cmyk.jpg -modulate 110 -colorspace RGB monalisa_rgb.jpg .. so I tried it in a desperate attempt (first setting profile to null to remove any existing ones):
$r = $master_img->profileImage('icc', null);
$r = $master_img->profileImage('icc', file_get_contents($cmyk_icc_profile_file));
$r = $master_img->profileImage('icc', file_get_contents($srgb_icc_profile_file));
$r = $master_img->transformImageColorspace(imagick::COLORSPACE_SRGB);
To my great surprise, with my local version 6.9.10-86 and old version 6.9.11-14 on prod, the output file appears correct in Gnome Image Viewer and web browser, so I think that's one problem solved. I've still not figured out why the latest version of ImageMagick 6.9.11-19 doesn't seem to work, will keep at it.
Edit 5 The ImageMagick guys accepted and fixed my bug report :) https://github.com/ImageMagick/ImageMagick6/issues/83

convert wmf to jpg with graphicsmagick, unable to read font

I try to convert a wmf file into a jpg. gm says it is "Unable to read font (n021004l.pfb) [No such file or directory]." My command is like that:
gm convert 456.wmf 456.jpg
What could be wrong? I am using the latest gm version 1.3.29 and have ghostscript installed. OS: Windows 7
Here's the small file that I am trying to convert: https://mycloud.m-box.at/index.php/s/lBMeCG0cjK45sI1
And here's the version log (wmf is enabled):
GraphicsMagick 1.3.29 2018-04-29 Q16 http://www.GraphicsMagick.org/
Copyright (C) 2002-2018 GraphicsMagick Group.
Additional copyrights and licenses apply to this software.
See http://www.GraphicsMagick.org/www/Copyright.html for details.
Feature Support:
Native Thread Safe yes
Large Files (> 32 bit) yes
Large Memory (> 32 bit) yes
BZIP yes
DPS no
FlashPix no
FreeType yes
Ghostscript (Library) no
JBIG yes
JPEG-2000 yes
JPEG yes
Little CMS yes
Loadable Modules yes
OpenMP yes (200203)
PNG yes
TIFF yes
TRIO no
UMEM no
WebP yes
WMF yes
X11 no
XML yes
ZLIB yes
Windows Build Parameters:
MSVC Version: 1500
This looks a bit odd. I would suggest you enable debugging messages and see if you can trace what is going wrong yourself.
So, use:
gm convert -debug all 456.wmf 456.jpg > debug.txt 2>&1
and then look at the last few lines in debug.txt.
Sorry I can't help further!

lcms error using ImageMagick compare on JPEGs with colour profile

When running compare
compare -metric MSE some-image-1.jpg some-image-2.jpg /dev/null
1) When images are identical (OK)
0 (0)
This is what I would expect to see.
2) When images differ (OK)
Error: Command failed: 13.1266 (0.000200299)
I can get the data that I want, but I'm not sure why it's presented as an error.
3) When image is a JPEG with a Photoshop colour profile (ISSUE)
Error: Command failed: lcms: Error #12288; Corrupted memory profile
This is the real issue, and only happens on JPEG images with a Photoshop colour profile - can anyone point me to why this would happen?
Environment
OS X 10.9.4
GhostScript 9.07 at /opt/Ghostscript
compare at /opt/ImageMagick/bin/compare
Version: ImageMagick 6.8.8-6 Q16 x86_64 2014-02-17 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2014 ImageMagick Studio LLC
Features: DPC
Delegates: bzlib fftw jng jpeg lcms lzma png tiff webp xml zlib
Thank you for your time.

Resources