Open an image using as little RAM as possible - delphi

I want to open some relative big files (jpg, gif, bmp) using as little RAM as possible.
Inside my program I need all open files converted to BMP so I can process them. However the conversion from JPG to BMP takes 27.1MB of RAM if I use the classic conversion code:
function ConvertJPG2BMP(FullFileName: string; BMP: TBitmap);
VAR JPG: TJpegImage;
begin
JPG:= TJpegImage.Create;
TRY
TRY
JPG.LoadFromFile(FullFileName);
BMP.Assign(JPG);
EXCEPT
END;
FINALLY
FreeAndNil(JPG);
end;
end;
because it uses two images (a jpeg that is transferred to a bitmap then).
--
However, if I use a TPicture to load the file, I use only 7.1MB of RAM. But in this case the TPicture.Bitmap is empty and I need a valid TBitmap object.
Is there any way to load images from disk while keeping the mem footprint small?
--
(Test file: 1.JPG 2.74MB 3264x1840 pix)

Back of the envelope calculation gives 6 mega pixels. Assuming 32 bit colour this takes you to 24MB.
You aren't going to do any better than your current code.

The memory usage does not come from the JPEG library, but in the way you use it.
If you convert a JPEG into a TBitmap, it will create a bitmap resource, then uncompress the JPEG into the bitmap memory buffer.
You can paint directly from the JPEG content into the screen. Depending on the JPEG implementation, it will use (or not) a temporary TBitmap.
You are not tied to the JPEG unit supplied by Borland.
For instance, you may try calling directly the StretchDIBits() windows API from the uncompressed memory buffer, as such (this code is extracted from our SSE JPEG decoder):
procedure TJpegDecode.DrawTo(Canvas: TCanvas; X, Y: integer);
var BMI: TBitmapInfo;
begin
if #self=nil then
exit;
ToBMI(BMI);
StretchDIBits(Canvas.Handle,X,Y,width,height,0,0,width,height,pRGB,
BMI,DIB_RGB_COLORS,SrcCopy);
end;
Creating a huge bitmap is sometimes not possible (at least under Windows XP), because it uses shared GDI resources, whereas using plain RAM and StretchDIBits will always work, even for huge content. You can create a memory mapped file to handle the binary content, but just allocating the memory at once would suffice (and Windows will use hard drive only if short of RAM). With today's PCs, you should have enough RAM available even for big pictures. (17 MB is not a big deal, even for your 3264x1840 pix).
Then, from this global uncompressed memory buffer containing raw pixel triplets, you can use
a smaller bitmap corresponding to a region of the picture, then work on the region using StretchDIBits(aBitmap.Handle,.... It will use less GDI resource.
You could also rely for instance on GDI+ drawing, which will draw it without any temporary bitmap. See e.g. this OpenSource unit. From our testing, it's very fast and can be used without any TBitmap. You could also ask only for a region of the whole picture, and draw it using GDI+ on your bitmap canvas: this will use less RAM. And your exe will be a bit smaller than the default JPEG unit. And you'll be able to display and save not only JPEG, but GIF and TIFF formats.
If you want to minimize even further the memory usage, you'll have to call directly the JPEG library, at lowest-level. It's able to uncompress only a region of the JPEG. So you would be able to minimize used RAM. You may try using the IJL library with Delphi, a bit old, but still working.

Related

DELPHI : What is the best way to compress max and to keep quility of BITMAP

I have a function that I made for compressing bitmap to JPEG.
procedure BmptoJpeg(Bmp : TBitmap; JpgStream: TStream);
var
Jpg: TJPEGImage;
begin
Jpg := TJPEGImage.Create;
try
Jpg.Assign(Bmp);
Jpg.CompressionQuality := 10;
Jpg.Compress;
Jpg.SaveToStream(JpgStream);
finally
Jpg.Free;
end;
end;
But this is not suitable for good quality of Original Bitmap.
Is there another way to keep quality and to compress max Bitmap?
Another method without compressing to JPEG is also good for me..
Only compressing max and keeping quality is my target
I would say to use PNGs instead (which can be lossy or lossless), but it turns out I've had real problems with bugs with them in newer versions of Delphi relating to Alpha. Bitmaps don't support Alpha anyway though so... go figure.
But assuming you want to keep it a TBitmap at its core, and assuming you're using a not terribly outdated version of Delphi,
you can compress the bitmap using TCompressionStream. TCompressionStream can be used on any kind of data actually and used in conjuction with TFileStream or TMemoryStream or any other kind of TStream derivative for the most part.
For this you'll create a TCompressionStream object and use TBitmap.SaveToSTream() to save the bitmap to the TCompressionStream. The result will be a lossless version of your bitmap data.

Large Bitmap on Delphi

I want to create large Bitmap with code
LargeBmp := TBitmap.Create;
try
LargeBmp.Width := 1000; // fine
LargeBmp.Height := 15000; // XP - EOutOfResources, Not enough memory, Win 7 - works fine
...
finally
FreeAndNil(LargeBmp);
end;
This code raises an EOutOfResources exception with message "Not enough memory" on Windows XP but works fine in Windows 7.
What is wrong? Why Not enough memory? It's only 60 MB.
Set the pixel format like this:
LargeBmp.PixelFormat := pf24Bit;
I had the same problem several times and that always solved it.
As was discussed already, if you don't set the pixel format Windows will see it as a device-dependent bitmap. When you set the pixel format you create a DIB (device-independent bitmap). This means it's independent of the display device (graphics card).
I've had this same problem, and wanted to point out that pf24bit is not the only option. In the Graphics unit, you also have:
TPixelFormat = (pfDevice, pf1bit, pf4bit, pf8bit, pf15bit, pf16bit, pf24bit, pf32bit, pfCustom);
For a project I'm working on, I found the 8 bit option worked the best for what I needed, since I had a very large bitmap (high resolution), but limited colors (I was creating the whole bitmap from some simple code).
So try out a few others, other than just pf24bit, to find what's optimal for your production environment. I saved quite a bit of internal memory with the pf8bit option.
Created bitmaps (by default) is stored in some buffer. That buffer's size depends on videodriver, os and God knows what else.
This buffer can be pretty small (about 20-25mb) and if you will try to create more, it will fail.
To avoid this try to create DIB instead of TBitmap, or try to change Pixelformat to pf24bit. This will tell system to create Bitmap in user's memory instead of GDI buffer.
Now, why it not fails in win7 you ask? Ok, probably cause there is no GDI, but GDI+ and Direct2D uses in win 7 instead. Maybe other driver's version, dunno.

"Not enough storage" error while working with bitmaps

I get an "EOutofresources - Not enough storage" when I try to work with BMP files when I try to set BMP.Height or BMP.Width. Imediatelly after these instructions, the stack trace is (in this order):
ntdll.dll.RtlLeaveCriticalSection, kernel32.dll.FileTimeToDosDateTime, GDI32.dll.GdiReleaseDC, GDI32.dll.PatBlt, kernel32.dll.ReadFile or like this:
|7E429130|user32.dll GetParent
|7C90FF2D|ntdll.dll RtlGetNtGlobalFlags
|77F15A00|GDI32.dll GdiReleaseDC
|7C83069E|kernel32.dll FileTimeToDosDateTime
|7C9010E0|ntdll.dll RtlLeaveCriticalSection
| |my function (where I set BMP.Height or BMP.Width)
At a moment I was sure that it has to do something with memory fragmentation - the system had enough free ram to process my image BUT the memory was fragmented so there was no block large enough to hold my image. But then I have seen it happening once 11 seconds after Windows start up. My program cycled through the loop where I process the images ONLY once! So, this could not be related to RAM fragmentation.
A different situation (but still related to drawing) when I got this error is below:
|77F16A7E|GDI32.dll IntersectClipRect
|77F16FE5|GDI32.dll BitBlt
|7E429011|user32.dll OffsetRect
|7E42A97D|user32.dll CallWindowProcA
|7E42A993|user32.dll CallWindowProcA
|7C9010E0|ntdll.dll RtlLeaveCriticalSection
|7E4196C2|user32.dll DispatchMessageA
|7E4196B8|user32.dll DispatchMessageA
|0058A2E1|UTest.exe UTest.dpr
|7C90DCB8|ntdll.dll ZwSetInformationThread
I think there is always a 'RtlLeaveCriticalSection' call in the stack trace after BMP.Height.
There is this post pointing to a possible solution by editing a Windows registry key. However, the post says that it applies only to Win XP. While my error appears also on Win 7.
I see many similar posts (some of them are close connected to saving a file to disk) but until nobody came back to report that he fixed the error.
Update:
As you requested, this is the code where the error appears:
procedure TMyBitmap.SetLargeSize(iWidth, iHeight: Integer);
CONST ctBytesPerPixel= 3;
begin
{ Protect agains huge/empty images }
if iWidth< 1 then iWidth:= 1 else
if iWidth> 32768 then iWidth:= 32768;
if iHeight< 1 then iHeight:= 1 else
if iHeight> 32768 then iHeight:= 32768;
{ Set image type }
if iWidth * iHeight * ctBytesPerPixel > 9000000 {~9MB}
then HandleType:= bmDIB { Pros and cons: -no hardware acceleration, +supports larger images }
else HandleType:= bmDDB;
{ Total size is higher than 1GB? }
if (iWidth* iHeight* ctBytesPerPixel) > 1*GB then
begin
Width := 8000; { Set a smaller size }
Height := 8000; { And rise an error }
RAISE Exception.Create('Image is too large.');
end;
{ Set size }
Width := iWidth; <----------------- HERE
Height:= iHeight;
end;
From my experiment, the maximum bitmap size depends on:
The OS version (e.g. XP seems to allow smaller bitmap resources than Seven);
The OS edition (64 bit OS allows bigger resource allocation than 32 bit OS);
The current RAM installed (and free);
The number of bitmaps already allocated (since those are shared resources).
So you can not be sure that a bitmap allocation will be successful, when you start working with huge data (more than an on-screen bitmap resolution).
Here are some potential solutions (I've used some of them):
Allocate not a bitmap resource, but a plain memory block to work with, then use direct Win32 BitBlt API to draw it - but you'll have to write some dedicated process functions (or use some third party libraries), and on 32 bit OS, IMHO VirtualAlloc API (the one called by FastMM4 for big blocks of memory) won't be able to allocate more than 1 GB of contiguous memory;
Enhancement of the previous version: either use a 64 bit process to handle huge RAM block (welcome XE2 compiler), or use a file for temporary storage, then memory-map its content for processing (it is how PhotoShop or other handle huge memory) - if you have enough RAM, using a temporary file won't be necessary slower (no data will be written on disk);
Tile you big pictures into smaller pictures - the JPEG library is able to render only a portion of the picture, and it will fit easily into a bitmap resource;
In all cases, prevent any duplicated bitmap resource (since all bitmap resource are shared): for instance, if you are reading from a bitmap, copy its content into a temporary memory block or file, release its resource, then allocate your destination bitmap;
About performance, make it right, then make it fast - do not include "tricks" too early in your implementation (your customer may accept waiting some seconds, but won't accept a global failure).
There is no perfect solution (my preference is to use smaller pictures, because it has the advantage of being easily multi-threaded for the process, so may speed up a lot with new CPU), and be aware that resource allocation may work on your brand new 64 bit Windows Seven PC, but fail on the 32 bit XP computer of your customer.

Getting JPEG resolution without decoding the image

I am trying to get the resolution of a JPEG image without decoding the file. I got several samples from internet but none is working properly. It seems to be this way because many JPEG files are not standard, though any graphic application (Irfan, PSP, Firefox etc) can open them.
The header of a JPEG was supposed to be:
typedef struct _JFIFHeader
{
BYTE SOI[2]; /* 00h Start of Image Marker */
BYTE APP0[2]; /* 02h Application Use Marker */
BYTE Length[2]; /* 04h Length of APP0 Field */
BYTE Identifier[5]; /* 06h "JFIF" (zero terminated) Id String */
BYTE Version[2]; /* 07h JFIF Format Revision */
BYTE Units; /* 09h Units used for Resolution */
BYTE Xdensity[2]; /* 0Ah Horizontal Resolution */
BYTE Ydensity[2]; /* 0Ch Vertical Resolution */
BYTE XThumbnail; /* 0Eh Horizontal Pixel Count */
BYTE YThumbnail; /* 0Fh Vertical Pixel Count */
} JFIFHEAD;
However, when I looked into one of those non-standard files, the Xdensity and Ydensity fields were wrong. But again, all graphic applications can read this non-standard file.
Does anybody knows a piece of Delphi code that can actually read all JPEG files?
Delphi 7, Win 7 32 bit
I don't know about ALL JPEG files, but you will need to handle the two common file formats for JPEG. Since JPEG is a compression method and not a file format, the world at large has developed a few ways of storing JPEG image data in files. The two you are most likely to encounter are JFIF and EXIF. The above code covers JFIF, but doesn't handle EXIF. These two are largely incompatible but both are JPEG, so you'll need to detect and handle if you are using header information, as they defer.
For resolution, as an example. EXIF's field are x-Resolution and y-Resolution, vs the X/Y Density approach.
I would:
Do some reading on the two formats (JFIF and EXIF). I find
Wikipedia is a great place to start
on this reference (for some past
projects I've done), but SO most
likely has some great info on this
topic as well.
JFIF:
http://en.wikipedia.org/wiki/JPEG_File_Interchange_Format
EXIF:
http://en.wikipedia.org/wiki/Exif
Write code to detect the format using the starting headers
Handle each format independently
Wrap the whole thing so you can just toss a JPEG at it and get the
density. This will also give you a great spot to toss other helper code to deals with the "fun" world of JPEG handling
Here is some code which could help you get the data you want:
function GetJpegSize(jpeg: TMemoryStream; out width, height, BitDepth: integer): boolean;
var n: integer;
b: byte;
w: Word;
begin
result := false;
n := jpeg.Size-8;
jpeg.Position := 0;
if n<=0 then
exit;
jpeg.Read(w,2);
if w<>$D8FF then
exit; // invalid format
jpeg.Read(b,1);
while (jpeg.Position<n) and (b=$FF) do begin
jpeg.Read(b,1);
case b of
$C0..$C3: begin
jpeg.Seek(3,soFromCurrent);
jpeg.Read(w,2);
height := swap(w);
jpeg.Read(w,2);
width := swap(w);
jpeg.Read(b,1);
BitDepth := b*8;
Result := true; // JPEG format OK
exit;
end;
$FF:
jpeg.Read(b,1);
$D0..$D9, $01: begin
jpeg.Seek(1,soFromCurrent);
jpeg.Read(b,1);
end;
else begin
jpeg.Read(w,2);
jpeg.Seek(swap(w)-2, soFromCurrent);
jpeg.Read(b,1);
end;
end;
end;
end;
Units, Xdensity and Ydensity members of JPEG file header specifies unit of measurement used to describe physical dot density when a file is printed.
If Units is 1, Xdensity and Ydensity are dots per inch.
If Units is 2, Xdensity and Ydensity are dots per cm.
The point is that dot resolution (the scaled printing resolution) stored in an image file simply does not matter on the screen. Thus, Windows programs will always show you 96 logical ppi on the screen for any file. Note, some applications prefer using 72 logical ppi to display pictures on the screen, e.g. Adobe applications.
Graphics applications such as ACDSee, Adobe Photoshop, CorelDRAW, simply ignores Units, Xdensity and Ydensity members when displaying JPG files on the screen, but graphics applications consider the value of those members when printing JPG files if they exist. In case, a JPG file does not have Units, Xdensity and Ydensity members, graphics applications use their custom default values (usually 150 dpi) to print the JPG file.
So, for the question about a Delphi code that can read all JPEG header files, the answer is simple, just read JPG file header information; in case the optional members did not exist in a file, just ignore the optional members or tell end-users that they were currently not specified in the file.
Further Reading on DPI and PPI confusions
Say No to 72 dpi
A few scanning tips
References on JPEG File Format Specification
JPEG File Interchange Format File Format Summary
JPEG File Interchange Format, v1.02
There is a TP/TPW/Delphi (1-4, but will probably work till unicode versions without big mods) package, pasjp(e)g that can read most of the older JPG types (but not e.g. JPEG2000)
FPC also includes this package.
The original site from J. Nommsi has disappeared, but the package is still available, e.g. from
http://pascal.sources.ru/graph/pasjpg11.htm

Problem converting bmp to jpg using TJpegImage Component

I have a bmp file and am trying to convert it to jpeg format. The jpeg created using the following code loses a lot of clarity. I have tried tweaking many settings to no avail.
Does anyone have a function which will convert a bmp file to a jpeg?
var
Bmp: TBitmap;
Jpg: TJPEGImage;
begin
Bmp := TBitmap.Create;
Jpg := TJPEGImage.Create;
try
Bmp.LoadFromFile(BmpFileName);
Jpg.Assign(Bmp);
jpg.PixelFormat :=jf24bit; // or jf8bit
Jpg.CompressionQuality := 100;
Jpg.ProgressiveDisplay := False;
Jpg.ProgressiveEncoding := False;
Jpg.SaveToFile(JpgFileName);
finally
Jpg.Free;
Bmp.Free;
end;
end;
Update II A lot of people have responded that jpeg is not the graphic type to use in this case. Understood. Not to beat a dead horse, but I have been able to use other programs (i.e. Photoshop) and convert this to a nice looking jpeg. And the tool I am using to create the chart (fusioncharts) is able to export it to a nice looking jpeg too (see below). What is the difference?
If you want to loose as little quality as possible, then you should compress as little as possible, i.e. set CompressionQuality := 100.
Update
The JPG image format is designed to be used with photographs and similar raster pictures. JPG employs destructive compression to reduce the file size. However, the visual effect of this compression is negligible for large quality values (> ~80) because photographs do not contain large areas of the same colour, very regular shapes, etc.
The JPG format should never be used for anything other than photographic pictures. For instance, diagrams, screenshots, etc, that do contain large areas of the exact same colour, very regular shapes, etc, will display clear artefacts when JPG compressed. For this type of raster images, use PNG. PNG is also a compressed format (a very, very good format too!), but the compression is non-destructive.
So please do not use JPG for your diagram! Use PNG!
I had the same problem as you. It's my solution.
1) Use TsdJpegGraphic class from the NativeJPG library instead of Delphi's TJPEGImage. It's free and open source.
2) The code:
var
Bmp: TBitmap;
Jpg: TsdJpegGraphic;
begin
Bmp := TBitmap.Create;
Jpg := TsdJpegGraphic.Create;
try
Bmp.LoadFromFile(BmpFileName);
Jpg.CompressionQuality := 100; // !!!
Jpg.Assign(Bmp);
Jpg.SaveToFile(JpgFileName);
finally
Jpg.Free;
Bmp.Free;
end;
end;
It's important to set the CompressionQuality property before you assign the bitmap to Jpg! If you put this command after the assigning then the library will use the default value (=80%) for CompressionQuality.
As others have said, JPEG is simply the wrong format for that kind of image - and with the high-frequency chroma edges, you have possibly one of the worst-case images for JPEG!
I recommend you get a copy of Paint.Net, and try saving/previewing images in different formats (PNG, JPEG, GIF, BMP, etc) and get a feel for the trade-offs between image size, quality and performance yourself.
I've tried saving your test image with Paint.net as JPEG, and get exactly the same quality as Delphi gives you, so there's nowt wrong with what you (or Delphi) are doing...
UPDATE:
According to Wikipedia...
There is an optional lossless mode
defined in the JPEG standard; however,
that mode is not widely supported in
products.
I'd guess that FusionCharts and Photoshop have a way of specifying this mode. It's not normally used, because JPEG is usually considered synonymous with LOSSY compression.
In summary, while JPEG files can support lossless compression, JPEG is not an ideal format to because it's not widely supported. If you want lossless, PNG remains a better choice. The PNG-compressed version of your image is 30% smaller than the losslessly compressed JPEG one (71K vs 101K)
As Andreas said, jack up the CompressionQuality to 100. If you'd like to try another example, Dr. Bob (frequent SO contributor) has a nice writeup here:
https://web.archive.org/web/1/http://articles.techrepublic%2ecom%2ecom/5100-10878_11-5031886.html#
He has code that uses a stream approach. I would expect the same result, but you never know. Here's the code:
https://web.archive.org/web/1/http://techrepublic%2ecom%2ecom/html/tr/sidebars/5031886-1.html
You just need to call Compress method after assignment bmp to jpeg. This is necessary for the CompressionQuality parameter to take effect. When Delphi performs Jpg.Assign(Bmp) it only assigns TJPEGImage.FBitmap field.
var
Bmp: TBitmap;
Jpg: TJPEGImage;
begin
Bmp := TBitmap.Create;
Jpg := TJPEGImage.Create;
try
Bmp.LoadFromFile(BmpFileName);
Jpg.Assign(Bmp);
Jpg.CompressionQuality := 100;
Jpg.Compress; //!!!
Jpg.SaveToFile(JpgFileName);
finally
Jpg.Free;
Bmp.Free;
end;
end;
I don't think just setting jpg.CompressionQuality alone is enough.
You should also call jpg.Compress() afterwards to actually compress image...
Something like:
jpg := TJPEGImage.Create();
jpg.LoadFromFile('foo.jpg');
jpg.DIBNeeded();
jpg.CompressionQuality := 25;
jpg.Compress();
jpg.SaveToFile('bar.jpg');
jpg.Free();
HTH,
Dejan

Resources