I have a MIDP application using GameCanvas, and I'm using this sequence to get scaled image as javax.microedition.lcdui.Image for rendering into the canvas:
Bitmap.createBitmapFromBytes(data from file)
bitmap.scaledInto(largerBitmap..)
largerBitmap.getARGB(..)
Image.createRGBImage(..)
The obvious disadvantage of this is that consumes a lot of memory, especially at the moment when pulling RGB data from (large) scaled bitmap to a (large) buffer to create Image from it. Performance is also not very good, either due to memory usage or wrong approach - dont' know.
Is there a more efficient way to do what I want? Thank you!
Related
This question's kind of a 4-parter:
Is it true that all BitmapData is immediately cached to the GPU as soon as it's created (even if it's never applied to a Bitmap or added to stage?)
Does this still happen if the GPU texture buffer is already full? Bonus points: if so, what's the preferential swap method the GPU chooses to select which textures to remove from memory?
If (1), then does setting the width/height of any BitmapData uncache it and/or does replacing its pixels therefore upload the new pixels to the same memory address on the GPU? Bonus: What if the size changes?
To bring this all together, would a hybrid class that extends BitmapData but stores its actual data in a ByteArray be able to use setPixels/getPixels on itself to control upload/download from the GPU as necessary, to buffer a large number of bitmaps? Bonus: Would speed improve for actually placing them in Bitmaps if the instances of this class were static?
Here are some answers
No. In AIR, you manually upload bitmaps to GPU and have control WHEN to do it
As far as I've reached, if the buffer is full, you simply get an error for it - the GPU cannot make a choice what do to. Removing a random texture won't be nice if it's important to you, right? :)
You can check for example Starling and how it uploads textures to GPU. Once you force it to do so, it doesn't care what you do with the bitmap. It's like making a photo image of an object so that you can just show it instead of explaining it with words. It won't matter if you change the object, the photo will be still the same.
Simplified answer: no. Again - it's best to check out how textures are created and how you upload stuff to GPU.
I want to allow the user to select a photo, without limiting the size, and then edit it.
My idea is to create a thumbnail of the large photo with the same size as the screen for editing, and then, when the editing is finished, use the large photo to make the same edit that was performed on the thumbnail.
When I use UIGraphicsBeginImageContext to create a thumbnail image, it will cause a memory issue.
I know it's hard to edit the whole large image directly due to hardware limits, so I want to know if there is a way I can downsample the large image to less then 2048*2048 wihout memory issues?
I found that there is a BitmapFactory Class which has an inSampleSize option which can downsample a photo in Android platform. How can this be done on iOS?
You need to handle the image loading using UIImage which doesn't actually load the image into memory and then create a bitmap context at the size of the resulting image that you want (so this will be the amount of memory used). Then you need to iterate a number of times drawing tiles from the original image (this is where parts of the image data are loaded into memory) using CGImageCreateWithImageInRect into the destination context using CGContextDrawImage.
See this sample code from Apple.
Large images don't fit in memory. So loading them into memory to then resize them doesn't work.
To work with very large images you have to tile them. Lots of solutions out there already for example see if this can solve your problem:
https://github.com/dhoerl/PhotoScrollerNetwork
I implemented my own custom solution but that was specific to our environment where we had an image tiler running server side already & I could just request specific tiles of large images (madea server, it's really cool)
The reason tiling works is that basically you only ever keep the visible pixels in memory, and there isn't that many of those. All tiles not currently visible are factored out to the disk cache, or flash memory cache as it were.
Take a look at this work by Trevor Harmon. It improved my app's performance.I believe it will work for you too.
https://github.com/coryalder/UIImage_Resize
I am not quite sure whether it is beneficial to draw the visual elements of my app with Core Graphics instead of providing the images. In terms of memory preservation and runtime speed which way is better ?
In terms of memory preservation and runtime speed which way is better?
+UIImage:imageNamed: is most efficient. It caches images, i.e. only one copy of an image is in memory and the image is decoded (from its PNG, JPEG, TIFF, etc. data) when it is needed and kept around for future reuse. If you are worried about memory use, iOS will purge the UIImage cache if you are running low or go into the background.
Using Core Graphics to draw an image does not do any caching for you, unless you write the code to draw your image into a context, save the context as a bitmap, cache the bitmap and then reuse it later on. So you end up drawing the same thing over and over every time it is needed. For example, if you override UIView's -drawRect: to draw imagery, then during animations it will be called for every single frame (60 times a second). This needlessly burns CPU cycles and battery life.
Bottom line is it depends on what your app is and does.
If you dont need your images to Change or animate much ,then you shoud directly use an image.Dont worry so much about performace unless you have like 100 images in a single view controller.
If iPhone can handle games like need for speed , to run an app with various images is an easy task.
Hope this helps.
In background thread, my application needs to read images from disk, downscale them to the size of screen (1024x768 or 2048x1536) and save them back to disk. Original images are mostly from the Camera Roll but some of them may have larger sizes (e.g. 3000x3000).
Later, in a different thread, these images will frequently get downscaled to different sizes around 500x500 and saved to the disk again.
This leads me to wonder: what is the most efficient way to do this in iOS, performance and memory-wise? I have used two different APIs:
using CGImageSource and CGImageSourceCreateThumbnailAtIndex from ImageIO;
drawing to CGBitmapContext and saving results to disk with CGImageDestination.
Both worked for me but I'm wondering if they have any difference in performance and memory usage. And if there are better options, of course.
While I can't definetly say it will help, I think it's worth trying to push the work to the GPU. You can either do that yourself by rendering a textured quad at a given size, or by using GPUImage and its resizing capabilities. While it has some texture size limitations on older devices, it should have much better performance than CPU based solution
With libjpeg-turbo you can use the scale_num and scale_denom fields of jpeg_decompress_struct, and it will decode only needed blocks of an image. It gave me 250 ms decoding+scaling time in background thread on 4S with 3264x2448 original image (from camera, image data placed in memory) to iPhone's display resolution. I guess it's OK for an image that large, but still not great.
(And yes, that is memory efficient. You can decode and store the image almost line by line)
What you said on twitter does not match your question.
If you are having memory spikes, look at Instruments to figure out what is consuming the memory. Just the data alone for your high resolution image is 10 megs, and your resulting images are going to be about 750k, if they contain no alpha channel.
The first issue is keeping the memory usage low, for that, make sure that all of the images that you load are disposed as soon as you are done using them, that will ensure that the underlying C/Objective-C API disposes the memory immediately, instead of waiting for the GC to run, so something like:
using (var img = UIImage.FromFile ("..."){
using (var scaled = Scaler (img)){
scaled.Save (...);
}
}
As for the scaling, there are a variety of ways of scaling the images. The simplest way is to create a context, then draw on it, and then get the image out of the context. This is how MonoTouch's UIImage.Scale method is implemented:
public UIImage Scale (SizeF newSize)
{
UIGraphics.BeginImageContext (newSize);
Draw (new RectangleF (0, 0, newSize.Width, newSize.Height));
var scaledImage = UIGraphics.GetImageFromCurrentImageContext();
UIGraphics.EndImageContext();
return scaledImage;
}
The performance will be governed by the context features that you enable. For example, a higher-quality scaling would require changing the interpolation quality:
context.InterpolationQuality = CGInterpolationQuality.High
The other option is to run your scaling not on the CPU, but on the GPU. To do that, you would use the CoreImage API and use the CIAffineTransform filter.
As to which one is faster, it is something left for someone else to benchmark
CGImage Scale (string file)
{
var ciimage = CIImage.FromCGImage (UIImage.FromFile (file));
// Create an AffineTransform that makes the image 1/5th of the size
var transform = CGAffineTransform.MakeScale (0.5f, 0.5f);
var affineTransform = new CIAffineTransform () {
Image = ciimage,
Transform = transform
};
var output = affineTransform.OutputImage;
var context = CIContext.FromOptions (null);
return context.CreateCGImage (output, output.Extent);
}
If either is more efficient of the two then it'll be the former.
When you create a CGImageSource you create just what the name says — some sort of opaque thing from which an image can be obtained. In your case it'll be a reference to a thing on disk. When you ask ImageIO to create a thumbnail you explicitly tell it "do as much as you need to output this many pixels".
Conversely if you draw to a CGBitmapContext then at some point you explicitly bring the whole image into memory.
So the second approach definitely has the whole image in memory at once at some point. Conversely the former needn't necessarily (in practice there'll no doubt be some sort of guesswork within ImageIO as to the best way to proceed). So across all possible implementations of the OS either the former will be advantageous or there'll be no difference between the two.
I would try using a c-based library like leptonica. I'm not sure whether ios optimizes Core Graphics with the relatively new Accelerate Framework, but CoreGraphics probably has more overhead involved just to re-size an image. Finally... If you want to roll your own implementation try using vImageScale_??format?? backed with some memory mapped files, I can't see anything being faster.
http://developer.apple.com/library/ios/#documentation/Performance/Conceptual/vImage/Introduction/Introduction.html
PS. Also make sure to check the compiler optimization flags.
I think if you want to save the memory you can read the source image from tile to tile and compress the tile and save to the destination tile.
There is an example from apple. It is the implementation of the way.
https://developer.apple.com/library/ios/samplecode/LargeImageDownsizing/Introduction/Intro.html
You can download this project and run it. It is MRC so you can use it very smoothly.
May it help. :)
I have a stupid question. I want to load a JPEG file and do some image processing. In the processing, the image file must be in pixel=by-pixel (unsigned char*), or in the format .bmp. I want to know how can I do this?
After processing, I want to save this file as .bmp or .jpg, how can I do it?
Thanks very much and sorry for my poor English.
The link int the comments is useful but not exactly what I think you want to do. You will have to read up a bit on Quartz, Apple's technology for drawing and image processing. Apple has great documentation on this, but its not a simple one hour type of effort - plan on at least a day maybe more.
Assume you start with a UIImage (I'm guessing, its not all critical):
get a CGImageRef to that image (possibly [UIImage CGImage])
create a CGBitMapContext that is large enough to render that image into (you can ask a CGImage for its width and height, etc).
if you want to ultimately create a PNG with alpha, you will need to create a bitmap context big enough for alpha, even if the JPEG image does not have it. If you want another JPEG your job is a bit easier.
render the image into the bit map context (there is a CGContextDraw... routine for that)
Now your image is contained in a bitmap that you can read and modify. The layout will probably be r-g-b unless you specified alpha, in which case you will have to determine where the alpha byte is.
Once you have the bit map modified, you can create a new CGImage from that bitmap, and with that you can get a UIImage if you want.
PS: you will find much sample code and posting about this if you search, so you will not be totally on your own.