I'm currently using a UIGraphicsBeginImageContext(resultingImageSize); to create an image.
But when I call this function, I don't know exactly the width of resultingImageSize.
Indeed, I developed some kind of video processing which consume lots of memory, and I cannot process first then draw after: I must draw during the video process.
If I set, for example UIGraphicsBeginImageContext(CGSizeMake(300, 400));, the drawn part over 400 is lost.
So is there a solution to set a variable size of CGContext, or resize a CGContext with very few memory consume?
I found a solution by creating a new larger Context each time it must be resized. Here's the magic function:
void MPResizeContextWithNewSize(CGContextRef *c, CGSize s) {
size_t bitsPerComponents = CGBitmapContextGetBitsPerComponent(*c);
size_t numberOfComponents = CGBitmapContextGetBitsPerPixel(*c) / bitsPerComponents;
CGContextRef newContext = CGBitmapContextCreate(NULL, s.width, s.height, bitsPerComponents, sizeof(UInt8)*s.width*numberOfComponents,
CGBitmapContextGetColorSpace(*c), CGBitmapContextGetBitmapInfo(*c));
// Copying context content
CGImageRef im = CGBitmapContextCreateImage(*c);
CGContextDrawImage(newContext, CGRectMake(0, 0, CGBitmapContextGetWidth(*c), CGBitmapContextGetHeight(*c)), im);
CGImageRelease(im);
CGContextRelease(*c);
*c = newContext;
}
I wonder if it could be optimized, for example with memcpy, as suggested here. I tried but it makes my code crash.
Related
sorry, for this question, I know there is a similar question, but I can not get the answer to work. Probably some dumb error on my side ;-)
I want to overlay two images with Alpha on iOS. The images taken from two videos, read by an AssetReader and stored in two CVPixelBuffer. I know that the Alpha channel is not stored in the video, so I get it from a third file. All data looks fine. The Problem is the overlay, if I do it onscreen with [CIContext drawImage] everything is fine !
But if I do it offscreen because the format of the video is not identical to the screen format, I can not get it to work:
1. drawImage does work, but only on-screen
2. render:toCVPixelBuffer works, but ignores Alpha
3. CGContextDrawImage seems to do nothing at all (not even an error message)
So can somebody give me an idea what is wrong:
Init:
...(a lot of code before)
Setup color space and bitmap context
if(outputContext)
{
CGContextRelease(outputContext);
CGColorSpaceRelease(outputColorSpace);
}
outputColorSpace = CGColorSpaceCreateDeviceRGB();
outputContext = CGBitmapContextCreate(CVPixelBufferGetBaseAddress(pixelBuffer), videoFormatSize.width, videoFormatSize.height, 8, CVPixelBufferGetBytesPerRow(pixelBuffer), outputColorSpace,(CGBitmapInfo) kCGBitmapByteOrderDefault |kCGImageAlphaPremultipliedFirst);
...
(a lot code after)
Drawing:
CIImage *backImageFromSample;
CGImageRef frontImageFromSample;
CVImageBufferRef nextImageBuffer = myPixelBufferArray[0];
CMSampleBufferRef sampleBuffer = NULL;
CMSampleTimingInfo timingInfo;
//draw the frame
CGRect toRect;
toRect.origin.x = 0;
toRect.origin.y = 0;
toRect.size = videoFormatSize;
//background image always full size, this part seems to work
if(drawBack)
{
CVPixelBufferLockBaseAddress( backImageBuffer, kCVPixelBufferLock_ReadOnly );
backImageFromSample = [CIImage imageWithCVPixelBuffer:backImageBuffer];
[coreImageContext render:backImageFromSample toCVPixelBuffer:nextImageBuffer bounds:toRect colorSpace:rgbSpace];
CVPixelBufferUnlockBaseAddress( backImageBuffer, kCVPixelBufferLock_ReadOnly );
}
else
[self clearBuffer:nextImageBuffer];
//Front image doesn't seem to do anything
if(drawFront)
{
unsigned long int numBytes = CVPixelBufferGetBytesPerRow(frontImageBuffer)*CVPixelBufferGetHeight(frontImageBuffer);
CVPixelBufferLockBaseAddress( frontImageBuffer, kCVPixelBufferLock_ReadOnly );
CGDataProviderRef provider = CGDataProviderCreateWithData(NULL, CVPixelBufferGetBaseAddress(frontImageBuffer), numBytes, NULL);
frontImageFromSample = CGImageCreate (CVPixelBufferGetWidth(frontImageBuffer) , CVPixelBufferGetHeight(frontImageBuffer), 8, 32, CVPixelBufferGetBytesPerRow(frontImageBuffer), outputColorSpace, (CGBitmapInfo) kCGBitmapByteOrderDefault | kCGImageAlphaPremultipliedFirst, provider, NULL, NO, kCGRenderingIntentDefault);
CGContextDrawImage ( outputContext, inrect, frontImageFromSample);
CVPixelBufferUnlockBaseAddress( frontImageBuffer, kCVPixelBufferLock_ReadOnly );
CGImageRelease(frontImageFromSample);
}
Any ideas anyone ?
So obviously I should stop to ask questions on stackflow. Every time I do that after hours of debugging I find the answer myself shortly afterwards. Sorry for that. The problem is in the initialisation, you can't do CVPixelBufferGetBaseAddress without locking the adresss first O_o. The adress gets NULL and this seems to be allowed, with the action then beeing not to do anything. So the correct code is:
if(outputContext)
{
CGContextRelease(outputContext);
CGColorSpaceRelease(outputColorSpace);
}
CVPixelBufferLockBaseAddress(pixelBuffer);
outputColorSpace = CGColorSpaceCreateDeviceRGB();
outputContext = CGBitmapContextCreate(CVPixelBufferGetBaseAddress(pixelBuffer), videoFormatSize.width, videoFormatSize.height, 8, CVPixelBufferGetBytesPerRow(pixelBuffer), outputColorSpace,(CGBitmapInfo) kCGBitmapByteOrderDefault |kCGImageAlphaPremultipliedFirst);
CVPixelBufferUnlockBaseAddress(pixelBuffer);
Let's say you have an original image that is
200 high, 100 wide
Let's say you want to draw only a square of it. Let's say, just the bottom square.
Let's say you want to draw it on to a new small image that is
20 high, 20 wide
Of course, you simply do this:
CGRect imageRect = CGRectMake( -10,0, 20,20);
.. begin graphics context ..
[originalImage drawInRect:imageRect];
With drawRect, you supply a rectangle the same full shape (same proportions) of the original image, but expressed in the size of the new canvas. No problem.
BUT:
in the example, you are drawing THE WHOLE ORIGINAL IMAGE -- THE WHOLE 200 HEIGHT on to the new small square.
(Of course the "top half" misses the new canvas, and you only get the bottom half on the new canvas -- which is what you wanted.)
My impression is iOS renders or calculates the "whole" original image, and it only "puts on" the bottom half (in the example) on to the new canvas.
This seems very wasteful.
IS THERE A FASTER WAY TO DO THIS?
It seems like there should be a command, something like this:
drawThisPartOfTheOriginalImage: (0,100 to 100,200)
ontoThisPartOfTheNewCanvas: (0,20 to 20,20)
What's the situation? Is there a more efficient command than drawRect when you are only drawing a small part of the original image? Cheers
CGContextClipToRect approach...(doesn't work!)
.
I experimented with CGContextClipToRect as Peter suggested below.
CGContextClipToRect indeed sets the area you will draw to on your "result" canvas. I simply set it to the size of that result canvas (it would be 20.20 in the example above). To repeat the aim here being to have iOS save time by avoiding pointlessly drawing the, err, not-drawn part of the original.
This example is for an original image 2000.2000 drawing on to a 500.500 (ie, only drawing the top left quarter of the original on to the result).
In fact notice it is slightly slower when you include the CGContextClipToRect, again suggesting iOS "knows when to stop" anyways.
// no need to "overdraw"... quickener turned OFF
//CGContextRef c = UIGraphicsGetCurrentContext();
//CGContextClipToRect(c, CGRectMake(0, 0, resultSize.width,resultSize.height));
//Execution Time .................................. 0.443669
// no need to "overdraw"... quickener turned ON
CGContextRef c = UIGraphicsGetCurrentContext();
CGContextClipToRect(c, CGRectMake(0, 0, resultSize.width,resultSize.height));
//Execution Time .................................. 0.461845
As you can see it's a hair slower, actually, adding the CGContextClipToRect trick.
For the record, here is the exact routine used to crop an image:
-(UIImage *)simplishTopCrop:(UIImage *)fromImage
{
// check for zero fromImage.size.width etc etc
CGSize resultSize = CGSizeMake(640,640);
CGFloat scale = MAX(
resultSize.width/fromImage.size.width,
resultSize.height/fromImage.size.height);
CGFloat width = fromImage.size.width * scale;
CGFloat height = fromImage.size.height * scale;
CGRect imageRect = CGRectMake(0,0, width,height);
UIGraphicsBeginImageContextWithOptions(resultSize, NO, 0);
// INSERT 'CGContextClipToRect' TRICK ABOVE, RIGHT HERE
[fromImage drawInRect:imageRect];
UIImage *newImage = UIGraphicsGetImageFromCurrentImageContext();
UIGraphicsEndImageContext();
return newImage;
}
This is where clipping comes in. Clip to your dirty rect, then draw the whole image into your bounds. The clipping path will keep the rest of the image at least from appearing, and hopefully from being composited or sampled at all.
If your profiling in Instruments finds that that is not efficient enough, you might try cropping the image itself, using CGImageCreateWithImageInRect, and then drawing that image into your dirty rect. You may want to keep your cropped image around and only throw it away when the rect changes. One way or the other, cropping the image may be more efficient—but don't forget to profile both before and after to prove that.
I'm currently working on a project that must generate a collage of a 9000x6000 pixels resolution, generated from 15 photos. The problem that I'm facing is that when I finish drawing I'm getting an empty image (those 15 images are not being drawn in the context).
This problem is only present on devices with 512MB of RAM like iPhone 4/4S or iPad 2 and I think that this is a problem caused by the system because it cannot allocate enough memory for this app. When I run this line: UIGraphicsBeginImageContextWithOptions(outputSize, opaque, 1.0f); the app's memory usage raises by 216MB and the total memory usage gets to ~240MB RAM.
The thing that I cannot understand is why on Earth the images that I'm trying to draw within the for loop are not being rendered always on the currentContext? I emphasized the word always, because only once in 30 tests the images were rendered (without changing any line of code).
Question nr. 2: If this is a problem caused by the system because it cannot allocate enough memory, is there any other way to generate this image, like a CGContextRef backed by a file output stream, so that it won't keep the image in the memory?
This is the code:
CGSize outputSize = CGSizeMake(9000, 6000);
BOOL opaque = YES;
UIGraphicsBeginImageContextWithOptions(outputSize, opaque, 1.0f);
CGContextRef currentContext = UIGraphicsGetCurrentContext();
CGContextSetFillColorWithColor(currentContext, [UIColor blackColor].CGColor);
CGContextFillRect(currentContext, CGRectMake(0, 0, outputSize.width, outputSize.height));
for (NSUInteger i = 0; i < strongSelf.images.count; i++)
{
#autoreleasepool
{
AGAutoCollageImageData *imageData = (AGAutoCollageImageData *)strongSelf.layout.images[i];
CGRect destinationRect = CGRectMake(floorf(imageData.destinationRectangle.origin.x * scaleXRatio),
floorf(imageData.destinationRectangle.origin.y * scaleYRatio),
floorf(imageData.destinationRectangle.size.width * scaleXRatio),
floorf(imageData.destinationRectangle.size.height * scaleYRatio));
CGRect sourceRect = imageData.sourceRectangle;
// Draw clipped image
CGImageRef clippedImageRef = CGImageCreateWithImageInRect(((ALAsset *)strongSelf.images[i]).defaultRepresentation.fullScreenImage, sourceRect);
CGContextDrawImage(currentContext, drawRect, clippedImageRef);
CGImageRelease(clippedImageRef);
}
}
// Pull the image from our context
strongSelf.result = UIGraphicsGetImageFromCurrentImageContext();
// Pop the context
UIGraphicsEndImageContext();
P.S: The console doesn't show anything but 'memory warnings', which are expected to see.
Sound like a cool project.
Tactic: try also releasing imageData at the end of every loop (explicitly, after releasing the clippedImageRef)
Strategic:
If you do need to support such "low" RAM requirements with such "high" input, maybe you should consider 2 alternative options:
Compress (obviously): even minimal, naked to the eye, JPEG compression can go a long way.
Split: never "really" merge the image. Have an arrayed datastructure which represents a BigImage. make utilities for the presentation logic.
I have a question about Tesseract OCR principle. As far as I understand, after shapes detection , symbols (their forms) are scaled(resized) to have some specific font size.
Such font size is based on trained data. Basically, trained set defines symbols (their geometry,shape), maybe their representation.
I am using Tesseract 3.01 (the latest) version on iOS platform.
I check Tesseract FAQ, looked at forum, but I do not understand why for some images I have low quality of recognition.
It is said that font should be bigger than 12pt & image should have more than 300 DPI. I did all necessary preprocessing such as blurring (if it is needed), contrast enhancement.
I even used other engine in Tesseract OCR - it is called CUBE.
But for some images (in spite of fact that they are bigger MIN(width, height) >1000 - I rescale them for tesseract, I get bad results for recognition
http://goo.gl/l9uJMe
However on other set of images results are better:
http://goo.gl/cwA9DC
Those images smaller I do not resize them, (just convert to grayscale mode).
If what I wrote about engine is correct.
Suppose trained set is based on font with size 14pt. Symbols from pictures are resized to some specific size, and I do not see any reason why they are not recognised in such case.
I also tried custom dictionaries, to penalise non dictionary words - did not give too much benefit to recognition.
tesseract = new tesseract::TessBaseAPI();
GenericVector<STRING> variables_name(1),variables_value(1);
variables_name.push_back("user_words_suffix");
variables_value.push_back("user-words");
int retVal = tesseract->Init([self.tesseractDataPath cStringUsingEncoding:NSUTF8StringEncoding], NULL,tesseract::OEM_TESSERACT_ONLY, NULL, 0, &variables_name, &variables_value, false);
ok |= retVal == 0;
ok |= tesseract->SetVariable("language_model_penalty_non_dict_word", "0.2");
ok |= tesseract->SetVariable("language_model_penalty_non_freq_dict_word", "0.2");
if (!ok)
{
NSLog(#"Error initializing tesseract!");
}
So my question is should I train tesseract on another font?
And ,honestly speaking, why I should train it? on default trained data text from Internet, or screen of PC(Mac) I get good recognition.
I also checked original tesseract English trained data it has 38 tiff files, that belong to the following families:
1) Аrial
2) verdana
3 )trebuc
4) times
5) georigia
6 ) cour
It seems that font from image does not belong to this set.
In your case the size of the image is not the problem. As I can see from your attached images (and I'm surprised that nobody mentioned it before) the problem is that the text on images from which you get bad results is not placed on straight lines.
One of the things that Tesseract does at early stages of OCR process is to detect image layout and extracting whole lines of text.
This image is the best example to illustrate this part of the process:
As you can see the engine is expecting the text to be perpendicular to the edge of the image.
If you done with all necessary image processing then try this, It may helpful for you
CGSize size = [image size];
int width = size.width;
int height = size.height;
uint32_t* _pixels = (uint32_t *) malloc(width * height * sizeof(uint32_t));
if (!_pixels) {
return;//Invalid image
}
// Clear the pixels so any transparency is preserved
memset(_pixels, 0, width * height * sizeof(uint32_t));
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
// Create a context with RGBA _pixels
CGContextRef context = CGBitmapContextCreate(_pixels, width, height, 8, width * sizeof(uint32_t), colorSpace,kCGBitmapByteOrder32Little | kCGImageAlphaPremultipliedLast);
// Paint the bitmap to our context which will fill in the _pixels array
CGContextDrawImage(context, CGRectMake(0, 0, width, height), [image CGImage]);
// We're done with the context and color space
CGContextRelease(context);
CGColorSpaceRelease(colorSpace);
_tesseract->SetImage((const unsigned char *) _pixels, width, height, sizeof(uint32_t), width * sizeof(uint32_t));
_tesseract->SetVariable("tessedit_char_whitelist", ".#0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz/-!");
_tesseract->SetVariable("tessedit_consistent_reps", "0");
char* utf8Text = _tesseract->GetUTF8Text();
NSString *str = nil;
if (utf8Text) {
str = [NSString stringWithUTF8String:utf8Text];
}
I have a simple drawing issue. I have prepared a 2 dimensional array which has an animated wave motion. The array is updated every 1/10th of a second (this can be changed by the user). After the array is updated I want to display it as a 2 dimensional image with each array value as a pixel with color range from 0 to 255.
Any pointers on how to do this most efficiently...
Appreciate any help on this...
KAS
If it's just a greyscale then the following (coded as I type, probably worth checking for errors) should work:
CGDataProviderRef dataProvider =
CGDataProviderCreateWithData(NULL, pointerToYourData, width*height, NULL);
CGColorSpaceRef colourSpace = CGColorSpaceCreateDeviceGray();
CGImageRef inputImage = CGImageCreate( width, height,
8, 8, width,
colourSpace,
kCGBitmapByteOrderDefault,
dataProvider,
NULL, NO,
kCGRenderingIntentDefault);
CGDataProviderRelease(dataProvider);
CGColorSpaceRelease(colourSpace);
UIImage *image = [UIImage imageWithCGImage:inputImage];
CGImageRelease(inputImage);
someImageView.image = image;
That'd be for a one-shot display, assuming you didn't want to write a custom UIView subclass (which is worth the effort only if performance is a problem, probably).
My understanding from the docs is that the data provider can be created just once for the lifetime of your C buffer. I don't think that's true of the image, but if you created a CGBitmapContext to wrap your buffer rather than a provider and an image, that would safely persist and you could use CGBitmapContextCreateImage to get a CGImageRef to be moving on with. It's probably worth benchmarking both ways around if it's an issue.
EDIT: so the alternative way around would be:
// get a context from your C buffer; this is now something
// CoreGraphics could draw to...
CGColorSpaceRef colourSpace = CGColorSpaceCreateDeviceGray();
CGContextRef context =
CGBitmapContextCreate(pointerToYourData,
width, height,
8, width,
colourSpace,
kCGBitmapByteOrderDefault);
CGColorSpaceRelease(colourSpace);
// get an image of the context, which is something
// CoreGraphics can draw from...
CGImageRef image = CGBitmapContextCreateImage(context);
/* wrap in a UIImage, push to a UIImageView, as before, remember
to clean up 'image' */
CoreGraphics copies things about very lazily, so neither of these solutions should be as costly as the multiple steps imply.