How to load black white image into metal texture - ios

I use two methods to load black white image texture in metal as below:
//1.
mtltexture01 = [textureLoader newTextureWithName:#"texture02"
scaleFactor:1.0
bundle:nil
options:textureLoaderOptions
error:&error];
//2.
UIImage *img = [UIImage imageNamed:#"texture02"];
mtltexture01 = [textureLoader newTextureWithCGImage:img.CGImage options:textureLoaderOptions error:&error];
but both crash, the error log is
"Error Domain=MTKTextureLoaderErrorDomain Code=0 "Image decoding
failed" UserInfo={NSLocalizedDescription=Image decoding failed,
MTKTextureLoaderErrorKey=Image decoding failed}",
how to fix this issue? Also if I load the colorful image into metal, it runs.

-(id<MTLTexture>)textureWithName:(NSString*)imgname UsingDevice:(id<MTLDevice>)device {
MTKTextureLoader* textureLoader = [[MTKTextureLoader alloc] initWithDevice:device];
NSDictionary *textureLoaderOptions = #{
MTKTextureLoaderOptionTextureUsage : #(MTLTextureUsageShaderRead),
MTKTextureLoaderOptionTextureStorageMode : #(MTLStorageModePrivate)
};
return [textureLoader newTextureWithName:imgname
scaleFactor:1.0
bundle:nil
options:textureLoaderOptions
error:nil];
}
and in your metal configurations
id<MTLTexture> mtltexture01;
mtltexture01 = [self textureWithName:#"texture02" UsingDevice:device];
Keep in mind texture02 is a filename and the file needs to be available in your apps assets. You can store the image as MTLTexture into your asset in Xcode and so the conversion is done at build time.
Also check if the image contains at least proper opacity and is one of PNG, JPEG, or TIFF format. There is known loader trouble when textures contain only gray/ black colors and 0% transparency is used.
later integrate your texture to the renderEncoder/commandEncoder
screenshot: Xcode Assets Texture Configurator

Related

Tesseract OCR not recognizing the image taken from device

I'm using the https://github.com/gali8/Tesseract-OCR-iOS/ to make an app that detects text on business cards.
I'm stuck at making the Tesseract detect the text in image.
If I pass the image through code, Tesseract is able to detect it. If I provide the image taken from the camera, tesseract is not able to recognize it.
-(void)startTess:(UIImage *)img{
G8Tesseract *tesseract = [[G8Tesseract alloc] initWithLanguage:#"eng"];
tesseract.delegate = self;
tesseract.engineMode=G8OCREngineModeTesseractCubeCombined;
// Optional: Limit the character set Tesseract should try to recognize from
tesseract.charWhitelist = #"#.,()-,abcdefghijklmnopqrstuvwxyz0123456789";
// Specify the image Tesseract should recognize on
tesseract.image = [img g8_blackAndWhite];
// Optional: Limit the area of the image Tesseract should recognize on to a rectangle
CGRect tessRect = CGRectMake(0, 0, img.size.width, img.size.height);
tesseract.rect = tessRect;
// Optional: Limit recognition time with a few seconds
tesseract.maximumRecognitionTime = 4.0;
// Start the recognition
[tesseract recognize];
// Retrieve the recognized text
NSLog(#"text %#", [tesseract recognizedText]);
// You could retrieve more information about recognized text with that methods:
NSArray *characterBoxes = [tesseract recognizedBlocksByIteratorLevel:G8PageIteratorLevelSymbol];
NSArray *paragraphs = [tesseract recognizedBlocksByIteratorLevel:G8PageIteratorLevelParagraph];
NSArray *characterChoices = tesseract.characterChoices;
UIImage *imageWithBlocks = [tesseract imageWithBlocks:characterBoxes drawText:YES thresholded:NO];
self.imgView.image = imageWithBlocks;
NSString * result = [[characterBoxes valueForKey:#"description"] componentsJoinedByString:#"\n"];
_txtView.text=result;
}
Result when image provided from .xcassets:
Result when image taken directly from the camera:
In both the cases, Tesseract is recognizing the empty space with some random characters. I marked that area in both the images (top-left portion of image).
I made sure that image taken from device camera has the orientation up, as some reported Tesseract doesn't recognize the image taken from camera as it has 180 degree shift.
UIImage *chosenImage = info[UIImagePickerControllerOriginalImage];
// Redraw the image (if necessary) so it has the corrent orientation:
if (chosenImage.imageOrientation != UIImageOrientationUp) {
UIGraphicsBeginImageContextWithOptions(chosenImage.size, NO, chosenImage.scale);
[chosenImage drawInRect:(CGRect){0, 0, chosenImage.size}];
chosenImage = UIGraphicsGetImageFromCurrentImageContext();
UIGraphicsEndImageContext();
}
What is the best way of debugging this and going forward ?
I submitted an issue at git:
https://github.com/gali8/Tesseract-OCR-iOS/issues/358
Edit:
I have changed the iterator level to G8PageIteratorLevelTextline, and now the image taken by device camera gives the following output:
Still it is not accurate. If someone can point out on how to improve this, it would be nice.
On the official github source of tesseract there are various preprocessing methods mentioned and along with those measures I would suggest using .tiff images instead of .jpg or .png because using any other kind of image other than tiff compresses the image and reduces it binarizing quality.

CGImageDestinationFinalize or UIImageJPEGRepresentation - Crash when saving a large file on IOS 10

I am trying to create tiles for a larger image, and it seems that as of IOS 10 the following code no longer works and crashes with EXC_BAD_ACCESS.
This happens on IOS 10 Device only, IOS 9 works fine.
The crash happens with any image that is larger than ~1300x1300.
Profiling in instruments doesn't yield anything interesting and points to CGImageDestinationFinalize.
There is no memory spike.
I tried both ways below:
UIImage* tempImage = [UIImage imageWithCGImage:tileImage];
NSData* imageData = UIImageJPEGRepresentation(tempImage, 0.8f); // CRASH HERE.
OR
+ (BOOL) CGImageWriteToFile: (CGImageRef) image andPath: (NSString *)path
{
CFURLRef url = (__bridge CFURLRef)[NSURL fileURLWithPath:path];
CGImageDestinationRef destination = CGImageDestinationCreateWithURL(url, kUTTypeJPEG, 1, NULL);
if (!destination) {
NSLog(#"Failed to create CGImageDestination for %#", path);
return NO;
}
CGImageDestinationAddImage(destination, image, nil);
if (!CGImageDestinationFinalize(destination)) { // CRASH HERE
NSLog(#"Failed to write image to %#", path);
CFRelease(destination);
return NO;
}
CFRelease(destination);
As I understand it UIImageJPEGRepresentation in turn calls CGImageDestinationFinalize eventually.
Edit: Forgot to mention the images are written to hdd, but about half way through.
This really does seem like something to do with IOS 10, could this be a bug?
I also filed the bug with apple just in case.
Any help or workarounds on how to write out large images is greatly appreciated.
I am going to answer my own question here.
What I have gathered is that IOS 10 has trouble saving a file that has color mode "grayscale".
So if you do
UIImageJPEGRepresentation(tempImage, 0.8f)
and if tempImage is grayscale you will get a crash.
I was processing many images and it didn't click until later that it could be related to the color mode of the image.
I already filed a bug with apple to see what they say, but meanwhile I had to find a temporary fix.
My fix was to convert image to RGB by drawing it and capturing it into a new image.
Sample method:
+ (UIImage *)convertImage:(UIImage *)sourceImage
{
UIGraphicsBeginImageContext(sourceImage.size);
[sourceImage drawInRect:CGRectMake( 0, 0, sourceImage.size.width, sourceImage.size.height)];
UIImage *targetImage = UIGraphicsGetImageFromCurrentImageContext();
UIGraphicsEndImageContext();
return targetImage;
}
Hopefully this helps someone.
tempimage is one of the parameters you passed to UIImageJPEGRepresentation; if it points to a dead object, then that will cause a crash when UIImageJPEGRepresentation tries to use that dead object. check whether tempImage is nil.
check whether you are adding image after calling CGImageDestinationFinalize at destination

How to compress images in iOS?

I have been looking through every stack overflow post and video online and I can't find a solution that works correctly. Right now the user selects a photo that I want to compress before uploading to an AWS S3 bucket. The upload works perfectly but for some reason, the compressed image is larger than the original image! For example, if the user selects a 9KB photo, when I upload to S3 the photo is 28.5KB. I tried a different photo and it's 48KB and after "compression" on S3 its 378.9KB! (I am using the latest software version of everything and compiling with the simulator)
I want to compress the original image as much as I can before uploading.
This is what I have so far:
How I "compress" the image:
UIImage *compressedProductImage;
NSData *NSproductImage;
NSUInteger productImageSize;
- (void)imagePickerController:(UIImagePickerController *)picker didFinishPickingMediaWithInfo:(NSDictionary<NSString *,id> *)info{
self.productImage = info[UIImagePickerControllerOriginalImage];
[self.productImageImageView setImage:self.productImage];
[self dismissViewControllerAnimated:YES completion:nil];
NSproductImage = UIImageJPEGRepresentation(self.productImage, 0.5f);
productImageSize = [NSproductImage length];
compressedProductImage = [UIImage imageWithData: NSproductImage];
How I upload the photo:
//Convert product UIImage
NSArray *productImagePaths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
NSString *productImageFilePath = [[productImagePaths objectAtIndex:0] stringByAppendingPathComponent:[NSString stringWithFormat:#".png"]];
[UIImagePNGRepresentation(compressedProductImage) writeToFile:productImageFilePath atomically:YES];
NSURL *productImageFileUrl = [NSURL fileURLWithPath:productImageFilePath];
uploadRequest.body = productImageFileUrl;//Needs to be a NSURL
uploadRequest.bucket = AWS_BUCKET_NAME;
uploadRequest.key = productImageKey;
uploadRequest.contentType = #"image/png";
uploadRequest.ACL = AWSS3BucketCannedACLPublicRead;
[[transferManager upload:uploadRequest] continueWithExecutor:[AWSExecutor mainThreadExecutor] withBlock:^id(AWSTask *task) {
if (task.error != nil) {
NSLog(#"%s %#","Error uploading (product image):", uploadRequest.key);
}else{
NSLog(#"Product image upload completed");
}
return nil;
}];
As rmaddy points out, you're taking the picked image, converting it to JPEG, converting back to a UIImage (losing any benefit of the JPEG compression), and then converting it to a PNG, which offers modest compression, generally far less compression than the original JPEG from the users photo library.
You have a few options.
You can retrieve the original imageData from the asset in your photos library as shown in https://stackoverflow.com/a/32845656/1271826, thereby avoiding the round-tripping it through a UIImage at all. Thus, you preserve the quality of the original image, preserve the meta data associated with this image, and enjoy the decent compression of the original asset.
You could take the picked image as a UIImage and do a combination of:
reduce the dimensions of the image before you call UIImageJPEGRepresentation (see https://stackoverflow.com/a/10491692/1271826 for sample algorithm); and/or
use UIImageJPEGRepresentation with a quality less than 1.0, where the smaller the number, the more compression but the more image quality loss.
You don't actually compress anything. You start with a UIImage. This is a full pixel by pixel representation that takes (typically) width x height x 4 bytes.
You then convert that to a JPG with some compression. So NSproductImage is a much smaller representation, in memory, of the JPG version of the image. This is the supposed smaller size you see where you think it is now compressed.
But then you convert that JPG data back into a UIImage as compressedProductImage. This new UIImage still has the same width and heigh of the original UIImage. As a result, it still takes the same width x height x 4 bytes as the original. It's just of lower quality than the original due to the JPG compression.
Now you convert the updated UIImage into a PNG. Since PNG is lossless, it doesn't compress nearly as much as the JPG attempt. You then send this larger PNG version of the image to Amazon.
You should first remove the pointless code that first converts to JPG and then back to UIImage.
At this point you should either live with the size of the PNG or use JPG instead and send the smaller JPG to Amazon.
Another option would be to scale the image before sending it to Amazon.

Generate an image of the contents of a SKScene which is not displayed

In my current SKScene, I use UIGraphicsGetImageFromCurrentImageContext to generate an image from the current graphics on screen.
However, I would like to generate an image from a scene not on screen. One I have created but have not displayed. Is this possible?
The reason for doing is this is to create a custom image for users to share when they achieve a high score, which is similar, but not the same as my main Scene.
Here is a method that captures the contents of a node as a PNG. Be aware that current SpriteKit seems to have a memory leak on the access of the CGImage property, so use this in DEBUG mode.
+ (NSData*) captureNodeAsPNG:(SKSpriteNode*)node skView:(SKView*)skView
{
NSData *data = nil;
#autoreleasepool {
SKTexture *captureTexture = [skView textureFromNode:node];
CGImageRef cgImageRef = captureTexture.CGImage;
NSLog(#"capture texture from node as pixels : %d x %d", (int)CGImageGetWidth(cgImageRef), (int)(CGImageGetHeight(cgImageRef)));
UIImage *capImg = [UIImage imageWithCGImage:cgImageRef];
data = UIImagePNGRepresentation(capImg);
}
return data;
}

How to insert images through Open Gl in ios?

I have just started learning OpenGl, i have read about Rendering and texture image. If anyone can provide a simple example to insert image.
OpenGL itself doesn’t provide API to generate a texture from image file. We had to read image file and convert it to bitmap data to use texture in OpenGL before, but GLKTextureLoader class in GLKit enables us generating texture from image automatically.
NSString* filePath = [[NSBundle mainBundle] pathForResource:#"mushroom" ofType:#"png"];
GLKTextureInfo* textureInfo =
[GLKTextureLoader textureWithContentsOfFile:filePath options:nil error:nil];
if (textureInfo) {
NSLog(#"Texture loaded successfully. name = %d size = (%d x %d)",
textureInfo.name, textureInfo.width, textureInfo.height);
}
return value of textureWithContentsOfFile:options:error: method is an instance of GLKTextureInfo class. A GLKTextureInfo object contains information about a texture such as width, height… There is a property named ‘name’ (documented as ‘glName’ and it should be misprint). We use value in this property to specify texture in GLKBaseEffect (Will be described later)
GLKTextureLoader is a class for loading textures, so we should release texture by our self. Call glDeleteTextures with value of ‘name’ property in GLKTextureInfo object.
Found this nice discussion in a blog. Hope this helps.. :)

Resources