I am trying to use iOS 5 facedetection, and I'm finding that when I run the code on the simulator, it detects a face with correct frame. However, if I run the exact same code on the same image on the device, it returns incorrect dimensions.
Here's my code:
CIImage* image = [CIImage imageWithCGImage:someImage.CGImage];
NSDictionary *detectorOptions =
[NSDictionary dictionaryWithObjectsAndKeys:
CIDetectorAccuracyHigh, CIDetectorAccuracy,
nil];
CIDetector* detector = [CIDetector detectorOfType:CIDetectorTypeFace
context:nil
options:detectorOptions];
NSArray *faceFeatures = [detector featuresInImage:image];
for (CIFeature *f in faceFeatures) {
NSLog(#"Feature: %#", NSStringFromRect(f.bounds));
}
The output from the simulator (correct):
Feature: {{78, 153}, {200, 200}}
The output from the device (incorrect):
Feature: {{104, 199}, {272, 272}}
Is this a bug? Or am I improperly using the face detection code? I've also tried using featuresInImage:options and passing in another dictionary with the device orientation
Related
Using following code in my application which was performing quiet fine to draw a CIImage on a GLKView again and again as recieved from AVCaptureOutput -didOutputSampleBuffer until I was using iOS <= 10.1.*
After updating device to iOS 10.2.1 it has stopped working. I am calling it for few frames the app just crashes with low memory warning. Whereas with iOS 10.1.1 and below I smoothly runs the app even on older device like iPhone 5S.
[_glkView bindDrawable];
if (self.eaglContext != [EAGLContext currentContext])
[EAGLContext setCurrentContext:self.eaglContext];
glClearColor(0.0, 0.0, 0.0, 1.0);
glClear(GL_COLOR_BUFFER_BIT);
glEnable(GL_BLEND);
glBlendFunc(GL_ONE, GL_ONE_MINUS_SRC_ALPHA);
if (ciImage) {
[_ciContext drawImage:ciImage inRect:gvRect fromRect:dRect];
}
[_glkView display];
This is how I am making the CIImage.
- (CIImage*)ciImageFromPixelBuffer:(CVPixelBufferRef)pixelBuffer ofSampleBuffer:(CMSampleBufferRef)sampleBuffer {
CIImage *croppedImage = nil;
CFDictionaryRef attachments = CMCopyDictionaryOfAttachments(kCFAllocatorDefault, sampleBuffer, kCMAttachmentMode_ShouldPropagate);
CIImage *ciImage = [CIImage imageWithCVPixelBuffer:pixelBuffer options:(NSDictionary *)attachments];
if (attachments)
CFRelease(attachments);
croppedImage = ciImage;
CIFilter *scaleFilter = [CIFilter filterWithName:#"CILanczosScaleTransform"];
[scaleFilter setValue:croppedImage forKey:#"inputImage"];
[scaleFilter setValue:[NSNumber numberWithFloat:self.zoom_Resize_Factor == 1 ? 0.25 : 0.5] forKey:#"inputScale"];
[scaleFilter setValue:[NSNumber numberWithFloat:1.0] forKey:#"inputAspectRatio"];
croppedImage = [scaleFilter valueForKey:#"outputImage"];
NSDictionary *options = #{(id)kCIImageAutoAdjustRedEye : #(false)};
NSArray *adjustments = [ciImage autoAdjustmentFiltersWithOptions:options];
for (CIFilter *filter in adjustments) {
[filter setValue:croppedImage forKey:kCIInputImageKey];
croppedImage = filter.outputImage;
}
CIFilter *selectedFilter = [VideoFilterFactory getFilterWithType:self.selectedFilterType]; //This line needs to be removed from here
croppedImage = [VideoFilterFactory applyFilter:selectedFilter OnImage:croppedImage];
CVPixelBufferUnlockBaseAddress(pixelBuffer, 0);
return croppedImage;
}
Here is imgur link http://imgur.com/a/u6Vyo of VM Tracker and OpenGL ES instruments result. Incase it eases to understand. Thanks.
Your GLKView rendering implementation looks fine, the issue seems to be coming from the amount of processing you're doing on PixelBuffer after converting it into CIImage.
Also the Imgur link you shared shows that GLKView is unable to prepare VideoTexture object correctly, most probably due to the memory overload being created in each iteration. You need to optimise this CIFilter Processing.
I put the QR code in an UIImageView and use the system method to read it.It works on iPhone6 or 6s but can't read anything on iPhone5.Does anyone know how to fix it?
CIDetector* detector = [CIDetector detectorOfType:CIDetectorTypeQRCode context:nil options:#{ CIDetectorAccuracy : CIDetectorAccuracyHigh }];
NSArray *features = [detector featuresInImage:[CIImage imageWithCGImage:self.QRImageView.image.CGImage]];
Looks you'll need to use AVCaptureMetadataOutput.
// Device
self.device = [AVCaptureDevice defaultDeviceWithMediaType:AVMediaTypeVideo];
// Input
self.input = [AVCaptureDeviceInput deviceInputWithDevice:self.device error:nil]
// Output
self.output = [[AVCaptureMetadataOutput alloc] init];
[self.output setMetadataObjectsDelegate:self queue:dispatch_get_main_queue()];
self.output.metadataObjectTypes = #[AVMetadataObjectTypeQRCode];
// Session
self.session = [[AVCaptureSession alloc] init];
[self.session addInput:self.input];
[self.session addOutput:self.output];
Other people have noted that CIDetector only works on the iPhone 5s and up, iPad Air and up.
(device reference: https://forums.developer.apple.com/thread/20887)
I make a FFMPEG-based player for ios. It works fine on simulator, but on real-device (iPhone 4) the frame rate is low and make my audio and video out of sync. the player works fine on iPhone 4s, so I guess it's just problem about device's computing power.
So, is there anyway to build FFMPEG optimized for iOS device (armv7, arvm7s arch)? or is there anyway to utilize ios device hardware to decode video stream?
My video stream is encode in H264/AAC.
Those streams should play just fine, I assume since your using ffmpeg you are not using a video protocol that iOS supports directly.
We use ffmpeg to do rtsp/rtmp and we get good performance with h264/aac
There are a number of factors that contribute to av/sync issues, usually some type of pre-buffering of the video is required, also network plays a big part in it.
As to your second question, hardware encoding is only available via avfoundation, you can use avassetwriter to encode your video, but again depends wether or not you need real-time.
see this link https://github.com/mooncatventures-group/FFPlayer-beta1/blob/master/FFAVFrames-test/ViewController.m
-(void) startRecording {
// // create the AVComposition
// [mutableComposition release];
// mutableComposition = [[AVMutableComposition alloc] init];
movieURL = [NSURL fileURLWithPath:[NSString stringWithFormat:#"%#/%llu.mov", NSTemporaryDirectory(), mach_absolute_time()]];
NSError *movieError = nil;
assetWriter = [[AVAssetWriter alloc] initWithURL:movieURL
fileType: AVFileTypeQuickTimeMovie
error: &movieError];
NSDictionary *assetWriterInputSettings = [NSDictionary dictionaryWithObjectsAndKeys:
AVVideoCodecH264, AVVideoCodecKey,
[NSNumber numberWithInt:FRAME_WIDTH], AVVideoWidthKey,
[NSNumber numberWithInt:FRAME_HEIGHT], AVVideoHeightKey,
nil];
assetWriterInput = [AVAssetWriterInput assetWriterInputWithMediaType: AVMediaTypeVideo
outputSettings:assetWriterInputSettings];
assetWriterInput.expectsMediaDataInRealTime = YES;
[assetWriter addInput:assetWriterInput];
assetWriterPixelBufferAdaptor = [[AVAssetWriterInputPixelBufferAdaptor alloc]
initWithAssetWriterInput:assetWriterInput
sourcePixelBufferAttributes:nil];
[assetWriter startWriting];
firstFrameWallClockTime = CFAbsoluteTimeGetCurrent();
[assetWriter startSessionAtSourceTime:kCMTimeZero];
startSampleing=YES;
}
The one drawback right now is that a way needs to be determined to read the encoded data as its being written, believe me when I say there are a few of us developers trying to figure out how to do that as we I write this.
I want to use AVFoundation to perform face recognition on an image taken from the camera roll.
Is this possible, in the first place?
I've found many tutorials on how to do this using live camera stream as input. I studied them and found no way to bind an AVCaptureInputDevice to an image.
P.S.
I don't want to use CoreImage. Actually I want to ditch CoreImage and use AVFoundation face recognition.
Thank you a lot.
What's the problem with CoreImage? It uses the same detector:
NSURL *url = [NSURL URLWithString:#"..."];
CIImage *img = [CIImage imageWithContentsOfURL:url]; // uiImageObj.CIImage
CIContext *context = [CIContext contextWithOptions:nil];
NSDictionary *detectorOptions = [[NSDictionary alloc] initWithObjectsAndKeys:CIDetectorAccuracyLow, CIDetectorAccuracy, nil];
CIDetector *detector = [CIDetector detectorOfType:CIDetectorTypeFace context:context options:detectorOptions];
NSArray *features = [detector featuresInImage:img];
I am a newbie in AVFoundation and decoding process.I need to decode a h264 video file and play it in iphone...can anyone give me guideline to do it.
I dont want to use ffmpeg or any third party library to do that. As far as I know using Avfoundation encoding is possible...here is the code which I thought is used for encoding but not sure at all...
float bitsPerPixel;
CMVideoDimensions dimensions = CMVideoFormatDescriptionGetDimensions(currentFormatDescription);
int numPixels = dimensions.width * dimensions.height;
int bitsPerSecond;
// Assume that lower-than-SD resolutions are intended for streaming, and use a lower bitrate
if ( numPixels < (640 * 480) )
bitsPerPixel = 4.05; // This bitrate matches the quality produced by AVCaptureSessionPresetMedium or Low.
else
bitsPerPixel = 11.4; // This bitrate matches the quality produced by AVCaptureSessionPresetHigh.
bitsPerSecond = numPixels * bitsPerPixel;
NSDictionary *videoCompressionSettings = [NSDictionary dictionaryWithObjectsAndKeys:
AVVideoCodecH264, AVVideoCodecKey,
[NSNumber numberWithInteger:dimensions.width], AVVideoWidthKey,
[NSNumber numberWithInteger:dimensions.height], AVVideoHeightKey,
[NSDictionary dictionaryWithObjectsAndKeys:
[NSNumber numberWithInteger:bitsPerSecond], AVVideoAverageBitRateKey,
[NSNumber numberWithInteger:30], AVVideoMaxKeyFrameIntervalKey,
nil], AVVideoCompressionPropertiesKey,
nil];
if ([assetWriter canApplyOutputSettings:videoCompressionSettings forMediaType:AVMediaTypeVideo]) {
assetWriterVideoIn = [[AVAssetWriterInput alloc] initWithMediaType:AVMediaTypeVideo outputSettings:videoCompressionSettings];
assetWriterVideoIn.expectsMediaDataInRealTime = YES;
assetWriterVideoIn.transform = [self transformFromCurrentVideoOrientationToOrientation:self.referenceOrientation];
if ([assetWriter canAddInput:assetWriterVideoIn])
[assetWriter addInput:assetWriterVideoIn];
else {
NSLog(#"Couldn't add asset writer video input.");
return NO;
}
}
else {
NSLog(#"Couldn't apply video output settings.");
return NO;
}
return YES;
I am completely naive about this, please help...from where to start///
thanks
The simpler solution is to use MPMoviePlayerController. It takes in input a mov or mv4 file (from local file system or through an URL).
The another option is to use AVPlayer class.
Hope it helps,
David
You can refer and build the official sample code: AVPlayerDemo to see how it works. It uses AV Foundation framework, mainly the AVPlayer APIs to play video files and the performance is excellent.
To play a video file by AVPlayerDemo you should copy the video files to your iOS device by itunes and select iPod library on the AVPlayerDemo App.