Fast video capture at 640x480#60fps - ios

Question

I want to capture video at 60fps using minimal resources, because I need to use at most 16-20ms per frame and most of the time will be occupied by heavy computation on the frame.
I am currently using the preset AVCaptureSessionPreset1280x720, but I want to do the computations at 640x480, otherwise the device will not keep up. Here starts the problem: I cannot directly capture at 640x480#60fps according to what Apple lets developers do and my current resizing is very slow.
I am using an image resizing with Metal kernel shaders, but 98% of the time (seen in Instruments) is spent in these two lines:
[_inputTexture replaceRegion:region mipmapLevel:0 withBytes:inputImage.data bytesPerRow:inputImage.channels() * inputImage.cols];
...
[_outputTexture getBytes:outputImage.data bytesPerRow:inputImage.channels() * outputImage.cols fromRegion:outputRegion mipmapLevel:0];
basically in memory load/store instructions. This part puzzles me as in theory the memory is shared between CPU and GPU on iPhone.
Do you think Metal code should be faster? I also have video presentation with Metal and it doesn't break a sweat (~1ms on GPU, while resizing takes up to ~20ms).
Is there no faster way of resizing an image? How is your experience with Image I/O?
UPDATE:
When I increased the size of my work-groups to be 22x22x1, the performance of image resizing improved from 20ms of GPU to 8ms. Still not quite what I want, but better.
UPDATE 2:
I switched to CoreGraphics and it goes fast enough. See this post.

Answer 1

I think you have created UIImage from Capture session, and than you have to convert it to the Metal texture (MTLTexture). But you shouldn't do it. Here is an example how to get MTLTexture from CaptureOutput delegate:
- (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection {
CVPixelBufferRef pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
CVPixelBufferLockBaseAddress(pixelBuffer, 0);
id<MTLTexture> inTexture = nil;
size_t width = CVPixelBufferGetWidth(pixelBuffer);
size_t height = CVPixelBufferGetHeight(pixelBuffer);
MTLPixelFormat pixelFormat = MTLPixelFormatBGRA8Unorm;
CVMetalTextureRef metalTextureRef = NULL;
CVReturn status = CVMetalTextureCacheCreateTextureFromImage(NULL, _textureCache, pixelBuffer, NULL, pixelFormat, width, height, 0, &metalTextureRef);
if(status == kCVReturnSuccess) {
inTexture = CVMetalTextureGetTexture(metalTextureRef);
CFRelease(metalTextureRef);
}
CVPixelBufferUnlockBaseAddress(pixelBuffer,0);
}