Get Disparity or Depth Data from PHImageManager - ios

Question

I try to get disparity or depth map from PHAsset. I've found example where the map is got from a image loaded with PHImageMager and implemented it:
- (AVDepthData*)getDepthDataFromSource:(CGImageSourceRef)source
{
NSDictionary* depthData = CFBridgingRelease(CGImageSourceCopyAuxiliaryDataInfoAtIndex(source, 0, kCGImageAuxiliaryDataTypeDepth));
if (!depthData){
depthData = CFBridgingRelease(CGImageSourceCopyAuxiliaryDataInfoAtIndex(source, 0, kCGImageAuxiliaryDataTypeDisparity));
}
if (!depthDat)
return nil; //code returns here
NSError* creationError = nil;
AVDepthData* data = [AVDepthData depthDataFromDictionaryRepresentation:depthData
error:&creationError];
return data;
}
//from a image
[[PHImageManager defaultManager] requestImageForAsset:asset
targetSize:size
contentMode:contentMode
options:requestOptions
resultHandler:^(UIImage * _Nullable result, NSDictionary * _Nullable info) {
CGImageRef im = image.CGImage;
CGImageSourceRef source = CGImageSourceCreateWithDataProvider(CGImageGetDataProvider(im), (CFDictionaryRef)#{});
AVDepthData* data = [self getDepthDataFromSource:source];//data is nil
if (source != nil)
{
CFRelease(source);
}
}];
//from a data
PHImageRequestOptions* imageDataRequestOptions = [[PHImageRequestOptions alloc] init];
imageDataRequestOptions.networkAccessAllowed = YES;
imageDataRequestOptions.deliveryMode = PHImageRequestOptionsDeliveryModeHighQualityFormat;
PHImageRequestID requestId = [[PHImageManager defaultManager] requestImageDataForAsset:asset
options:imageDataRequestOptions
resultHandler:^(NSData * _Nullable imageData, NSString * _Nullable dataUTI, UIImageOrientation orientation, NSDictionary * _Nullable info) {
CGImageSourceRef source = CGImageSourceCreateWithData((CFDataRef)imageData, (CFDictionaryRef)#{});
AVDepthData* data = [self getDepthDataFromSource:source];//data is nil
if (source != nil)
{
CFRelease(source);
}
}
];
//from full resolution image
__block FADisparityDataReader* selfStong = self;
PHContentEditingInputRequestOptions* options = [[PHContentEditingInputRequestOptions alloc] init];
options.networkAccessAllowed = YES;
[asset requestContentEditingInputWithOptions:options
completionHandler:^(PHContentEditingInput * _Nullable contentEditingInput, NSDictionary * _Nonnull info) {
NSURL* fullSizePath = [contentEditingInput fullSizeImageURL];
CGImageSourceRef source = CGImageSourceCreateWithURL((CFURLRef)fullSizePath, (CFDictionaryRef)#{});
#onExit{
CFRelease(source);
};
AVDepthData* data = [self readDepthDataFromSource:source]; //data is not nil
complition(data);
}];
I get depth data only in requestContentEditingInputWithOptions block, but it is too long and I believe that I can get a deep map from PHImageManager images.
How I can get the data from PHImageManager?

Answer 1

The 2017 WWDC session 'Editing with Depth' has some guidance, but does not have a full code example.
You can use the Apple sample app 'PhotoBrowse' to test the methods to access the depth data.
See my modified PhotoBrowse that loads depth and disparity images. It uses the Accelerate framework vDSP vector functions to normalize disparity to 0..1 rang
https://github.com/racewalkWill/PhotoBrowseModified
#IBAction func showDepthBtn(_ sender: UIBarButtonItem) {
// show the depth data from portrait mode from
// iPhone 7 Plus and later camera
requestDepthMap(selectedAsset: asset)
}
func requestDepthMap(selectedAsset: PHAsset) {
// may not have depthData in many cases
// PH completionHandler may be invoked multiple times
var auxImage: CIImage?
let options = PHContentEditingInputRequestOptions()
selectedAsset.requestContentEditingInput(with: options, completionHandler: { input, info in
guard let input = input
else { NSLog ("contentEditingInput not loaded")
return
}
auxImage = CIImage(contentsOf: input.fullSizeImageURL!, options: [CIImageOption.auxiliaryDepth: true])
if auxImage != nil {
let uiImage = UIImage(ciImage: auxImage! )
self.imageView.image = uiImage
}
} )
}
The key point is the PHAsset requestContentEditingInput completion handler is used to get the auxiliaryDepth (or disparity) ciImage.
The session demo discussed the scaling and normalization but did not publish any sample code or app. See my modified PhotoBrowse app for normalizing the CVPixelBuffer depth data.
Also be sure to use the iPhone portrait mode to take a picture with depth. Almost all the images in my personal photo library do not have depth data.
It's been a while since your question, so you may have worked this out already.