iOS version: 13.1
iPhone: X
I'm currently using DBAttachmentPickerController to choose from a variety of images, the problem comes when I take a picture directly from the camera and try to upload it to our server. The SDImageWebPCoder.shared.encodedData loads for about 30 seconds more less. The same image in the Android app takes about 2-3 seconds.
Here is the code I use
let attachmentPickerController = DBAttachmentPickerController(finishPicking: { attachmentArray in
self.images = attachmentArray
var currrentImage = UIImage()
self.images[0].loadOriginalImage(completion: { image in
self.userImage.image = image
currrentImage = image!
})
//We transform it to webP
let webpData = SDImageWebPCoder.shared.encodedData(with: currrentImage, format: .webP, options: nil)
self.api.editImageUser(data: webpData!)
}, cancel: nil)
attachmentPickerController.mediaType = DBAttachmentMediaType.image
attachmentPickerController.allowsSelectionFromOtherApps = true
attachmentPickerController.present(on: self)
Should I change the Pod I'm using? Should I just compress it? Or am I doing something wrong?
WebP encoding speed is related slow, it use software encoding and VP8 compression algorithm (complicated), compared to the Hardware accelerated JPEG/PNG encoding. (Apple's SoC).
picture directly from the camera
The original image taken on iPhone camera may be really lark, like 4K resolution. If you don't do some pre-scale and try to encode it, you may consume much more time.
The suggestion can be like this:
Try to use the options like the compressionQuality, the higher cost
more time, but compress more.By default it's 1.0, which is the higest and most time consuming.
Try to pre-scale the original image. For image from Photos Libraray, you can always use the API to control the size. Or, you can use SDWebImage's transform method like - [UIImage sd_resizedImage:].
Do all the encoding in background thread, never block main thread
If all these is not suitable, the better solution, it's to use JPEG and PNG format instead of WebP. Then, on your image server side code, transcoding the JPEG/PNG to WebP. Server side processing is always the best idea for this thing.
If you're intersted the real benchmark or something, compared to JPEG/PNG (Hardware) and WebP (Software). You can try to use my benchmark code demo here, to help you do your decision.
https://github.com/dreampiggy/ModernImageFormatBenchmark
Related
In my iOS app I need to take an imageURL string and convert it into a UIImage.
I wrote the below function to handle this:
func getImage(urlString: String) -> UIImage {
let url = URL(string: urlString)!
do {
let data = try Data(contentsOf: url)
let image = UIImage(data: data)!
return image
} catch {
print(error, " This was the error in p2")
}
return UIImage(named: "media")!
}
The issue is that this takes too long. I believe it's a solid second or longer for this to complete.
I need that time to be significantly shorter.
Question: Is there a faster way to get the UIImage based on an imageURL from Firebase? (maybe a cocoa-pod? or better way to write the code?)
Additional questions:
Would this be any faster if the image in Firebase were of lower quality?
Would it be a viable solution to lower the quality of the image right before being passed into this function?
A lot the prominent iOS apps (and web and mobile and general) that do a lot of downloading of images take advantage of progressive jpeg. This way your user will see at least something while the image loads and over time the image will get progressively better. As a lot of commenters have mentioned, you’re not in control of Firebase like you would be if you had your own backend server delivering the pictures that you could do performance optimizations. Therefore one of the best things you can do is implement progressive jpeg in your app.
The first link is a library that will allow you to use progressive jpeg in your iOS app. The second link is a detailed approach used at FaceBook on faster loading of images.
https://www.airpair.com/ios/posts/loading-images-ios-faster-with-progressive-jpegs
https://code.fb.com/ios/faster-photos-in-facebook-for-ios/
I am working on an app that connects multiple devices using the networking protocol UPnP.
Among others, I can exchange images and they are files stored inside a folder in my project folder.
For sending the images I am given predefined methods.
if let imageRequested = UIImage(named:"logo.png"){
let fileType = "image/png"
let pngRepresentation = [UInt8](UIImagePNGRepresentation(imageRequested)!)
let fileSize = UInt32(pngRepresentation.count)
writeBegin!(aWriterData, fileSize, fileType)
aWriteResource!(aWriterData, pngRepresentation, fileSize)
aWriteEnd!(aWriterData)
}
However, the receiver gets the image with black background when it is transparent. I know It is not a receiver issue because there is an Android version that sends the same images and they are received with transparency.
any ideas why?
UIImageJPEGRepresentation will convert the resulting image into a JPEG, which doesn't support transparency. Are you sure that you use UIImagePNGRepresentation, not UIImageJPEGRepresentation?
If you use UIImagePNGRepresentation - it's strange, it should work fine, could you provide more details?
I want to build an iOS 10 app that lets you shoot a RAW (.dng) image, edit it, and then saved the edited .dng file to the camera roll. By combining code from Apple's 2016 "AVCamManual" and "RawExpose" sample apps, I've gotten to the point where I have a CIFilter containing the RAW image along with the edits.
However, I can't figure out how to save the resulting CIImage to the camera roll as a .dng file. Is this possible?
A RAW file is "raw" output direct from a camera sensor, so the only way to get it is directly from a camera. Once you've processed a RAW file, what you have is by definition no longer "raw", so you can't go back to RAW.
To extend the metaphor presented at WWDC where they introduced RAW photography... a RAW file is like the ingredients for a cake. When you use Core Image to create a viewable image from the RAW file, you're baking the cake. (And as noted, there are many different ways to bake a cake from the same ingredients, corresponding to the possible options for processing RAW.) But you can't un-bake a cake — there's no going back to original ingredients, much less a way that somehow preserves the result of your processing.
Thus, the only way to store an image processed from a RAW original is to save the processed image in a bitmap image format. (Use JPEG if you don't mind lossy compression, PNG or TIFF if you need lossless, etc.)
If you're writing the results of an edit to PHPhotoLibrary, use JPEG (high quality / less compressed if you prefer), and Photos will store your edit as a derived result, allowing the user to revert to the RAW original. You can also describe the set of filters you applied in PHAdjustmentData saved with your edit — with adjustment data, another instance of your app (or Photos app extension) can reconstruct the edit using the original RAW data plus the filter settings you save, then allow a user to alter the filter parameters to create a different processed image.
Note: There is a version of the DNG format called Linear DNG that supports non-RAW (or "not quite RAW") images, but it's rather rare in practice, and Apple's imaging stack doesn't support it.
Unfortunately DNG isn't supported as an output format in Apple's ImageIO framework. See the output of CGImageDestinationCopyTypeIdentifiers() for a list of supported output types:
(
"public.jpeg",
"public.png",
"com.compuserve.gif",
"public.tiff",
"public.jpeg-2000",
"com.microsoft.ico",
"com.microsoft.bmp",
"com.apple.icns",
"com.adobe.photoshop-image",
"com.adobe.pdf",
"com.truevision.tga-image",
"com.sgi.sgi-image",
"com.ilm.openexr-image",
"public.pbm",
"public.pvr",
"org.khronos.astc",
"org.khronos.ktx",
"com.microsoft.dds",
"com.apple.rjpeg"
)
This answer comes late, but it may help others with the problem. This is how I saved a raw photo to the camera roll as a .dng file.
Just to note, I captured the photo using the camera with AVFoundation.
import Photos
import AVFoundation
//reading in the photo data in as a data object
let photoData = AVCapturePhotoOutput.dngPhotoDataRepresentation(forRawSampleBuffer: sampleBuffer, previewPhotoSampleBuffer: previewPhotoSampleBuffer)
// put it into a temporary file
let temporaryDNGFileURL = NSURL(fileURLWithPath: NSTemporaryDirectory()).appendingPathComponent("temp.dng")!
try! photoData?.write(to: temporaryDNGFileURL)
// get access to photo library
PHPhotoLibrary.requestAuthorization { status in
if status == .authorized {
// Perform changes to the library
PHPhotoLibrary.shared().performChanges({
let options = PHAssetResourceCreationOptions()
options.shouldMoveFile = true
//Write Raw:
PHAssetCreationRequest.forAsset().addResource(with: .photo, fileURL: temporaryDNGFileURL, options: options)
}, completionHandler: { success, error in
if let error = error { print(error) }
})
}
else { print("cant access photo album") }
}
Hope it helps.
The only way to get DNG data as of the writing of this response (iOS 10.1) is:
AVCapturePhotoOutput.dngPhotoDataRepresentation(forRawSampleBuffer: CMSampleBuffer, previewPhotoSampleBuffer: CMSampleBuffer?)
Noting the OP refers to Core Image. As mentioned by rickster, CI works on processed image data, therefore only offers processed image results (JPEG, TIFF):
CIContext.writeJPEGRepresentation(of:to:colorSpace:options:)
CIContext.writeTIFFRepresentation(of:to:format:colorSpace:options:)
My server doesn't support the HEIF format. So I need to transform it to JPEG before uploading from my app.
I do this:
UIImage *image = [UIImage imageWithData:imageData];
NSData *data=UIImageJPEGRepresentation(image, 1.0);
But how can I know that the data is HEIF (or HEIC) ? I can look at a file:
([filePath hasSuffix:#".HEIC"] || [filePath hasSuffix:#".heic"])
But I don't think it's a good answer. Is there any other solution?
Both existing answers have good recommendations, but to attempt to tell the whole story...
UIImage doesn't represent an image file or even binary data in an image-file format. A UIImage is best thought of as an abstract representation of the displayable image encoded in that data — that is, a UIImage is the result of the decoding process. By the time you have a UIImage object, it doesn't care what file format it came from.
So, as #Ladislav's answer notes, if you have a UIImage already and you just want to get data in a particular image file format, call one of the convenience functions for getting a UIImage into a file-formatted data. As its name might suggest, UIImageJPEGRepresentation returns data appropriate for writing to a JPEG file.
If you already have a UIImage, UIImageJPEGRepresentation is probably your best bet, since you can use it regardless of the original image format.
As #ScottCorscadden implies, if you don't have a UIImage (yet) because you're working at a lower level such that you have access to the original file data, then you'll need to inspect that data to divine its format, or ask whatever source you got the data from for metadata describing its format.
If you want to inspect the data itself, you're best off reading up on the HIEF format standards. See nokiatech, MPEG group, or wikipedia.
There's a lot going on in the HEIF container format and the possible kinds of media that can be stored within, so deciding if you have not just a HEIF file, but an HEIF/HEVC file compatible with this-or-that viewer could be tricky. Since you're talking about excluding things your server doesn't support, it might be easier to code from the perspective of including only the things that your server does support. That is, if you have data with no metadata, look for something like the JPEG magic number 0xffd8ff, and use that to exclude anything that isn't JPEG.
Better, though, might be to look for metadata. If you're picking images from the Photos library with PHImageManager.requestImageData(for:options:resultHandler:), the second parameter to your result handler is the Uniform Type Identifier for the image data: for HEIF and HEIC files, public.heif, public.heif-standard, and public.heic have been spotted in the wild.
(Again, though, if you're looking for "images my sever doesn't support", you're better off checking for the formats your server does support and rejecting anything not on that list, rather than trying to identify all the possible unsupported formats.)
When you are sending to your server you are most likely decoding the UIImage and sending it as Data so just do
let data = UIImageJPEGRepresentation(image, 0.9)
Just decide what quality works best for you, here it is 0.9
A bit late to the party, but other than checking the extension (after the last dot), you can also check for the "magic number" aka file signature. Byte 5 to 8 should give you the constant "ftyp". The following 4 bytes would be the major brand, which I believe is one of "mif1", "heic" and "heix".
For example, the first 12 bytes of a .heic image would be:
00 00 00 18 66 74 79 70 6d 69 66 31
which, after removing 0s and trim the result, literally decoded to ftypmif1.
Well, you could look at magic bytes - JPEG and PNG certainly are known, and I seem to see some references that HEIF (.heic) starts with a NUL byte. If you're using any of the PHImageManager methods like requestImageDataForAsset:options:resultHandler, that resultHandler will be passed a NSString * _Nullable dataUTI reference. There's a decent WWDC video/slides on this (possibly here) that suggest if the UTI is not kUTTypeJPEG you convert it (and the slides have some lower-level sample code in swift to do it that preserve orientation too).
I should also mention, if you have control at your app layer and all uploads come from there, do all this there.
If you're using Photos framework and are importing images from photo library, there's a solution that was mentioned briefly during WWDC17. First, import core services:
import MobileCoreServices
Then, when you request the image, check the UTType that is returned as a second parameter to your block:
// asset: PHAsset
PHImageManager.default().requestImageData(for: asset, options: nil) { imageData, dataUTI, orientation, info in
guard let dataUTI = dataUTI else { return }
if !(UTTypeConformsTo(dataUTI as CFString, kUTTypeJPEG) || UTTypeConformsTo(dataUTI as CFString, kUTTypePNG)) {
// imageData is neither JPG not PNG, possibly subject for transcoding
}
}
Other UTTypes can be found here
I'm developing an iOS today extension, that can read an image from UIPasteboard and save it on disk. This process fails with large images because iOS extensions can't use much memory. To workaround this issue, I'm checking the size of the image first and try to decide, if the widget can save it or should delegate this task to its host app:
let MAXIMUM_IMAGE_SIZE_BYTES = <SomeMagicNumber>
if let clipboardImage = UIPasteboard.generalPasteboard().image {
let imageSize = CGImageGetHeight(clipboardImage.CGImage) * CGImageGetBytesPerRow(clipboardImage.CGImage)
if imageSize > MAXIMUM_IMAGE_SIZE_BYTES {
// Open host app to save image
}
else {
// Save image directly
}
}
I have the following questions:
Is my size calculation correct? I took it from this thread. I cannot instantiate a JPEG or PNG representation and read its size because of the memory limitations mentioned above.
Can I get rid of that magic number for the maximum image size in bytes? If not, are there any official specifications from Apple that I can use? I cannot test my app on every available iOS model and don't want to risk crashes on older devices.
Thanks a lot for your help!
I'm just starting to look at the memory that a notification service extension is using. I found this presentation. Might be helpful for others.
https://cocoaheads.tv/memory-use-in-extensions-by-conrad-kramer/
What was your solution to this issue?