Curious, is it possible to get [UIImage] from NSMutableData - ios

On my iOS side, i received a stream of images via NSInputStream. NSMutableData will be containing images in [JPG IMAGE][JPG IMAGE][PNG IMAGE].
Is there any elegant way of getting the images from NSMutableData?
EDIT: Thanks for all the replies. I do not know the start/end of the images. I converted the buffer to NSData and append it to NSMutableData. Do i need to include a delimiter in between the images?
Server(.Net): Continuous sending images, so on the client side (iOS) i will never encounter NSStreamEvent.EndEncountered . Closing the NetoworkStream will throw me ObjectDisposedException was unhandled, cannot access a disposed object (fixing it now)
Client: Does not know when the server has completed sending, all it has is a NSMutableData. (fixing it now) I convert the buffer to NSData and append it to NSMutableData
Basically what i want to achieve is:
1. I press connect on my client (iOS app),
2. Establish connection with the server (.Net)
3. Server create and send images to client
4. Client displays the images.
5. Connect and Disconnect initiated by client.
Update: I got it working via:
1. At server side, after each image sent, i append a delimeter.
2. At client side, i append the data to NSMutableData as usual and check the last few bytes for delimeter. If it matches, i can safely proceed to chunk the NSMutableData by the delimeter.
via that, i am able to receive & display multiple images!

You can use
[UIImage imageWithData:[yourMutableData copy]]
copy is not necessary. It makes your mutable data to immutable and you got copy.

You Can Use
NSData *data = yourData;
UIImage *image = [UIImage imageWithData:data];

It is possible, but it will be quite tricky in your case - because you indicated that the NSMutableData will contain more than one type of images (JPEG/PNG).
If we assume, that the data arrived in good format, the best way to explore and separate it is by the file signature:
For PNG it's the first 8 bytes
For JPEG it's the first 4 bytes
Incorporate this to separate the huge blob of data into an array of data chunks. You'll have to go byte by byte and look for subsequences and check if the subsequence happens to be of a JPEG or PNG file. Then simply initialize your UIImage objects like this:
var dataArray: [NSMutableData] = [imageData1, imageData2, imageData3]
var imageArray: [UIImage] = []
for imgData in dataArray {
if let image = UIImage(data: imgData) {
imageArray.append(image)
}
}

Related

Decompress Base64 Encoded Image Bytes in Swift

I am developing an iOS Application in Swift for use by customers of the company for which I work. In addition to the iOS application, I am also making the backend API with ASP.Net Core. The method I have chosen to use for sending images is a compressed base64 encoded string representation of the image in a JSON array. This is to save on bandwidth and to make it easier to send multiple images.
On the API side, I am using the .NET Class System.IO.Compression.DeflateStream to compress the image bytes in memory before encoding them to a base64 string and sending them to the iOS application.
On the iOS side, I am a little confused as to what the process would be for decoding the string and decompressing the data to create a UIImage object for display in a UIImageView.
This all works without compression, but I wanted to try this to save on bandwidth. However, it's very possible that this is not the optimal solution, and I am open to change. Below is the snippet of code I am using to convert the base64 string to a Data object, then creating a UIImage object from that.
static func imageComplete(data json: JSON) -> [UIImage] {
var images: [UIImage] = []
for imageString in json.arrayValue {
if let compressedImageData = Data(base64Encoded: imageString.stringValue),
let image = UIImage(data: compressedImageData) {
images.append(image)
}
}
return images
}
TL;DR
In Swift, I want to decompress image bytes encoded as a Base64 string returned from an ASP.Net Core WebAPI.

Detecting that iOS image data is HEIF or HEIC

My server doesn't support the HEIF format. So I need to transform it to JPEG before uploading from my app.
I do this:
UIImage *image = [UIImage imageWithData:imageData];
NSData *data=UIImageJPEGRepresentation(image, 1.0);
But how can I know that the data is HEIF (or HEIC) ? I can look at a file:
([filePath hasSuffix:#".HEIC"] || [filePath hasSuffix:#".heic"])
But I don't think it's a good answer. Is there any other solution?
Both existing answers have good recommendations, but to attempt to tell the whole story...
UIImage doesn't represent an image file or even binary data in an image-file format. A UIImage is best thought of as an abstract representation of the displayable image encoded in that data — that is, a UIImage is the result of the decoding process. By the time you have a UIImage object, it doesn't care what file format it came from.
So, as #Ladislav's answer notes, if you have a UIImage already and you just want to get data in a particular image file format, call one of the convenience functions for getting a UIImage into a file-formatted data. As its name might suggest, UIImageJPEGRepresentation returns data appropriate for writing to a JPEG file.
If you already have a UIImage, UIImageJPEGRepresentation is probably your best bet, since you can use it regardless of the original image format.
As #ScottCorscadden implies, if you don't have a UIImage (yet) because you're working at a lower level such that you have access to the original file data, then you'll need to inspect that data to divine its format, or ask whatever source you got the data from for metadata describing its format.
If you want to inspect the data itself, you're best off reading up on the HIEF format standards. See nokiatech, MPEG group, or wikipedia.
There's a lot going on in the HEIF container format and the possible kinds of media that can be stored within, so deciding if you have not just a HEIF file, but an HEIF/HEVC file compatible with this-or-that viewer could be tricky. Since you're talking about excluding things your server doesn't support, it might be easier to code from the perspective of including only the things that your server does support. That is, if you have data with no metadata, look for something like the JPEG magic number 0xffd8ff, and use that to exclude anything that isn't JPEG.
Better, though, might be to look for metadata. If you're picking images from the Photos library with PHImageManager.requestImageData(for:options:resultHandler:), the second parameter to your result handler is the Uniform Type Identifier for the image data: for HEIF and HEIC files, public.heif, public.heif-standard, and public.heic have been spotted in the wild.
(Again, though, if you're looking for "images my sever doesn't support", you're better off checking for the formats your server does support and rejecting anything not on that list, rather than trying to identify all the possible unsupported formats.)
When you are sending to your server you are most likely decoding the UIImage and sending it as Data so just do
let data = UIImageJPEGRepresentation(image, 0.9)
Just decide what quality works best for you, here it is 0.9
A bit late to the party, but other than checking the extension (after the last dot), you can also check for the "magic number" aka file signature. Byte 5 to 8 should give you the constant "ftyp". The following 4 bytes would be the major brand, which I believe is one of "mif1", "heic" and "heix".
For example, the first 12 bytes of a .heic image would be:
00 00 00 18 66 74 79 70 6d 69 66 31
which, after removing 0s and trim the result, literally decoded to ftypmif1.
Well, you could look at magic bytes - JPEG and PNG certainly are known, and I seem to see some references that HEIF (.heic) starts with a NUL byte. If you're using any of the PHImageManager methods like requestImageDataForAsset:options:resultHandler, that resultHandler will be passed a NSString * _Nullable dataUTI reference. There's a decent WWDC video/slides on this (possibly here) that suggest if the UTI is not kUTTypeJPEG you convert it (and the slides have some lower-level sample code in swift to do it that preserve orientation too).
I should also mention, if you have control at your app layer and all uploads come from there, do all this there.
If you're using Photos framework and are importing images from photo library, there's a solution that was mentioned briefly during WWDC17. First, import core services:
import MobileCoreServices
Then, when you request the image, check the UTType that is returned as a second parameter to your block:
// asset: PHAsset
PHImageManager.default().requestImageData(for: asset, options: nil) { imageData, dataUTI, orientation, info in
guard let dataUTI = dataUTI else { return }
if !(UTTypeConformsTo(dataUTI as CFString, kUTTypeJPEG) || UTTypeConformsTo(dataUTI as CFString, kUTTypePNG)) {
// imageData is neither JPG not PNG, possibly subject for transcoding
}
}
Other UTTypes can be found here

REST web service to recover an array of pictures

I want to implement a web service to recover an array of my entity PictureCaptures :
PictureCaptures
---------------
- description : string
- captureDate : DateTime
- photoBinary : byte[]
The web service will be mainly called by an iOS application.
What's the best way to implement it, because of the byte array attribute?
Am I suppose to return the byte array without any transformation, as a simple JSON attribute? If yes, how to interpet the JSON response ? -In this case JSONObjectWithData:options:error: doesn't work, too much data and memory issue)-
Thank you for your help.
I would suggest you add two resources: one for the meta data (description, captureDate and so on) and one for the binary data. Let the meta data resource contain a link to the binary photo data.
Like this:
GET /images/1234
Response:
{
description: "Nice photo",
captureDate: "2012-04-23T18:25:43.511Z",
photoData: "http://example.org/images/1234/photo"
}
and http://example.org/images/1234/photo returns the raw photo data
(see also See also The "right" JSON date format for a discussion on date formats).
when you get JSON responce you shoud convert the btye array to NSData.
first add Base64.h and m file to the project ( you can find it easily on internet)
then import Base64.h
from your JSON data
NSString *data= [yourJSONDict objectForKey:#"photoBinary"];
NSData* imageData = [data base64DecodedData];
UIImage *imag=[UIImage imageWithData:imageData];
[yourImageView setImage:imag];
this might help you.

iOS NSJSONSerialization of JPG image encoded binary data from mongodb

I think I need some assistance in figuring out the correct NSJSONSerialization option to make my problem go away.
On my app I allow the user to select an image from the gallery - the image undergoes the following:
NSData *imageData = UIImageJPEGRepresentation(self.profileImageView.image, 0.0);
then
NSString *stringOfImageData = [imageData base64EncodedStringWithOptions:0];
before it is serialized like this:
NSData *jsonData = [NSJSONSerialization dataWithJSONObject:postDict
options:NSJSONWritingPrettyPrinted
error:&error];
and then sent to my REST API. I then decode it in python using base64 like so:
profileImageData = base64.b64decode(request.json['image'])
It is then loaded in GridFS (mongodb). On extracting the data to send back to the app I first encode in base to base64 before using dumps() to send it back:
dumps(base64.b64encode(fs.get_last_version(request.json['userID']).read()))
Within iOS after receiving the data it goes through the below de-serialization:
[NSJSONSerialization JSONObjectWithData:data options:NSJSONReadingMutableContainers|NSJSONReadingMutableLeaves error:&error]
I have narrowed by problem to the last NSJSONSerialization command. After the data is received by the app it is able to print to screen. After the Serialization I get a 'nil' :(
The Serialization and De-Serialization has been working great for strings, integers etc - it just doesn't work when I'm trying to move image data.
Thanks
EDIT: I am able to run a curl request against the API and then using an online base64 to image converter I can see my image. So it definitely means the issues is with the iOS side of decoding a json encoded base64 string.
EDIT: When I repeatedly run the deserialization - every 20th time or so the data is correctly converted. I think the solution might have to be to break up the data coming in.
EDIT: Error:
parsed error:Error Domain=NSCocoaErrorDomain Code=3840 "The operation couldn’t be completed. (Cocoa error 3840.)" (Unterminated string around character 17.) UserInfo=0x109c08790 {NSDebugDescription=Unterminated string around character 17.}
What you don't say is how you are receiving the data. My guess is you are trying to decode the data before you receive all of it, but since I don't know how it's a guess.
To better understand what's going on, try logging the size and hash of the data, to see if the length varies. You can also save each received data object to the file system - put them in the Documents folder and you can access them from your Mac. If the size never varies you will then have to compare a good data object to a bad one.
In fact you can write a little code to save an image as data and a base64 string, upload it, then pull it back, and save it. Now compare the data and strings. Once you find a difference, then look at. What is its offset from the start? How is it different?
When you understand this all you will be able to fix it.

sending images to server via json

I am posting data to a server from my ipad using json. But along with my data, i need to be able to send images aswell. I tried to add my image data to a dictionary and parse that into json, but json doesnt like nscfdata. What would be the easiest way i can post my images to the server? from other posts related to this topic, people have been converting to base64. Would i have to do that, or is there another easier or faster way? if i have to encode to base64 is there any tutorial on that?
I convert it to base64. Check out this tutorial to get started!
http://www.cocoawithlove.com/2009/06/base64-encoding-options-on-mac-and.html
Example:
NSImage *image = // some image
NSData *data = UIImagePNGRepresentation(image);
NSString *base64EncodedString = [data base64EncodedString];
You can then send the string via JSON.

Resources