I would like to add three "parts" to an NSInputStream: an NSString, an output from another stream and then another NSString. The idea is the following:
The first and last NSStrings represent the beginning and end of a SOAP request while the output from the stream is a result of loading a very large file and encoding it as Base64 string. So, in the end I would have the final NSInputStream hold the whole SOAP request like this:
< soap beginning > < Base64 encoded data > < soap ending >
The reason I want the whole request to be held in NSInputStream is two-fold:
I don't what to load the very large data file into memory
I think that this is the only way to enforce sending the final request in HTTP 1.1 chunks (which I need because otherwise, if the request becomes too big, the server won't accept it). So, I know that doing this:
NSInputStream *dataStream = ....;
[request setHTTPBodyStream:dataStream];
ensures that the request will be sent as HTTP 1.1 chunks and not as one huge raw SOAP request.
So, I wonder how this can be achieved - namely, how do I "enqueue" things into an NSInputStream? Can it be even done? Is there an alternative way?
Just for reference, in Java this can be done as follows
Vector<InputStream> streamVec = new Vector<InputStream>();
BufferedInputStream fStream = new BufferedInputStream(fileData.getInputStream());
Base64InputStream b64stream = new Base64InputStream(fStream, true);
String[] SOAPBody = GenerateSOAPBody(fileInfo).split("CUT_HERE");
streamVec.add(new ByteArrayInputStream(SOAPBody[0].getBytes()));
streamVec.add(b64stream);
streamVec.add(new ByteArrayInputStream(SOAPBody[1].getBytes()));
SequenceInputStream seqStream = new SequenceInputStream(streamVec.elements());
because Java has these objects available, but NSStreams in objective-c look like very low level objects and are very hard to work with.
Note: I completely re-wrote the original question as I asked it 2 days ago, since I think the new edit explains more clearly what the problem is. I hope it would help it be easier comprehended and maybe answered
UPDATE 2
Here is what I've been able to achieve so far: Instead of trying to enqueue into a stream, I am using a temp file to first write the < soap beginning >, then I set up an input stream to read from the file in chunks, encode each chunk as a Base64 string and write this to the same temp file, finally, when my stream closes, I write the < soap ending > to the temp file. Then I set up another input stream with the contents of this file which I pass to the NSMutableURLRequest:
NSMutableURLRequest* request = [NSMutableURLRequest requestWithURL:url];
...
NSInputStream *dataStream = [NSInputStream inputStreamWithFileAtPath:_tempFilePath];
[request setHTTPBodyStream:dataStream];
This ensures HTTP 1.1 chunked transfer of the contents of the file. After the connection finishes, delete the temp file.
This seems to work fine but of course this is an annoying work-about. I don't want to be writing to a temp file when it all could have been handled by streams (ideally.) If anybody still has better suggestions, let me know :)
UPDATE 3
OK, another update is in order. While my writing to file seems to work, I am now hitting an unexpected issue with some of my requests failing to upload to the server. Specifically, everything is going according to the plan, I am reading the contents of the temp file into a stream and set HTTP body of my request to be this stream and it starts transmitting the HTTP 1.1 chunks as I want it to - but for some reason some packets get dropped and the final request - this is my guess - gets malformed and thus fails. I think the issue of dropped packets is random, because I observe it on larger requests - that is, the issue just has more chance to show up - while my smaller requests usually go thru just fine. This is of course a separate issue from the original in this question. If anybody has a good idea what might be causing this, I asked about the problem here: Packets dropped during chunked HTTP 1.1 request sent by NSURLConnection
Your solution is an ok option, but you can do it with a stream. It means subclassing NSInputStream, and that isn't trivial because there are a bunch of methods you need to implement.
Basically your subclass would initially return the header bytes, then it would return bytes from the 'internal' stream to the file content, then when that's used up it returns the footer bytes. It means maintaining a record of how big the header and footer are and how much has been processed so far, but that isn't a big issue.
There's an example of creating a subclass here which shows the tricky hidden methods you need to implement to get the stream subclass to work properly without throwing exceptions.
Related
This is the code in Java to make the socket call, but I want to know how can I replicate this or something similar in iOS (Swift or Objective-C)
public String MakeSocketRequest() {
DataInputStream inputSt;
DataOutputStream outputSt;
Socket socket = new Socket(InetAddress.getByName("socketurl.io"), 40008);
String jsonStr = "{\"id\":1,\"method\":\"themethod\"}";
inputSt = new DataInputStream(socket.getInputStream());
outputSt = new DataOutputStream(socket.getOutputStream());
PrintWriter pw = new PrintWriter(outputSt);
pw.println(string);
Log.d("PrintWriter", jsonStr);
pw.flush();
BufferedReader bfr = new BufferedReader(new InputStreamReader(inputSt));
JSONObject json = new JSONObject(bfr.readLine());
Log.d("Json", json.toString());
inputSt.close();
outputSt.close();
return json.toString();}
If you want to do it natively without 3rd-party libraries,
then you can use CFStreamCreatePairWithSocketToHost function to create input and output streams (no socket object is needed).
Here's some example code to set this up
And the search shows many more
On iOS you can't write or read the streams immediately, and you have to wait until the socket is connected, and you get a permission to read/write. This is done by implementing NSStreamDelegate.
If you get NSStreamEventHasSpaceAvailable event there, you can write your string to the output stream. You don't need a PrintWriter to just write a string, because it is easy to convert NSString to NSData, and write NSData.
If you get NSStreamEventHasBytesAvailable event, means you can try to read data from the input stream to some buffer (like NSMutableData). There's no builtin BufferedReader with a readLine method, so you will have to buffer the data yourself and detect when a new line character appears there. After that you can cut a part of the buffer until the new line, and convert NSData to NSString (or a JSON object by using NSJSONSerialization).
Note: scheduleInRunLoop calls might look confusing, but they are required to start receiving events via the delegate. It kind of tells the system on which thread you want to receive them.
P.S. I agree with commenters that if you have control over the server code, it's better to use a standard protocol like Socket IO or msgpack instead of inventing your own, because they have better and nicer libraries and wider community support.
I have found many suggestion in stack overflow regarding usage of FFmpeg and link of github for DFURTSPPlayer but it is not compiling. But after integrating FFmpeg what I have to write? suppose i am having HTTP urls then I write:
code
moviePath = "http:/path.mp4"
movieURL = NSURL.URLWithString(moviePath!)
moviePlayer = MPMoviePlayerController(contentURL: movieURL)
moviePlayer!.play()
So for using RTSP urls what kind of code should i write?
Here is another post that has an example FFmpeg code that receives an RTSP stream (this one also decodes the stream to YUV420, stores it in pic, then converts the frame to RGB24, stores in picrgb and writes it to a file). So to achieve something similar to what you have for HTTP you should:
1) Write a wrapper Objective-C class for the FFmpeg C code, or just wrap the code in functions/functions that you will call directly from Objective-C code. You should have a way to pass the RTSP url to the class or function and provide a callback for a new frame. In the class/function start a new thread that will actually execute something similar to the code in the example and call a callback for each new decoded frame. NOTE: FFmpeg has a way to perform asynchronous I/O by using your own custom IO context and that would actually allow you to avoid creating the thread, but if you are new to FFmpeg maybe start with the basics and then you can improve your code later on.
2) In the callback update the view or whatever you are using for display with the decoded frame data.
I have some code that works fine with HTTP, but fails with (null) value returned from the NSJSONObjectserialization-thing on HTTPS.
I tried converting the received data to string, and it comes out to log as valid JSON. I validated it also to be sure. All good. The HTTPS certificate is valid, but I did try implementing the callbacks for invalid certs also, no change.
The EXACT same configuration works perfectly on HTTP, and if I go to the browser and input the same POST (I even tried removing the POST and just have the server return JSON no matter what you input) I get valid JSON also.
I tried converting the received data to an array and grab the first object in the array - didn't work either. I tried changing http headers in different configurations (application/json) etc. No dice.
Error code is Cocoa 3840. This error means, as far as I can tell, invalid JSON. HOW is this possible? It says character 3 is bad?
Any suggestions? I Googled the best I could but all I can find is "https and http should work in the same way" more or less.
Certificate is validated by RapidSSL.
NSDictionary* json = [NSJSONSerialization JSONObjectWithData:_responseDataNews options:kNilOptions error:&error];
_responseDataNews is the object that gets filled by didReceiveData delegate - and this exact config works on HTTP. What the #€%*# am I doing wrong here? It's driving me insane.
Exact error is:
Error Domain=NSCocoaErrorDomain Code=3840 "The operation couldn’t be completed. (Cocoa error 3840.)" (JSON text did not start with array or object and option to allow fragments not set.) UserInfo=0xbb2cde0 {NSDebugDescription=JSON text did not start with array or object and option to allow fragments not set.}
If I enable fragment option:
Error Domain=NSCocoaErrorDomain Code=3840 "The operation couldn’t be completed. (Cocoa error 3840.)" (Invalid value around character 3.) UserInfo=0xb939ab0 {NSDebugDescription=Invalid value around character 3.}
JSON looks like: {"r":1,"n":"Text here"}
And no, "text" contains no funky characters. This is the actual JSON value I'm testing.
Edit: NSLog of the data object reveals:
efbbbfef bbbfefbb bfefbbbf efbbbf7b 22687322 3a5b7b22 75736572 223a2254
65737465 73656e22 2c227363 6f726522 3a223233 3032227d 5d7d
(with JSON: {"hs":[{"user":"Testesen","score":"2302"}]} )
Edit: ... Okay, as it turns out, now it doesn't work on HTTP either, so it must be after changing the SSL setup or something on the server that it broke!? HTTP returns the exact same (invalid) data object :(
You report that the NSData for your string is:
<efbbbfef bbbfefbb bfefbbbf efbbbf7b 22687322 3a5b7b22 75736572 223a2254
65737465 73656e22 2c227363 6f726522 3a223233 3032227d 5d7d>
The ef bb bf is a byte order mark for a UTF-8 string. But if you have a BOM, you should have only one, and you have five of them.
If you're creating this response programmatically, you may have some server function that is called multiple times that is adding this BOM repeatedly. The BOM should appear only once (if at all), and the fact that it's appearing multiple times indicates a problem in your server code.
If you created this file manually, you might want to edit the hex data. Use whatever hex editor you want to fix this. For example, if you want to edit this file with Xcode, you can add it to your Xcode project and then right click on the file and choose "Open as..." - "Hex".
Okay - it turns out this had nothing to do with HTTPS anyway.
For some reason I cannot really understand, my php script was prefixing invalid characters after I restructured it to use only 1 .php file that responds to POST, includes all other .php files and returns the appropriate function depending on what you POST - of course I tried to just make it call one of the functions without POSTing anything, so that I could exclude POST errors from troubleshooting.
After I changed it so I request the specific file depending on what I want to do on the server - get news, get users, get score, whatever, like server.com/getscore.php instead of just server.com + POST data to get score, it works without issue and the JSON response is valid.
I don't really know how .php messes this up, but SOMETHING goes wrong. I tried enabling "Show all characters" in Notepad++, but I don't see anything funky anywhere. I made sure all .php files are encoded in UTF-8 also.
I am using Omniture SiteCatalyst in my iPhone app.It uses get request to hit the servers internally via its sdk.However i am facing an issue where some of the request are not reaching the Omniture servers.The get request which is being sent is of variable length depending on the type of request(around 900 + characters).
My question is whether there any limit for the get request length in an iOS app? and if yes
how it would behave in case the request crosses the limit?
Theoretically if URL conforms to RFC 2396 it is fine. According to documentation
The NSURL class fails to create a new NSURL object if the path being
passed is not well-formed; the path must comply with RFC 2396.
Examples of cases that will not succeed are strings containing space
characters and high-bit characters. Should creating an NSURL object
fail, the creation methods return nil, which you must be prepared to
handle. If you are creating NSURL objects using file system paths, you
should use fileURLWithPath: or initFileURLWithPath:, which handle the
subtle differences between URL paths and file system paths. If you
wish to be tolerant of malformed path strings, you’ll need to use
functions provided by the Core Foundation framework to clean up the
strings.
But some time there is issue with specail character e.g. space, accents and others. You must [NSURL URLWithString:[urlString stringByAddingPercentEscapesUsingEncoding: NSUTF8StringEncoding]];
It is also possible server could not handle very long urls, if there are any limitation on server, server will simple truncate the rest of url string, if this is an issue then server will return 414 error url too long.
I'm looking for ideas on how to improve a process of decoding a 40+MB NSString with base64 encoding and saving it to a file while being able to fit the process into iPad 1's 256 MB of RAM
I get the NSString from NSXMLParser:
id pointerToString;
- (void)parser:(NSXMLParser *)parser foundCharacters:(NSString *)string{
if ([currentElement isEqualToString:#"myElement"])
{
pointerToString = [string retain];
}
}
Then I use the pointerToString in a callback:
[handler performSelector: action withObject: pointerToString];
In the callback (id value is the pointerToString). I initialize NSData with the pointerToString while decoding it with base64 encoding.
^(id value)
{
if ( [[value class] isSubclassOfClass:[NSString class]] )
{
NSData *data = [NSData dataFromBase64String:value];
[data writeToFile:file.path atomically:YES];
}
}
the iPad 1 device runs out of memory and gets killed by the iOS when the memory allocation reaches around 130MB after or during the NSData call.
I have determined that in order to process the 40+MB NSString this way, I'd need about 180+MB of RAM (this is what the maximum memory allocation is on iPad 2 & 3, where the process works because of more RAM)
Any ideas/tips ?
Thank you
Edit:
When dealing with a file of this size, you probably do not want to load the entire multi-megabyte file in memory at one time, neither the huge input file nor the almost-as-huge output file. You should be parsing this in a streaming fashion, decoding the data in your foundCharacters as you go along, not holding any significant portions in memory.
The traditional techniques, though, may hold your entire XML file memory in three phases of the process:
As you download the XML file from the server;
As the XML parser parses that file; and
As you do the Base64-decode of the file.
The trick is to employ a streaming technique, that does these three processes at once, for small chunks of the single, large XML file. Bottom line, as you're downloading the entire 50mb file, grab a few kb, parse the XML, and if you're parsing the Base64-encoded field, perform the Base64-decode for that few kb, and the proceed to the next chunk of data.
For an example of this (at least the streaming XML downloading-and-parsing, not including the Base64-decoding), please see Apple's XMLPerformance sample project. You'll see that it will demonstrate two XML parsers, the NSXMLParser that we're all familiar with, as well as the less familiar LibXML parser. The issue with NSXMLParser is that, left to it's own devices, will load the entire XML file in memory before it starts parsing, even if you use initWithContentsOfURL.
In my previous answer, I mistakenly claimed that by using initWithContentsOfURL, the NSXMLParser would parse the URL's contents in nice little packets as they were being downloaded. The foundCharacters method of NSXMLParserDelegate protocol seems so analogous to the NSURLConnectionDelegate method, didReceiveData, that I was sure that NSXMLParser was going to handle the stream just like NSURLConnection does, namely returning information as the download was in progress. Sadly, it doesn't.
By using LibXML, though, like the Apple XMLPerformance sample project, you can actually use the NSURLConnection ability of streaming, and thus parse the XML on the fly.
I have created a little test project, but I might suggest that you go through Apple's XMLPerformance sample project in some detail. But in my experiment, a 56mb XML file consumed well over 100mb when parsing and converting via NSXMLParser but only consumed 2mb when using LibXML2.
In your comments, you describe the desire to download the Base64-encoded data to a file and then decode that. That approach seems a lot less efficient, but certainly could work. By the way, on that initial download, you have the same memory problem (that I solve above). I urge you to make sure that your initial download of the Base64-encoded data does not blithely load it into RAM like most routines do. You want to, assuming you're using NSURLConnection, write the data to the NSOutputStream as you receive the data in didReceiveData, not hold it in RAM.
See the didReceiveResponse in AdvancedGetController.m of Apple's AdvancedURLConnections example for an example of how to write a file as it's being received, rather than typical patterns of adding it to a NSMutableData (because most of these routines just assume you're dealing with a reasonably sized file). (Ignore all the stuff in that AdvancedURLConnections sample about authentication and the like, but focus on understanding how it's writing to the NSOutputStream as it goes.) This technique will address the first of the three problems listed at the top of this answer, but not the latter two. For that, you'll have to contemplate using LibXML2 as illustrated in Apple's XMLPerformance sample project, or other similar techniques.
The method
- (void)parser:(NSXMLParser *)parser foundCharacters:(NSString *)string
is probably not receiving all the data at once.
Doc is saying
"Sent by a parser object to provide its delegate with a string representing all or part of the characters of the current element."
So it is called multiple times.
It looks like you are trying to write the whole string at once (sorry if I am wrong).
So you could append the received data to the file by doing the following:
You can use a combination of
-writeData:
and
-seekToEndOfFile
methods from NSFileHandle class for writing NSData to the end of a file.
But be carefull with your base64 encoding on partial data receivment!