I would like to implement STARTTLS in an iOS application but I'm having trouble figuring out from the documentation how to do it.
So far I have:
I create the socket with CFStreamCreatePairWithSocketToHost and open the streams:
CFStreamCreatePairWithSocketToHost(
NULL, (__bridge CFStringRef)host, port,
&read_stream, &write_stream
);
reader = objc_retainedObject(read_stream);
[ reader setDelegate:self ];
[ reader scheduleInRunLoop:[NSRunLoop mainRunLoop] forMode:NSDefaultRunLoopMode ];
[ reader open ];
writer = objc_retainedObject(write_stream);
[ writer setDelegate:self ];
[ writer scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode ];
[ writer open ];
I get the right callbacks when data are available on the streams so the connection is working.
I successfully interact with the server in plain text and negociate STARTTLS.
Eventually, the server sends the go-ahead for STARTTLS:
. OK Begin TLS negotiation now.
Now it is time to upgrade the socket form plaintext to TLS. What do I do next?
I thought I should do this as per Apple's documentation:
[ reader setProperty:NSStreamSocketSecurityLevelNegotiatedSSL forKey:NSStreamSocketSecurityLevelKey ];
[ writer setProperty:NSStreamSocketSecurityLevelNegotiatedSSL forKey:NSStreamSocketSecurityLevelKey ];
But that doesn't seem to do anything. I'm not surprised that it doesn't work since the documentation says quite clearly:
You must set the property before you open the stream.
Yet of course the stream must be already opened in this case since it is used to conduct the plaintext STARTTLS negociation!
I cannot find any documentation on how to upgrade a socket from plaintext to SSL or perhaps how to layer a new set of SSL-encrypted streams on top of a set of input&output plaintext streams.
I could not find any way to implement STARTTLS using the high-level CFStream API. There is CFStreamCreatePairWithSocket which allows you to connect your own socket and then apply TLS to it afterwards, but there is no way to get the library to verify the remote host name against the certificate host name.
The only way to do it appears to be using this much lower level library: Secure Transport Reference.
Related
I am new to network programming, and I have some confusion with select function.
For a server program, we need to first create a fd to socket endpoint (server's ip and port without client's ip and port) with socket, bind and listen, then if there is a TCP connection to this socket endpoint, then accept returns the fd to the socket (server's ip, port and client's ip, port). Then we use recv on this socket's fd, and if there is no data to receive, the recv will block (for blocking socket).
I learned that select is used to handle non-blocking multiple connections. But which connection level does it handles? Does it handles multiple socket endpoints, or handles multiple sockets of one single socket endpoint?
For a normal server program, I think the socket endpoint is always single, and there are maybe hundreds of sockets connected to this endpoint. So I think handling multiple socket endpoints may be less useful to handling multiple sockets. But when talking about IO multiplexing, I find that many articles seems talking about handling multiple socket endpoints. While for handling multiple sockets to a single socket, I can't find a way to get all sockets, and put them to select's set of fds, since accept only accepts one sockets a time.
"Does it handles multiple socket endpoints or handles multiple sockets of one single socket endpoint?" -- There is no such thing as multiple sockets of a single socket endpoint. Every socket is an endpoint to the network communication. It just happens that the piece of code which deals with that socket might be different from the others. Consider the following socket descriptors:
int sock_acpt = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
listen(sock_acpt, 5);
int sock_cli = accept(sock_apt, ....);
Both the socket descriptors sock_acpt and sock_cli are the endpoints of different communications. After setting sock_acpt in passive mode by calling listen(), the socket just listens for TCP connection, and the server's TCP stack will manage any data that appears on that socket (most probably TCP Handshakes). While sock_cli is the end of an already established connection, and in general, the data on that socket is managed by the user-level application.
Now coming to select(), it is an IO Multiplexer, not a Network IO multiplexer. So any descriptor which can be viewed as an IO endpoint can be used with the select(). Referring to our socket descriptors sock_acpt and sock_cli, both are IO endpoints of different communications, so you can use both of them with select(). You generally do something like below
for ( ; ; ) {
fd_set rd_set;
FD_ZERO(&rd_set);
FD_SET(sock_acpt, &rd_set);
FD_SET(sock_cli, &rd_set);
if (select(sock_acpt > sock_cli ? sock_acpt + 1 : sock_cli + 1, \
&rd_set, NULL, NULL, NULL) <= 0) {
continue;
}
if (FD_ISSET(sock_acpt, &rd_set)) {
// Accept new connection
// accept(sock_acpt, ...);
}
if (FD_ISSET(sock_cli, &rd_set)) {
// Read from the socket
// read(sock_cli, ...);
}
}
But using select() is not limited to sockets, you can use with the file IO (fileno(stdin)), with the signal IO (signalfd()) and any other which can be viewed as IO endpoint.
Simple question:
Can I use NSInputStream to get data from the streaming API?
If so, can someone outline what this can look like.
I have tried something like
_twitterStream = [[NSInputStream alloc]initWithURL:[NSURL URLWithString:#"https://sitestream.twitter.com/1.1/site.json"]];;
_twitterStream.delegate = self;
[_twitterStream scheduleInRunLoop:[NSRunLoop currentRunLoop]
forMode:NSDefaultRunLoopMode];
and adding Delegate Methods, but this is not working. I realize I need to specify what users I need, however, above code is what I have so far.
Thanks!
One approach is using input streams and I do not think you would be able to pull it of easily as connecting to the streaming api requires a couple key http request headers.
CFReadStreamRef readStream;
CFStreamCreatePairWithSocketToHost(nil, (CFStringRef)#"Your IP Here", PORT_NUMBER, &readStream, nil);
self.stream = (__bridge NSInputStream *)(readStream);
[self.stream setDelegate:self];
[self.stream scheduleInRunLoop:[NSRunLoop currentRunLoop]
forMode:NSDefaultRunLoopMode];
Is a typical pattern of connecting to a host server with an input stream.
Another way is to just directly hit the URL like you do.
The problem with these approaches is that they do not address the authentication issues.
The easiest way I found was to use an NSURLSession task with the correct request headers set. Then you start receiving the data packets in the sessionDataDelegate.
Generating the twitter OAuth header can be worse than a prostate exam (trust me on this).
https://dev.twitter.com/oauth/overview/authorizing-requests
Here is a demo project where I hit the stream and dequeue searched for tweets in a tableview.
https://github.com/GregPrice24/TwitterStreamingAPI
I'm trying to write OS X app sending push notifications with use of NSStreams. Everything seems to be fine but i get back error -9844 when sending data to gateway.sandbox.push.apple.com. From SecureTransport.h
errSSLConnectionRefused = -9844, /* peer dropped connection before responding */
This is how I create and open output stream:
CFStreamCreatePairWithSocketToHost(NULL, (__bridge CFStringRef)#"gateway.sandbox.push.apple.com", 2195, &readStream, &writeStream);
self.outputPushStream = (__bridge_transfer NSOutputStream *)writeStream;
[self.outputPushStream setProperty:NSStreamSocketSecurityLevelNegotiatedSSL forKey:NSStreamSocketSecurityLevelKey];
self.outputPushStream.delegate = self;
[self.outputPushStream scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSRunLoopCommonModes];
[self.outputPushStream open];
And upon receiving NSStreamEventOpenCompleted event in NSStream delegate I send data:
const uint8_t *data = pushData.bytes;
[self.outputPushStream write:data maxLength:pushData.length];
I'm guessing that this may be some issue with certificates. I've got dev certificate and key for push notifications in keychain. I don't have any experience with SSL or NSStream so I don't really know where to look next.
I'm not sure if certificate in a keychain is enough for CFStreamCreatePairWithSocketToHost to set up an ssl connection. I found this code which does pretty much the same you want to do, but it passes certificate with code (see configureStreams and certificateArray methods).
I am making a simple UDP P2P Chat Program with a well known server.
The client's send and recieve data from server and clients through a single IdUDPServer.
The clients as of now can login and logout i.e. they can send data to the server.
Whenever the server sends any data it gets dropped at the NIC side of the node as the embedded ip header checksum is 0x00 as notified by wireshark.
IdUDPServer Settings (Client/Server)
Active : True
Bindings :
Broadcast : False
BufferSize : 8192
DefaultPort : 10000
IPVersion : Id_IPv4
ThreadedEvent : False
Command Used
only one command is used within
UDPServer.SendBuffer ( ED_Host.Text, StrToInt ( ED_Port.Text ), Buffer );
A similar configuration is working perfectly in another program of mine.
Most NICs will perform checksum validation and generation these days instead of the os network stack. This is to improve performance and is known as checksum offloading. As such wiresshark will report the fact the checksum is missing as an error but it can usually be ignored or the error turned off in the wire shark settings.
Some NIC drivers allow you to turn off checksum offloading. Try this and retest the code
I have read that HttpURLConnection supports persistent connections, so that a connection can be reused for multiple requests. I tried it and the only way to send a second POST was by calling openConnection for a second time. Otherwise I got a IllegalStateException("Already connected");
I used the following:
try{
URL url = new URL("http://someconection.com");
}
catch(Exception e){}
HttpURLConnection con = (HttpURLConnection) url.openConnection();
//set output, input etc
//send POST
//Receive response
//Read whole response
//close input stream
con.disconnect();//have also tested commenting this out
con = (HttpURLConnection) url.openConnection();
//Send new POST
The second request is send over the same TCP connection (verified it with wireshark) but I can not understand why (although this is what I want) since I have called disconnect.
I checked the source code for the HttpURLConnection and the implementation does keep a keepalive cache of connections to the same destinations. My problem is that I can not see how the connection is placed back in the cache after I have send the first request. The disconnect closes the connection and without the disconnect, still I can not see how the connection is placed back in the cache. I saw that the cache has a run method to go through over all idle connections (I am not sure how it is called), but I can not find how the connection is placed back in the cache. The only place that seems to happen is in the finished method of httpClient but this is not called for a POST with a response.
Can anyone help me on this?
EDIT
My interest is, what is the proper handling of an HttpUrlConnection object for tcp connection reuse. Should input/output stream be closed followed by a url.openConnection(); each time to send the new request (avoiding disconnect())? If yes, I can not see how the connection is being reused when I call url.openConnection() for the second time, since the connection has been removed from the cache for the first request and can not find how it is returned back.
Is it possible that the connection is not returned back to the keepalive cache (bug?), but the OS has not released the tcp connection yet and on new connection, the OS returns the buffered connection (not yet released) or something similar?
EDIT2
The only related i found was from JDK_KeepAlive
...when the application calls close()
on the InputStream returned by
URLConnection.getInputStream(), the
JDK's HTTP protocol handler will try
to clean up the connection and if
successful, put the connection into a
connection cache for reuse by future
HTTP requests.
But I am not sure which handler is this. sun.net.www.protocol.http.Handler does not do any caching as I saw
Thanks!
Should input/output stream be closed
followed by a url.openConnection();
each time to send the new request
(avoiding disconnect())?
Yes.
If yes, I can not see how the connection is being
reused when I call
url.openConnection() for the second
time, since the connection has been
removed from the cache for the first
request and can not find how it is
returned back.
You are confusing the HttpURLConnection with the underlying Socket and its underlying TCP connection. They aren't the same. The HttpURLConnection instances are GC'd, the underlying Socket is pooled, unless you call disconnect().
From the javadoc for HttpURLConnection (my emphasis):
Each HttpURLConnection instance is
used to make a single request but the
underlying network connection to the
HTTP server may be transparently
shared by other instances. Calling the
close() methods on the InputStream or
OutputStream of an HttpURLConnection
after a request may free network
resources associated with this
instance but has no effect on any
shared persistent connection. Calling
the disconnect() method may close the
underlying socket if a persistent
connection is otherwise idle at that
time.
I found that the connection is indeed cached when the InputStream is closed. Once the inputStream has been closed the underlying connection is buffered. The HttpURLConnection object is unusable for further requests though, since the object is considered still "connected", i.e. its boolean connected is set to true and is not cleared once the connection is placed back in the buffer. So each time a new HttpUrlConnection should be instantiated for a new POST, but the underlying TCP connection will be reused, if it has not timed out.
So EJP answer's was the correct description. May be the behavior I saw, (reuse of the TCP connection) despite explicitly calling disconnect() was due to caching done by the OS? I do not know. I hope someone who knows can explain.
Thanks.
How do you "force use of HTTP1.0" using the HttpUrlConnection of JDK?
According to the section „Persistent Connections” of the Java 1.5 guide support for HTTP1.1 connections can be turned off or on using the java property http.keepAlive (default is true). Furthermore, the java property http.maxConnections indicates the maximum number of (concurrent) connections per destination to be kept alive at any given time.
Therefore, a "force use of HTTP1.0" could be applied for the whole application at once by setting the java property http.keepAlive to false.
Hmmh. I may be missing something here (since this is an old question), but as far as I know, there are 2 well-known ways to force closing of the underlying TCP connection:
Force use of HTTP 1.0 (1.1 introduced persistent connections) -- this as indicated by the http request line
Send 'Connection' header with value 'close'; this will force closing as well.
Abandoning streams will cause idle TCP connections. The response stream should be read completely. Another thing I overlooked initially, and have seen overlooked in most answers on this topic is forgetting to deal with the error stream in case of exceptions. Code similar to this fixed one of my apps that wasn't releasing resources properly:
HttpURLConnection connection = (HttpURLConnection)new URL(uri).openConnection();
InputStream stream = null;
BufferedReader reader = null;
try {
stream = connection.getInputStream();
reader = new BufferedReader(new InputStreamReader(stream, Charset.forName("UTF-8")));
// do work on part of the input stream
} catch (IOException e) {
// read the error stream
InputStream es = connection.getErrorStream();
if (es != null) {
BufferedReader esReader = null;
esReader = new BufferedReader(new InputStreamReader(es, Charset.forName("UTF-8")));
while (esReader.ready() && esReader.readLine() != null) {
}
if (esReader != null)
esReader.close();
}
// do something with the IOException
} finally {
// finish reading the input stream if it was not read completely in the try block, then close
if (reader != null) {
while (reader.readLine() != null) {
}
reader.close();
}
// Not sure if this is necessary, closing the buffered reader may close the input stream?
if (stream != null) {
stream.close();
}
// disconnect
if (connection != null) {
connection.disconnect();
}
}
The buffered reader isn't strictly necessary, I chose it because my use case required reading one line at a time.
See also: http://docs.oracle.com/javase/1.5.0/docs/guide/net/http-keepalive.html