When I use Charles Proxy for iOS and play some games, I recognize that they etablish connections with the protocol prefix "socket://" followed by an IP address (instead of a hostname, which is always present for other HTTP(s) connections). I'd assume that those are websockets.
Currently, I'm trying to implement a tool to track rudimentary network activity. To archive that, I'm using the NEKit (https://zhuhaow.me/NEKit/) in combination with the NEPacketTunnelProvider extension for iOS. Using that, I was able to set up a local HTTP Proxy server and setup the network interface to redirect every HTTP(s) request over that local proxy. Through an observer, I was able to see all the requested hostnames.
Now I found out, that some games (those which are using websockets) are not working properly with my solution. Regarding to this discussion https://news.ycombinator.com/item?id=16694670 it seems like proxying the HTTP(s) data flow doesn't enable me to handle websocket connections:
Yes, but the problem with Charles (well, iOS related at least) is that iOS websockets don't go through the HTTP Proxy configured. They're just considered a raw socket. Thus, even on desktop Charles, it's a nogo.
Due to that, some apps don't even work when my tracker is enabled, since they can't etablish a connection to their servers.
Is there the possibility to archive something similar for the websocket connections since the combination of GCDHTTPProxyServer (NEKit) and NEProxySettings (NetworkExtension) is only working for HTTP(s)? How can I track and (even better) proxy websocket connections?
Related
I'm trying to build an iOS app that lists all requests that are made from the device, like Charles Proxy does. Much like Charles, my idea has been to create a Packet Tunnel Provider and have it setup a local VPN connection without an external VPN server. The traffic would then be internally routed to the packet tunnel, without requiring an external VPN server.
Since Charles Proxy does so, I know that it is technically possible, but I can't find any information about how to setup a tunnel with an internal VPN connection, instead of using a "real" external VPN server. The only resource people refers to is the SimpleTunnel Apple sample project, which is a couple years old and written in Swift 3.
I have downloaded the SimpleTunnel sample code project, created the correct entitlements and can now run the project and create a VPN configuration that gets listed under Settings, but I just can't connect the VPN tunnel. I have tried changing the server address to 127.1.0.0 and to use IKEv2, but without success. I have also downloaded the source code at https://github.com/lxdcn/NEPacketTunnelVPNDemo but didn't get it to work either.
Does anyone know how to setup an internal VPN server with NetworkExtensions and have the tunnel use that connection?
I'm using a free version (3.11.5 as of this writing) of Charles Proxy and proxying my iPhone through it to attempt to reverse engineer some real-time features of an app I'm using.
I'm successfully able to see all http/https request in and out of the device. There are, however, web sockets (was://) that are open on the app that I cant see. It's only listing http/https requests.
Is there a setting I'm missing? Is the free version limited? Thanks.
It could be that the app you are proxying is not using the proxy settings you have defined in your iPhone’s “Settings”. If this is the case, you’d need a version of the app which adheres to the user defined proxy settings in order to see the WebSocket traffic in Charles.
I'm using Charles Proxy and Wireshark to monitor http(s) traffic from various iOS apps I'm using on my iPhone. These apps require me to set the HTTP Proxy under the iOS Wifi settings (let's call these the Proxy Settings).
My business needs to see ALL URL's that are being called from my phone. From all apps. All URL's, not some of them.
Now Charles and Wireshark both work fine and I can see a ton of traffic coming from my phone.
However, I can't help but wonder whether I might be missing some HTTP calls. Maybe calls that don't use the Cocoa Core Foundation libraries as the basis for their networking.
For instance, I could write my own HTTP library out of TCP/IP and these would bypass the Proxy Settings.
So my question is: what is the likelihood that some apps are using custom-rolled HTTP libraries and side-stepping my Proxy Settings. Or worse, they're using raw TCP/IP to communicate with a server. I know it's possible, but do any APIs work this way? Does anyone do it?
I found the answer: Use mitmproxy in transparent mode. proxy is not used. harder to setup because it needs work on the router, but it reliably captures every packet on port 80 and 443 regardless of proxy settings.
Assuming that you are able to keep your device tethered, then you may be able to use the pcap service to monitor all traffic. According to the following paper (2014) the pcap service is running on every iOS device:
"Identifying back doors, attack points, and surveillance mechanisms in iOS devices"
You should be able to connect to it via usbmuxd. I'm not sure whether there is a pre-rolled client for the pcap service. There is a list of services supported by libimobiledevice here. Pcap is not on that list.
Alternatively, you can use wireshark to capture all traffic on your wifi network.
Is there a way to intercept HTTP requests in the iPhone? e.g. something in the NSNotificationCenter I can register for so my app would get notified each time an HTTP request will go out or an HTTP response will come in?
I searched a lot but no luck,
Thank you
no you cant intercept HTTP requests with your app - you can of course act as a proxy for your OWN app's request but you cant proxy other apps' requests
From your question its not clear what you want to do, but i would suggest either implementation NSURLProtocol class or using PonyDebugger.
NSURLProtocol:
Create a subclass of NSURLProtocol class that will handle all web protocols such as HTTP, HTTPS, SSL etc. This is a abstract class that provides the basic structure for performing protocol-specific loading of URL data.
Once your created your custom URL protocol, register it in appDelegate class so your protocol will have priority over any of the built-in protocols.
[NSURLProtocol registerClass:[MyURLProtocol class]];
PonyDebugger:
if you just looking to intercept all your http request i would try this tool. PonyDebugger is a remote debugging toolset. It is a client library and gateway server combination that uses Chrome Developer Tools on your browser to debug your application's network traffic and managed object contexts.
AFNetworking:
if your using the very popular AFNetworking for your network request they automatically send notification. try to register the following two:
AFNetworkingTaskDidResumeNotification
AFNetworkingTaskDidCompleteNotification
Do you just want to view all of the HTTP/HTTPS traffic going to / from a single device that you have control of (e.g for debugging purposes)? If so, just setup a proxy and use that to monitor the requests.
If you're talking about building this into an app for general release, I'd very much hope this wasn't possible, and that if it was, I'd suspect it wouldn't make it through app store approval.
If you just want a debugging proxy:
For Windows, download Fiddler (free), and use your and follow the instructions here (install a cert on your phone and use the IP address of the machine running Fiddler as the proxy address):
http://docs.telerik.com/fiddler/configure-fiddler/tasks/ConfigureForiOS
For Mac OSX, Charles will also do the job (I've used it for this purpose and it works well):
http://www.charlesproxy.com/documentation/faqs/ssl-connections-from-within-iphone-applications/
I'm afraid the answer to this is No, but I'm hoping someone can provide a definitive answer as it is not documented in the current iOS SDK documentation.
We're seeing a case where NSURLConnection is able to connect to https://mysite.com via an HTTP proxy but, because of the way the local DNS is setup in this case, DNS lookups for mysite.com will fail. In this case, it appears that SCNetworkReachability is trying to perform a DNS lookup for mysite.com and failing. Meanwhile, NSURLConnection is able to connect.
We have incorporated the Apple Reachability sample code into our app and are calling SCNetworkReachabilityCreateWithName with mysite.com.
I can't provide a definitive answer, but I can provide more empirical evidence, and some justification, that the answer is NO. I have an app that uses SCNetworkReachabilityGetFlags to check whether a particular host is reachable (e.g.: www.mysite.com). Depending on that reachability check, it then uses [NSMutableData dataWithContentsOfURL:] to download the data.
The app has always worked fine, but recently I've been doing some coding at work where network access is via the corporate HTTP proxy. When running the app in the iOS Simulator (which uses the proxy settings configured on my Mac) the reachability check fails. At first I thought that perhaps the iOS Simulator wasn't using the Mac's proxy settings, but Mobile Safari in the simulator worked fine. So I removed the reachability check in my app and the call to [NSMutableData dataWithContentsOfURL:] worked fine. This would appear to indicate that SCNetworkReachability does not respect the proxy settings.
Having thought about it, this is probably the correct behaviour if you view SCNetworkReachability as running at the TCP/IP level, not at the HTTP level, in the same way that ping google.com on a Mac/PC behind a corporate firewall doesn't work either. The HTTP proxy (as the name implies) is for the HTTP protocol, not the whole TCP/IP stack.
Having read the answers to this question on Reachability I'm inclined to bin my reachability check altogether. Even though it's only been a problem in the simulator until now, it could be problematic in other situations (e.g. public WiFi hotspot that requires authentication).