I'm using Charles Proxy and Wireshark to monitor http(s) traffic from various iOS apps I'm using on my iPhone. These apps require me to set the HTTP Proxy under the iOS Wifi settings (let's call these the Proxy Settings).
My business needs to see ALL URL's that are being called from my phone. From all apps. All URL's, not some of them.
Now Charles and Wireshark both work fine and I can see a ton of traffic coming from my phone.
However, I can't help but wonder whether I might be missing some HTTP calls. Maybe calls that don't use the Cocoa Core Foundation libraries as the basis for their networking.
For instance, I could write my own HTTP library out of TCP/IP and these would bypass the Proxy Settings.
So my question is: what is the likelihood that some apps are using custom-rolled HTTP libraries and side-stepping my Proxy Settings. Or worse, they're using raw TCP/IP to communicate with a server. I know it's possible, but do any APIs work this way? Does anyone do it?
I found the answer: Use mitmproxy in transparent mode. proxy is not used. harder to setup because it needs work on the router, but it reliably captures every packet on port 80 and 443 regardless of proxy settings.
Assuming that you are able to keep your device tethered, then you may be able to use the pcap service to monitor all traffic. According to the following paper (2014) the pcap service is running on every iOS device:
"Identifying back doors, attack points, and surveillance mechanisms in iOS devices"
You should be able to connect to it via usbmuxd. I'm not sure whether there is a pre-rolled client for the pcap service. There is a list of services supported by libimobiledevice here. Pcap is not on that list.
Alternatively, you can use wireshark to capture all traffic on your wifi network.
Related
When I use Charles Proxy for iOS and play some games, I recognize that they etablish connections with the protocol prefix "socket://" followed by an IP address (instead of a hostname, which is always present for other HTTP(s) connections). I'd assume that those are websockets.
Currently, I'm trying to implement a tool to track rudimentary network activity. To archive that, I'm using the NEKit (https://zhuhaow.me/NEKit/) in combination with the NEPacketTunnelProvider extension for iOS. Using that, I was able to set up a local HTTP Proxy server and setup the network interface to redirect every HTTP(s) request over that local proxy. Through an observer, I was able to see all the requested hostnames.
Now I found out, that some games (those which are using websockets) are not working properly with my solution. Regarding to this discussion https://news.ycombinator.com/item?id=16694670 it seems like proxying the HTTP(s) data flow doesn't enable me to handle websocket connections:
Yes, but the problem with Charles (well, iOS related at least) is that iOS websockets don't go through the HTTP Proxy configured. They're just considered a raw socket. Thus, even on desktop Charles, it's a nogo.
Due to that, some apps don't even work when my tracker is enabled, since they can't etablish a connection to their servers.
Is there the possibility to archive something similar for the websocket connections since the combination of GCDHTTPProxyServer (NEKit) and NEProxySettings (NetworkExtension) is only working for HTTP(s)? How can I track and (even better) proxy websocket connections?
I have two entities which I would like to be able to start a communication:
Hardware Device Using Atmel ATSAMS70 and WINC1500 Wifi Module
HTML 5 App
The idea to have a HTML 5 app is to be able to communicate easily with most of the commercial devices like: Windows computers, Android Phones, Mac OSX computer and iOS devices. Apart from that, I would avoid working with native code at all.
Currently, they can talk with each other using WebSockets but somehow the IP address of the custom board has to be known from HTML 5 to initiate the communication.
In order to do this, I can think of 3 options:
Using WebRTC I can get the local IP address of the browser and then I could do a scan of the local devices considering a 255.255.255.0 network mask.
Have an external server that the hardware device can send its local address which will be later retrieved by HTML 5.
Using Bonjour or some sort of device discovery service between the board and HTML5.
I could not find a way to achieve #3 but #2 seems feasible to me. #1 is what I am doing now, but WebRTC is currently not supported on iOS.
So, is there any other better possibility to achieve this communication?
You don't mention how the WINC1500 unit is being connected to the network but presumably this is in STA mode (acting as a wifi station rather than becoming a software access point or part of an ad-hoc network) and is being provided with its IP details through an existing access point?
Ordinarily I'd suggest that once connected, the device ought to start indicating its availability on the network via a regular UDP broadcast on a specific port but my (admittedly limited) understanding of WebSockets it is that it creates TCP connections. The only implementation of Bonjour that I've seen uses UDP messaging too, that may be why you've had trouble with your third approach.
Your second approach seems more likely to work well. A server at a known (or discoverable) IP on the local network which allows the Atmel device to register itself (and its IP address) and also allows other applications (your HTML 5 WebSockets applications) to request that connection information to allow them to create the WebSocket TCP connections they need.
I suppose that doesn't really answer your question as to "Is there a better way?", other than to say "Not that I can think of, your second approach looks good to me...". Sorry! Sounds like a very interesting project, overall,
I'd like to see the request / responses that an iphone app makes.
I mostly work on web apps, and I can use firebug / fiddler to see them. But how can I see incoming/outgoing traffic of an iOS app, if im running it on my wireless?
The Charles Web Proxy (and I believe Fiddler as well) allow connections from external hosts, when configured properly. In Charles, you will need to make sure your iPhone is added to the Access Control List in Proxy -> Access Control Settings.
After that, you can simply set your iPhone's proxy to your computer's Charles or Fiddler instance. In my case, my local desktop is at 192.168.10.1, thus my iPhone's proxy is set to:
192.168.10.1:8888
One problem, however, may be if you want to decrypt SSL traffic. It may be difficult to get the iPhone to add Fiddler or Charles' certificate to the keychain.
You could share your mac wireless to the iphone
And then use some tool, i personally use http://www.charlesproxy.com/ for these kind of issues
You could pick a tool from
https://superuser.com/questions/99870/mac-wireshark-alternatives
I am implementing a client (IOS device) and Server. I wish to keep the payload of request/responses to a minimum so plan to implement a custom TCP based service. My only concern is if I can rely on the port I use being always open through 3G etc. There must be other apps that use non standard ports and TCPon IOS?
If you can, you might use Port 80/443(SSL) for your custom TCP server. You only have to ensure that your have a free IP on your Server to bind your custom server-application to port 80.
If it's not HTTP(D) behind port 80/443(SSL), thats not a problem!
When using other port you may run into problems with blocked traffic in firewall rules.
But that's the standard case. Like a VOIP/SIP iOS App needs the SIP Port 5090. So if you are try to make a call within your University WiFi where only some Ports except 5090 are supported, the app won't run/connect.
So. You should make it like Viber or WhatsApp Messenger does. Use Port 80/443(SSL) and try to get the max possible connects trough possible firewalls.
I have got a couple of phones and another couple of PC's connected to a Wifi access point and need to send and receive messages between either of these, I mean anyone can send a message to anyone and receive a message from anyone.
I am willing to write apps on the phones(Symbian OS, S60 platform) or PC(Windows), but what I can't understand is how do I set up a client or server, since any one of these devices could be a client or server.
If I use sockets do I have to script for ServerSockets and also Sockets on each of these devices? Can I use the HTTP protocol?
Alternatively any standard protocol that I could use to implement this?
You would broadcast UDP packets which would arrive at every device on the Wifi network. You would have to invent your own protocol to decide on the identity of each device, since you wouldn't be able to easily infer the IP addresses of your network devices. Without writing an election algorithm you would find it difficult to use a client/server architecture, so just use point-to-point (P2P).
Google for UDP broadcasts and read the relevant RFCs at ietf.org.
It seems like you're looking for pretty typical peer-to-peer communication over IP. I suppose other requirements will dictate which transport you use (HTTP, raw sockets, etc), but yes: Each node will be both a client and a server. You could possibly use MDNS (http://www.multicastdns.org/) to help the nodes find eachother in an ad-hoc manner.