As I recently found this paper describing a sniffing mechanism for iOS using Apple's NEPacketTunnelProvider Extension, I got curious and it made me want to understand it from a technical point of view. As I usually don't work at a deep network layer like that, I'm not able to comprehend it in the detail I'd like to. As Charles Proxy for iOS must do something very similar without requiring supervised devices, I assume the approach which the author of the paper presented in 2016 might be still working nowadays.
The author claimed that "Everything like IP packet parsing, building
an IP packet or parsing a DNS response had to be implemented ourselves." As I want to fully understand that, I tried to build it myself. I build a NetworkExtension and a message loop for the packetFlow of the NEPacketTunnelProvider. I was able to obtain the ip datagrams and tried to parse them. I used unsigned integers of the corresponding size for the source and target ip, the transport protocol and ip version, but I'm unsure how to handle the treat the payload. My parser uses the ptr.load(fromByteOffset: <offset>, as:<DataType>.self) where ptr is a UnsafeRawPointer to access the packet flow information. Since the data might exceed the storage of UInt64, I don't know how to access and store the payload in a proper way.
Furthermore, I figured that the source IP is always 192.168.20.1 (set as my interface's NEIPv4Settings address) and my target ip is always 192.168.2.1 (my dummy NEDNSSettings server). This leads me to my first questions: Are those DNS queries? Will the datagram packet claim any further information about the actual target? Would that mean that I have to somehow execute the request to the DNS server and reroute the packet to the target which I will obtain from that DNS query?
The next step would be to implement a TCP / UDP handling, right? My current parsing approach is able to distinguish between UDP, TCP and ICMP (even though I don't have investigated in the last one yet). Therefore, I'd iterate over the datagrams and lookup whether they require a UPD or TCP session/connection and transfer the datagram. The problem I currently see their from a conceptional point of view: How do I know which source/target port to use for TCP/UPD connections/sessions? As far as I know, this information is not part of the IP Packet itself (since it's rather some information we need on transport layer level, not on network layer level).
Additionally, I found a project called Specht on github. It uses a self-written library called NEKit which somehow also uses the NEPacketTunnelProvider approach. When I understand their approach correctly, they managed to somehow build a local proxy server by writing some observer mechanisms in order to handle the requests, but since I'm relatively new to networking and swift, I'm not sure whether I understand that completely correct or whether I just haven't find all those TCP/UDP and/or DNS logic. Is this project comparable to the approach of the paper and charles proxy?
One last question: Charles proxy is in most cases able to show the hostname of the target. I'm currently just able to see destination ip addresses (which aren't real destination ip addresses, but the address of my DNS server). How am I able to see the hostname as human readable text? Does Charles do a nslookup somehow? Does Charles obtain that information out of the datagrams?
I know it's quite ambitious of me with much missing knowledge in this topic, to build something similar for test reasons, but I'm still motivated to look deeper into that topic and also have the feeling that I have understand already some key points, but unfortunately not enough to solve the puzzle... Maybe you're able to give me some more hints to get a better understanding. If there might be even an easier way to archive a similar behavior (to see outgoing connections on hostname level), I'd be interested in these as well :-)
I've published a Beta Proxyman iOS (website) - a Network Sniffer by using NEPacketTunnelProvider and Network Extension, so I might have experienced to answer some of your questions.
IP Package, IP Diagram, DNS, How to parse it?
Luckily, there is another way to set up a NEPacketTunnelProvider to provide you with an HTTP Message, not IP Package (it's too low-level, and you have to deal with the Parser, DNS, ...)
HTTP Message is easier to parse because there are plenty of reliable libraries (e.g. http-parser from nodeJS)
How to build a Network Sniffer on iOS?
It's a complicated question to answer, I would break it into small chunks:
MitM / Proxy Server
Firstly, you need a working MitM Proxy Server, which is capable of proxying and intercepting the HTTP/HTTPS Traffic. You can implement it by using SwiftNIO or CocoaAsyncSocket.
How does it work?
In general, the data flow might look like this:
The Internet -> iPhone -> Your Network Extension (VPN) -> Forward to your Local Proxy Server (in the Network Extension) -> Mitm/Proxy Server starts intercepting or monitoring the traffic -> Save to a local database (in Shared Container Group) -> Forward again to the destination server.
From the main app, you can receive the data by reading the local database.
The reason why we need a local database is that the Network Extension and the Main app are two different processes, so they could not communicate directly like a normal app.
Show me the code?
In the Network extension, let start a Proxy Server at Host:Port, then init the NetworkSetting, like the sample:
private func initTunnelSettings(proxyHost: String, proxyPort: Int) -> NEPacketTunnelNetworkSettings {
let settings: NEPacketTunnelNetworkSettings = NEPacketTunnelNetworkSettings(tunnelRemoteAddress: "127.0.0.1")
/* proxy settings */
let proxySettings: NEProxySettings = NEProxySettings()
proxySettings.httpServer = NEProxyServer(
address: proxyHost,
port: proxyPort
)
proxySettings.httpsServer = NEProxyServer(
address: proxyHost,
port: proxyPort
)
proxySettings.autoProxyConfigurationEnabled = false
proxySettings.httpEnabled = true
proxySettings.httpsEnabled = true
proxySettings.excludeSimpleHostnames = true
proxySettings.exceptionList = [
"192.168.0.0/16",
"10.0.0.0/8",
"172.16.0.0/12",
"127.0.0.1",
"localhost",
"*.local"
]
settings.proxySettings = proxySettings
/* ipv4 settings */
let ipv4Settings: NEIPv4Settings = NEIPv4Settings(
addresses: [settings.tunnelRemoteAddress],
subnetMasks: ["255.255.255.255"]
)
ipv4Settings.includedRoutes = [NEIPv4Route.default()]
ipv4Settings.excludedRoutes = [
NEIPv4Route(destinationAddress: "192.168.0.0", subnetMask: "255.255.0.0"),
NEIPv4Route(destinationAddress: "10.0.0.0", subnetMask: "255.0.0.0"),
NEIPv4Route(destinationAddress: "172.16.0.0", subnetMask: "255.240.0.0")
]
settings.ipv4Settings = ipv4Settings
/* MTU */
settings.mtu = 1500
return settings
}
Then start a VPN,
let networkSettings = initTunnelSettings(proxyHost: ip, proxyPort: port)
// Start
setTunnelNetworkSettings(networkSettings) { // Handle success }
Then forward the package to your local proxy server:
let endpoint = NWHostEndpoint(hostname: proxyIP, port: proxyPort)
self.connection = self.createTCPConnection(to: endpoint, enableTLS: false, tlsParameters: nil, delegate: nil)
packetFlow.readPackets {[weak self] (packets, protocols) in
guard let strongSelf = self else { return }
for packet in packets {
strongSelf.connection.write(packet, completionHandler: { (error) in
})
}
// Repeat
strongSelf.readPackets()
}
From that, your local server can receive the packages then forwarding to the destination server.
Don't forget to save all traffic log to the local database, then notifying the main app to reload it.
One last question: Charles proxy is in most cases able to show the hostname of the target. I'm currently just able to see destination ip addresses (which aren't real destination ip addresses, but the address of my DNS server). How am I able to see the hostname as human readable text? Does Charles do a nslookup somehow? Does Charles obtain that information out of the datagrams?
Since we don't deal with IP Package, we don't need to implement the DNS Resolver. If you need a DNS, you can config like the following code:
let dnsSettings = NEDNSSettings(servers: ["8.8.8.8", "1.1.1.1"])
settings.dnsSettings = dnsSettings
As we receive the HTTP Message package, you can get hostname for free (From the Request's URL or Host Header)
Hope that my answer could help you.
I am able to configure the esp8266 using the tapzu wifi manager, i would also like to send the mqtt server id and password from the wifi configure page so that the esp can connect to the mqtt server. I do not want to be flashing the esp every-time, any resources would be much appreciated. Thank you.
You can add custom parameters to WiFiManager by:
// id/name, placeholder/prompt, default, length
WiFiManagerParameter custom_mqtt_server("server", "mqtt server", mqtt_server, 40);
wifiManager.addParameter(&custom_mqtt_server);
And you can retrieve the values by
mqtt_server = custom_mqtt_server.getValue();
For more details see the documentation
I need to create a network of 10 motes. I want each mote to provide 3 services with s1 = 0.25, s2 = 0.5 and s3 = 0.025. Also, I want to identify / select selfish / malicious mote(s).
Any help will be highly appreciated.
A solution to this should not be complicated to write yourself but I guess you can look at using the service registration and dissemination hack (servreg-hack) application in contiki. The operation of the application is very simple, all the application does is enable nodes to advertise services that they offer by broadcasting a SERVICE_ID (which is just an unsigned 8-bit integer). When a another node in the vicinity of the broadcasting node receives a message it stores the SERVICE_ID and the address of the node that sent the message. If the node needs a service, it can then just look up the address of a node that offers the service by calling the servreg_hack_lookup function.
The unicast-sender and unicast receiver applications in the examples section of the contiki distribution (Contiki/examples/ipv6/simple-udp-rpl) use servreg_hack.
So on both nodes you would initialise the servreg app by calling
servreg_hack_init();
then on the service provider you would register a service by calling
servreg_hack_register(SERVICE_ID, service_provider_ip_addr);
this service would then be received and registered on the service user node. The service user would can then call
service_provider_ipaddr = servreg_hack_lookup(SERVICE_ID);
to get the address of the node that provides the service identified by SERVICE_ID.
I am currently developing a phone system using Twilio and want to set the dial code for GB automatically in my code without the user having to specify +44 and removing the 0 from the number when dialing out using Twilio
In its simplest form it will be something like the below. May need to expand on this if you are likely to make international calls.
//presuming your twilio callback is providing a To number in the request
if(To.startsWith("0"))
{
To = To.Replace("0", "+44")
}
1> Just want to understand how SignalR 1.x functions in a particular scenario
Lets say we have a 10 clients connected to Hub and one of the connected clients say client-1 performs a postback so OnDisconnected is called than OnConnected is called right ?
What happens if during this phase if client-2 try's to send message to client-1 exactly between the said scenario ie (msg is sent after client-1 is disconnected and before connected again )will client-1 miss the message or there's internal mechanism which makes sure client-1 does not miss the message sent by client-2
2> Second query I have is that I'm trying to pass a querystring using following code
var chat = $.connection.myHub;
$.connection.myHub.qs = { "token": "hello" };
but not able to retrieve it on the server side from the Context object
using
Context.QueryString.AllKeys
I even tried
var chat = $.connection.myHub;
$.connection.myHub.qs = "token=hello" ;
But it does not work ie when I check the keys, token is not present in AllKeys
Will appreciate if someone just help me out.
1: If a postback occurs a client will disconnect and then connect. However, when the client performs a connect again it will have a different Connection Id than it had prior to the postback. Therefore, any message sent to the old connection id will be missed because when the users browser connects again it will be known as a different client.
2: You're trying to set the query string on the hub proxy, not the connection. What you should be doing is:
$.connection.hub.qs = { foo: "bar" };