I'm using the Network Extension framework provided by Apple to build a packet sniffing/monitoring application similar to Charles Proxy and Surge 4 for iOS.
So far, I have the basic structure of the project up and running with the Main Application triggering the PacketTunnelProvider Extension where I can see packets being forwarded via the packetFlow.readPackets(completionHandler:) method. My background isn't in networking so I'm confused on the basic structure of these kinds of apps. Do they host a server on the device that act as the proxy which intercepts network requests? Could anyone provide a diagram of the general flow of the network requests? I.e. what is the relationship between the Packet Tunnel Provider, Proxy Server, Virtual Interface, and Tunnel?
If these apps do use a local on-device server, how do you configure the NEPacketTunnelNetworkSettings to allow for a connection? I have tried incorporating a local on-device server such as GCDWebServer with no luck in establishing a link between the two.
For example, if the GCDWebServer was reachable at 192.168.1.231:8080, how would I change the code below for the client to communicate with the server?
Main App:
let proxyServer = NEProxyServer(address: "192.168.1.231", port: 8080)
let proxySettings = NEProxySettings()
proxySettings.exceptionList = []
proxySettings.httpEnabled = true
proxySettings.httpServer = proxyServer
let providerProtocol = NETunnelProviderProtocol()
providerProtocol.providerBundleIdentifier = self.tunnelBundleId
providerProtocol.serverAddress = "My Server"
providerProtocol.providerConfiguration = [:]
providerProtocol.proxySettings = proxySettings
let newManager = NETunnelProviderManager()
newManager.localizedDescription = "Custom VPN"
newManager.protocolConfiguration = providerProtocol
newManager.isEnabled = true
saveLoadManager()
self.vpnManager = newManager
PacketTunnelProviderExtension:
func startTunnel(options: [String : NSObject]?, completionHandler: #escaping (Error?) -> Void) {
...
let settings = NEPacketTunnelNetworkSettings(tunnelRemoteAddress: "127.0.0.143")
settings.ipv4Settings = NEIPv4Settings(addresses: ["198.17.203.2"], subnetMasks: ["255.255.255.255"])
settings.ipv4Settings?.includedRoutes = [NEIPv4Route.default()]
settings.ipv4Settings?.excludedRoutes = []
settings.dnsSettings = NEDNSSettings(servers: ["8.8.8.8", "8.8.4.4"])
settings.dnsSettings?.matchDomains = [""]
self.setTunnelNetworkSettings(settings) { error in
if let e = error {
NSLog("Settings error %#", e.localizedDescription)
} else {
completionHandler(error)
self.readPackets()
}
}
...
}
I'm working on the iOS version of Proxyman and my experience can help you:
Do they host a server on the device that acts as the proxy which intercepts network requests?
Yes, you have to start a Listener on the Network Extension (not the main app) to act as a Proxy Server. You can write a simple Proxy Server by using Swift NIO or CocoaAsyncSocket.
To intercept the HTTPS traffic, it's a quite big challenge, but I won't mention here since it's out of the scope.
Could anyone provide a diagram of the general flow of the network requests?
As the Network Extension and the Main app are two different processes, so they couldn't communicate directly like normal apps.
Thus, the flow may look like:
The Internet -> iPhone -> Your Network Extension (VPN) -> Forward to your Local Proxy Server -> Intercept or monitor -> Save to a local database (in Shared Container Group) -> Forward again to the destination server.
From the main app, you can receive the data by reading the local database.
how do you configure the NEPacketTunnelNetworkSettings to allow for a connection?
In the Network extension, let start a Proxy Server at Host:Port, then init the NetworkSetting, like the sample:
private func initTunnelSettings(proxyHost: String, proxyPort: Int) -> NEPacketTunnelNetworkSettings {
let settings: NEPacketTunnelNetworkSettings = NEPacketTunnelNetworkSettings(tunnelRemoteAddress: "127.0.0.1")
/* proxy settings */
let proxySettings: NEProxySettings = NEProxySettings()
proxySettings.httpServer = NEProxyServer(
address: proxyHost,
port: proxyPort
)
proxySettings.httpsServer = NEProxyServer(
address: proxyHost,
port: proxyPort
)
proxySettings.autoProxyConfigurationEnabled = false
proxySettings.httpEnabled = true
proxySettings.httpsEnabled = true
proxySettings.excludeSimpleHostnames = true
proxySettings.exceptionList = [
"192.168.0.0/16",
"10.0.0.0/8",
"172.16.0.0/12",
"127.0.0.1",
"localhost",
"*.local"
]
settings.proxySettings = proxySettings
/* ipv4 settings */
let ipv4Settings: NEIPv4Settings = NEIPv4Settings(
addresses: [settings.tunnelRemoteAddress],
subnetMasks: ["255.255.255.255"]
)
ipv4Settings.includedRoutes = [NEIPv4Route.default()]
ipv4Settings.excludedRoutes = [
NEIPv4Route(destinationAddress: "192.168.0.0", subnetMask: "255.255.0.0"),
NEIPv4Route(destinationAddress: "10.0.0.0", subnetMask: "255.0.0.0"),
NEIPv4Route(destinationAddress: "172.16.0.0", subnetMask: "255.240.0.0")
]
settings.ipv4Settings = ipv4Settings
/* MTU */
settings.mtu = 1500
return settings
}
Then start a VPN,
let networkSettings = initTunnelSettings(proxyHost: ip, proxyPort: port)
// Start
setTunnelNetworkSettings(networkSettings) { // Handle success }
Then forward the package to your local proxy server:
let endpoint = NWHostEndpoint(hostname: proxyIP, port: proxyPort)
self.connection = self.createTCPConnection(to: endpoint, enableTLS: false, tlsParameters: nil, delegate: nil)
packetFlow.readPackets {[weak self] (packets, protocols) in
guard let strongSelf = self else { return }
for packet in packets {
strongSelf.connection.write(packet, completionHandler: { (error) in
})
}
// Repeat
strongSelf.readPackets()
}
From that, your local server can receive the packages then forwarding to the destination server.
Related
I did ask same question on Apple dev portal and there is other people with the same problem.
I have created simple reproducible project on GitHub: (follow steps in README)
https://github.com/ChoadPet/NWListenerTest.git
I have screen and on present ConnectionListener is initialized and on dismiss it deinitialized (called stopListening()).
First time when open the screen everything is ok:
Listener stateUpdateHandler: waiting(POSIXErrorCode: Network is down)
Listener stateUpdateHandler: ready
"📞 New connection: 10.0.1.2:50655 establish"
"Connection stateUpdateHandler: preparing"
"Connection stateUpdateHandler: ready"
but for next n-tries only this messages:
[] nw_path_evaluator_evaluate NECP_CLIENT_ACTION_ADD error [48: Address already in use]
[] nw_path_create_evaluator_for_listener nw_path_evaluator_evaluate failed
[] nw_listener_start_locked [L3] nw_path_create_evaluator_for_listener failed
Listener stateUpdateHandler: waiting(POSIXErrorCode: Network is down)
Listener stateUpdateHandler: failed(POSIXErrorCode: Address already in use)
It happens on iPhone 6 iOS 12.4.1, iPhone Xs Max iOS 13.3, iPhone 11 Pro iOS 13.5.1(also iOS 13.6)
but NOT on iPhone 7 Plus iOS 12.1.4, iPhone 11 iOS 13.5.1.
Here is my code for listening inbound connection:
final class ConnectionListener {
var dataReceivedHandler: ((Data) -> Void)?
private let port: UInt16
private let maxLength: Int
private var listener: NWListener!
private var connection: NWConnection!
init(port: UInt16, maxLength: Int) {
self.port = port
self.maxLength = maxLength
}
deinit {
print("❌ Deinitialize \(self)")
}
// MARK: Public API
func startListening() {
let parameters = NWParameters.tcp
parameters.allowLocalEndpointReuse = true
self.listener = try! NWListener(using: parameters, on: NWEndpoint.Port(integerLiteral: port))
self.listener.stateUpdateHandler = { state in print("Listener stateUpdateHandler: \(state)") }
self.listener.newConnectionHandler = { [weak self] in self?.establishNewConnection($0) }
self.listener.start(queue: .main)
}
func stopListening() {
listener.cancel()
connection?.cancel()
}
// MARK: Private API
private func establishNewConnection(_ newConnection: NWConnection) {
connection = newConnection
debugPrint("📞 New connection: \(String(describing: connection.endpoint)) establish")
connection.stateUpdateHandler = { [weak self] state in
guard let self = self else { return }
debugPrint("Connection stateUpdateHandler: \(state)")
switch state {
case .ready:
debugPrint("Connection: start receiving ✅")
self.receive(on: self.connection)
default: break
}
}
self.connection.start(queue: .main)
}
private func receive(on connection: NWConnection) {
connection.receive(minimumIncompleteLength: 1, maximumLength: maxLength, completion: { [weak self] content, context, isCompleted, error in
guard let self = self else { return }
if let frame = content {
self.dataReceivedHandler?(frame)
}
self.receive(on: connection)
})
}
}
If there is more information, which you need, let me know.
Thanks!
After your app "fails", click STOP, wait 2 minutes, and try START again. I assure you, it will work normally again. I tried with your code, which also "failed" initially for me.
What's going on: you're running into a TCP close timeout. Imagine it as a postal service where you are the receiver. If the postal service says "we have stopped deliveries to you", you can remove whole mailbox immediately and there cannot be any delivery failures. This is an equivalent of THE CLIENT closing the TCP connection. But you tell the postal service that you accept deliveries, you actually have one already received, and you decide to move out. This is an equivalent of server closing the TCP connection. The mailbox (your iOS app process) is still there, you just don't want to receive anymore. In real world, you should ensure at least some more time of having your name on the mailbox, before everyone (in reasonable frame) notices. In TCP, this is called TIME_WAIT.
TCP was not constructed primarily for the second scenario, but anyway it is trying to do its best to avoid lost packets, which just were delivered out of band (later than other packets) due to TCP routing. Particular wait length is specified in units of TCP stack parameters of the particular Operation System mark and version. So it may be shorter on some, and longer on others. It should not be more than 2 minutes.
Think over your use case. What are you trying to achieve by closing whole server, while the app is still running?
clients should close, not the server
server can cancel just single connections, not the whole listener
server can reject new connections while it's still running
for the first time I'm developing IOS application for IOT using "MQTT" of framework "Moscapsule"
but, the problem was, I have written connection code as per the frame work but it is not connecting ,here my code
let RandomId : String = "IOS_\(UIDevice.current.identifierForVendor!.uuidString)"
// set MQTT Client Configuration
let mqttConfig = MQTTConfig(clientId: RandomId, host: "http://13.232.153.138", port: 1883, keepAlive: 60)
mqttConfig.mqttAuthOpts = MQTTAuthOpts(username: "byname", password: "password")
let mqttClient = MQTT.newConnection(mqttConfig)
print(mqttClient)
print("MQTT Connection Status : \(String(describing: mqttClient.isConnected))")
print("MQTT is running : \(String(describing: mqttClient.isRunning))")
}
console message was:::
MQTT Connection Status : false
I am making one iOS app communicating with Mqtt broker, mainly to publish message. But when I try to connect with broker using CocoaMQTT library it's always giving me error in connection.
I am trying with CocoaMQTT latest version and also 1.1.3 version. But both are failing in connection and giving me error
(Error Domain=kCFStreamErrorDomainNetDB Code=8 "nodename nor servname
provided, or not known" UserInfo={NSLocalizedDescription=nodename nor
servname provided, or not known})
formatter.dateFormat = "yyyy-MM-dd HH:mm:ss.SSS"
let dateString = formatter.string(from: date)
let clientID = "smart-curtain-"+dateString
mqttClient = CocoaMQTT.init(clientID: clientID, host:
contantData.MQTT_BROKER_URL, port: UInt16(1883))
mqttClient.username = nil
mqttClient.password = nil
mqttClient.autoReconnect = true
mqttClient.allowUntrustCACertificate = true
mqttClient.keepAlive = 60
mqttClient.enableSSL = false
So its always ending up withmqttDidDisconnect delegate method. My broker url is tcp://xyz.com (xyz is just example) and port is 1883. I have tried 2-3 Mqtt toll apps from my iPhone to connect with broker detail, but no one able to connect it.
But same settings working fine in Android app. (it is using net.igenius:mqttservice:1.6.4) (this broker is no need authentication)
As shown in the CocoaMQTT doc, the host entry in the init method should be just the hostname, not a URI:
let clientID = "CocoaMQTT-" + String(NSProcessInfo().processIdentifier)
let mqtt = CocoaMQTT(clientID: clientID, host: "localhost", port: 1883)
mqtt.username = "test"
mqtt.password = "public"
mqtt.willMessage = CocoaMQTTWill(topic: "/will", message: "dieout")
mqtt.keepAlive = 60
mqtt.delegate = self
mqtt.connect()
e.g. should be xyz.com not tcp://xyz.com
var session = CocoaMQTT.init(clientID: "user1", host: "xx.xx.xxx.xx", port: 1883)
session.allowUntrustCACertificate = true
No need to send tcp://xx.xx.xxx.xx:port as in android, You can just pass xx.xx.xxx.xx by removing tcp:// and port number separately.
I would like to inspect packets with NEPacketTunnelProvider without a specified proxy. Unfortunately my readPacketObjects completion handler is never being called and I don't understand why. My regular internet connection stops and Wireshark shows nothing on the new interface obviously and I don't see any of my HTTP connection on en0 either.
Is this possible? Apple Developer Forums suggests it might not be... but Charles Proxy for iOS seems to be able to inspect everything.
My code is below:
class TunnelProvider: NEPacketTunnelProvider {
override func startTunnel(options: [String : NSObject]? = nil, completionHandler: #escaping (Error?) -> Void) {
os_log("starting tunnel")
let settings = NEPacketTunnelNetworkSettings(tunnelRemoteAddress: "127.0.0.1")
settings.tunnelOverheadBytes = 80
settings.mtu = 1200
settings.ipv4Settings = NEIPv4Settings(addresses: ["127.0.0.1", "0.0.0.0"], subnetMasks: ["0.0.0.0", "0.0.0.0"]) // all addresses
settings.ipv4Settings?.includedRoutes = [NEIPv4Route.default()] // all routes
settings.ipv4Settings?.excludedRoutes = [NEIPv4Route(destinationAddress: "127.0.0.1", subnetMask: "255.255.255.255")] // avoid local routes
settings.dnsSettings = NEDNSSettings(servers: ["1.1.1.1", "8.8.8.8", "8.8.4.4"])
settings.dnsSettings?.matchDomains = [""] // if blank don't use this DNS, use system; if "" then use this
NSLog("default/included route %#:%#", NEIPv4Route.default().destinationAddress, NEIPv4Route.default().destinationSubnetMask)
self.setTunnelNetworkSettings(settings) { error in
if let e = error {
NSLog("Settings error %#", e.localizedDescription)
completionHandler(e)
} else {
os_log("Settings set without error")
completionHandler(nil)
}
}
self.readPacketObjects()
}
private func readPacketObjects() {
NSLog("Inside readPacketObjects %#", self.packetFlow)
self.packetFlow.readPacketObjects() { packets in
NSLog("Inside readPacketObjects completionHandler %#", packets)
self.packetFlow.writePacketObjects(packets)
self.readPacketObjects()
}
}
Console output
default 10:31:03.177424 -0400 tunnel starting tunnel
default 10:31:03.177796 -0400 tunnel default/included route 0.0.0.0:0.0.0.0
default 10:31:03.178602 -0400 tunnel Inside readPacketObjects <NEPacketTunnelFlow: 0x7f9a69d135e0>
default 10:31:03.356006 -0400 tunnel Settings set without error
Edit 1
I am able to read packets after modifying settings.ipv4Settings to a local address with a more specific subnet mask. However the internet connection is still down and route get outputs bad address.
Edit 2
I was able to get DNS to work by adding the DNS servers to the excludedRoutes as seen here. Writing to the packetFlow with no modifications still doesn't enable full routing tho.
everyone.
I am trying to override dns resolver settings in my iOS app.
I used NEVPNManager to install a personal vpn and then used onDemandRules to set specific dns servers.
So far my code works for some domains.
Below is my code.
When I put "*.com" in matchDomains, it works perfectly.
But what I want to do is to redirect all dns queries to specific dns server.
I tried empty matchDomains([]) and empty string([""]).
I also tried wildcard expression like ["*"] and ["*.*].
So far I had no success.
It's been a few days and I still can't figure it out.
Can anybody tell me what I am missing here?
Thanks in advance.
let manager = NEVPNManager.sharedManager()
manager.loadFromPreferencesWithCompletionHandler { error in
if let vpnError = error {
print("vpn error in loading preferences : \(vpnError)")
return
}
if manager.protocolConfiguration == nil {
let myIPSec = NEVPNProtocolIPSec()
myIPSec.username = "username"
myIPSec.serverAddress = "server address"
myIPSec.passwordReference = self.getPersistentRef()
myIPSec.authenticationMethod = NEVPNIKEAuthenticationMethod.SharedSecret
myIPSec.sharedSecretReference = self.getPersistentRef()
myIPSec.useExtendedAuthentication = true
manager.protocolConfiguration = myIPSec
manager.localizedDescription = "myDNS"
let evaluationRule = NEEvaluateConnectionRule(matchDomains: ["*.com"], andAction: NEEvaluateConnectionRuleAction.ConnectIfNeeded)
evaluationRule.useDNSServers = ["XXX.XXX.XXX.XXX"]
let onDemandRule = NEOnDemandRuleEvaluateConnection()
onDemandRule.connectionRules = [evaluationRule]
onDemandRule.interfaceTypeMatch = NEOnDemandRuleInterfaceType.Any
manager.onDemandRules = [onDemandRule]
manager.onDemandEnabled = true
manager.enabled = true
manager.saveToPreferencesWithCompletionHandler { error in
if let vpnError = error {
print("vpn error in saving preferences : \(vpnError)")
return
}
}
}
}
I found this is buggy in even the latest iOS (10.3.1) and using NEVPNProtocolIKEv2. One moment it works, the next moment it doesn't want to start a VPN connection because it seems to misinterpret the ondemand rules and gives the error back saying the VPN profile is not enabled. I ended up with configuring the IKEv2 server (Strongswan) to push DNS settings with the "rightdns" option in /etc/ipsec.conf. This gives me the desired result of having the DNS requests redirected to a custom resolver.