I've set up a networking project to communicate over LAN or WAN via hole punching. I'm using GCSAsyncUdpSocket for the clients. I have a rendezvous server which is port forwarded to be accessible from all incoming connections. My setup works like this:
Client A connects to the server.
Client B connects to the server.
Server observes the IP address and Port used by both clients.
Server tells Client A to communicate with public IP Address B and public Port B.
Server tells Client B to communicate with public IP Address A and public Port A.
Client A sends periodic data to Client B.
Client B sends periodic data to Client A.
This is where it goes a bit weird. While the clients are on the same network, which we've tried on both our networks, nothing works. But, if the devices are connected on different networks (both with different providers and are not linked directly) then one of the devices receive data, while the other does not.
This also fails to work on 3G completely.
What I can't understand is how it works 1-way with 1 device on each network, but not at all with both devices on the same network?
I've since found that one of our routers is detecting the messages being sent from one client to another contain different target IP addresses of that to the Server, and then our router decides that these messages should then be sent via a different external port. This is where the problem was coming from. Using a bit of port prediction, this hurdle can be overcome most of the time.
For those who are interested, most of the time if a connection cannot be found at the presumed port, it will most likely be at the very next port (ie. Port# + 1). If not, it gets a little bit complex from there to work out.
Related
Short version: How would you recommend going about connecting a client to a server that are on the same local network, without manually entering the ip, when broadcast is disabled?
Further details: I am working on an educational multiplayer game for children. Many schools appear to be blocking broadcasting for security reasons. The children will be rather young, so it could be difficult for them and error-prone to have to enter the IP manually. They will all be in the same room and will all see the server screen. The game is made in Unity (C#).
Potential solutions: Here's what I thought about:
Connecting both the local server and clients to an external server, communicating the local server ip through the clients through the external server, then connecting directly and disconnect from the server. Not ideal because of the extra hosting costs.
Send a regular UDP message periodically to all ips on the subnet? This will probably be picked up by any decent firewall and blocked though, right?
Putting a QR code on the server that kids would take a picture of with the client app and have it connect that way? May be more of a hassle.
Having the server play random tones corresponding to numbers that the client is listening for? (Speakers may not always work though)
Sounds like the first one is the most sane and easy solution. Do you have any other ideas on what someone in this situation could try?
Is UDP multicast possible?
If yes then a common solution is that all participants join the same multicast group and the server listens on a well known port. If a client wants to know the address of the server it sends a packet to the multicast group, which is received by the client and answered with another packet, which then can be used by the client to determine the servers address.
In addition to that servers can also announce their presence in regular intervals by sending a suitable message to the multicast group.
What I can think of is an ad-hoc communication protocol between all the devices. Say you have 1 server and 10 clients. All the devices should run a service(say server-discovery) that binds to a fixed port say 9999. Now any time the client wants to connect to the server and doesn't know the IP, it starts a scan. Loops through different IPs and tries to connect to 9999. If it manages to hit, it asks for the server IP. In case it manages to hit the server it will get the IP since the server knows it's own IP and the client will maintain the server IP in a cache. If the client hits another client. It can ask for the server IP. It the other client knows the IP it will share the info else decline.
I agree there is a lot of overhead, but I think this will be robust unlike sound and would reduce cost of printing QRs everytime.
on the local network the traffic is direct from host to host.
I don't understand which devices is blocking local broadcasts.
if there aren't too many peers on the LAN ( less then 100 ), I think udp broadcasts work fine and you dont pollute the network.
to have an idea of your "pond", I suggest you to sniff your local traffic.
there are many broadcasts : arp, windows, ipp, dropbox...
I have 2 nodeMCU modules, which I want to connect to an MQTT broker and send some data every 5 seconds.
The topology I am trying to achieve is sth like [router]<==[nodeMCU#1]<==[nodeMCU#2]
It looks like [nodeMCU#1] is a wifi extender, but at work we are planning on using multiple nodeMCU's to use in a mesh-like way for an IoT application.
On both of them I flashed the latest (float) release that I downloaded from here https://github.com/nodemcu/nodemcu-firmware/releases/tag/0.9.6-dev_20150704 using nodeMCU-flasher.
For this project wrote 2 lua scripts and I have uploaded them to the modules using ESPlorer.
For the first module it connects in STATIONAP mode to my work wifi, and creates its own network with SSID nodeMCUwifi, with a basic password of 10 characters "1234567890". After it connects the script sends random values to an MQTT topic.
For the second module connects to nodeMCUwifi correctly, it is assigned an IP address, but it cannot connect to the MQTT broker.
When I try to connect to nodeMCUwifi using my smartphone (just to test the connection, I have no intention of using this system for heavy internet load, only MQTT messages) I get a message "authentication error occured" even though I have typed the password correctly, or (in rarer cases)it connects but disconnects immediately.
I would appreciate any ideas to resolve this issue. Thank you.
EDIT: At the AP configuration I added the auth parameter set to 3, I am pasting a part of my code below.
cfg.ssid="ESP8266_"..node.chipid();
cfg.pwd="1234567890"
cfg.auth=3
wifi.ap.config(cfg);
Now my laptop and my smartphone connect to the wifi created by the module, but still have no connectivity to the internet. I can ping the module, but I can't ping 8.8.8.8 or the MQTT broker IP or anything else.
As I understand, [nodeMCU#1] creates a wifi network named nodeMCUwifi. Then the others are trying to connect to nodeMCUwifi. As You stated:
Now my laptop and my smartphone connect to the wifi created by the module, but still have no connectivity to the internet. I can ping the module, but I can't ping 8.8.8.8 or the MQTT broker IP or anything else.
I assume your laptop and smartphone connects to the nodeMCUwifi network. Now you want this [nodeMCU#1] act like layer 2 switch or something like router(since SOFTAP creates its own ip network).
With all this information on hand, I can say that the purpose you are trying to achieve is not possible with esp. Since esp has not enough resources to act like switch or similar. Also (currently) there is no software implemented in esp to achieve that.
However; if only you want the system to transfer certain type of traffic (such as MQTT) over a certain protocol (such as UDP) it is possible to relay the messages like a mesh structure but this will work one-way only. In this configuration the nodes will act more like repeater but on upper layers of OSI. For this idea i cannot provide sample code but I will provide pseudo-code below:
Configuration
> Each node must be configured to build a chain structure. That means each SOFTAP node should have different SSID or TCP settings. For example:
router(192.168.0.0)<----[NODE#1(SOFTAP0)]<----[NODE#2(SOFTAP1)]<---...
...
...
...<----[NODE#n(STATION)]
> Each node must have a TARGET IP address according to setup. This means every node will send the MQTT message to that address.
> If a node is configured as a SOFTAP, this means this system is a TARGET and must listen to a specific port.
> If a SOFTAP node receives some data on the port that is listening to, it should send the data immediately (or buffering) to its TARGET node.
> As a result of this chaining operation the message, no matter what is the source, will reach to the end of the chain.
CODE : SOFTAP
...soft ap config...
listento.port: udp.965
on("receive"):
create.conn:TARGET
send(DATA_RECEIVED)
close.conn:TARGET
CODE : STATION
...station config...
...Do some logic...
data=gather.data()
create.conn:TARGET
send(data)
close.conn:TARGET
This may be a silly question but I was googling a lot last days and I've just found related but not a direct answer to this. In order to express it better let me mention a comparison:
If I setup an http server in a computer behind a router , I must set port forwarding and also that computer must have an static internal IP address in order to receive the requests on that port for that particular IP address. For instance, my router does not allow port forwarding so I could not set it up for the HTTP server. However , Bittorent works like a charm behind that same router without any static IP , nothing. How bittorrent can get requests and it can also seed? How other peers will connect to my computer while seeding? I did verify that when Bittorrent is running the port it uses for example 39031, is actually open ( using yougetsignal_dot_com). How the router can be made to open the port without using its admin settings ( we can also use Bitorrent in a public hot spot where is no way to mess with the router). I know the trackers may do some help, but there are a lot of information about a decentralized way with no trackers or basically that the peers are also trackers but not details on how it works.
Finally, why this "trick" (if any) cannot be used with an HTTP server and we must always depend on the router settings ?
BitTorrent does work behind NAT, to a certain extent. It works better if port forwarding is configured, though.
BitTorrent works reasonably well behind NAT for the following reasons:
BitTorrent is peer-to-peer, and either peer can initiate a connection; therefore, communication will succeed if either of two peers is not behind NAT or properly port forwarded;
some BitTorrent clients are able to automatically set up port forwardings using uPNP or PMP;
modern BitTorrent clients use µTP instead of TCP, and some clients are able to perform UDP hole punching.
When creating a peer to peer audio connection using webRTC, the STUN server we use will return the public IP if a user is behind a router. Now in the ICE objects, I can see that the rport is always something between 50000 and up.
Is there a way to use a specific port so that the user does not have to open all those ports?
Is there a way to use a specific port so that the user does not have to open all those ports?
I think you have a misunderstanding. The whole point of STUN and ICE (including its WebRTC derivative) exists to avoid anyone having to open a port on their NAT. Instead, STUN and ICE dynamically open the port.
Here's how it works (in a really brief description).
Client opens a socket on a random port (e.g. 50001)
Contacts STUN server using that socket to discover the external IP:port mapping for this socket. (e.g. 192.168.1.2:50001 maps to 1.2.3.4:50001). Ports don't necessarily have to match between internal and external addresses, but they usually do, so I'll keep with that for this example.
Through an external mechanism (SIP, XMPP, Jingle, cups with strings), the candidate address list of both nodes are exchanged. This includes all known internal and external addresses collected (e.g. 192.168.1.2:50001 and 1.2.3.4:50001).
Using the same socket opened in step 1, both sides send (STUN) messages (UDP packets) directly between each other. The first pair of messages may be blocked by the router/firewall. But because one side initiated an outbound packet to the remote address, subsequent packets from that address are allowed back in. This is called the "hole punching step". Hence, the port is dynamically open without the router needing any specific configuration.
Hope this helps.
You can't programatically unless you are using webrtc API in your own application. The browser will pick specific local ports from a range locally; and then it will inform you about them in the SDP and ICE candidate information.
STUN server only helps discover whether a client is behind a NAT/firewall; and then ICE uses this information in establishing peer-to-peer connection.
I have heard somewhere there might be a way to control that port range via Chrome policy templates(used by enterprises to restrict Chrome settings) - http://www.chromium.org/administrators/policy-templates. It might worth looking into...
I would like to setup a network connection (RTP or UDP) between two computers at different locations, each of which is behind a NAT modem/firewall. I do not want any modification of the firewalls.
My working assumption is that I need a bot somewhere that both computers can reach (eg a shell account on an internet server). Each computer connects out to the bot and the bot allows the two computers to update and query status and to exchange data.
This is ok as far as it goes, but it means that all data travels via the bot. Is there a way I can connect the two computers without the bot, or failing that, allow the bot to drop out of the data exchange once a connection has been setup? My feeling is that there is no way to do this, but my TCP/IP is a bit rusty...
If you assume nothing on the NAT/Firewall your are correct.
Hole-Punching for example will not work with overloaded NAT (PAT) as far as I know, because the source port is randomized by the NAT device, and it maps/match both the destination public address and the picked up source port elected.
UPnP may work, but again you need to assume it exist and enabled on the NAT device.
As I see it, you got only two options if you want to be generic:
1. Configure the NAT.
2. Use a proxy (the bot you mentioned).
Skype for example uses the second, but does it in a distributed manner by using every Skype client as a potential proxy (probably only if it detects it is not behind a NAT or not limited by it).