I would like to setup a network connection (RTP or UDP) between two computers at different locations, each of which is behind a NAT modem/firewall. I do not want any modification of the firewalls.
My working assumption is that I need a bot somewhere that both computers can reach (eg a shell account on an internet server). Each computer connects out to the bot and the bot allows the two computers to update and query status and to exchange data.
This is ok as far as it goes, but it means that all data travels via the bot. Is there a way I can connect the two computers without the bot, or failing that, allow the bot to drop out of the data exchange once a connection has been setup? My feeling is that there is no way to do this, but my TCP/IP is a bit rusty...
If you assume nothing on the NAT/Firewall your are correct.
Hole-Punching for example will not work with overloaded NAT (PAT) as far as I know, because the source port is randomized by the NAT device, and it maps/match both the destination public address and the picked up source port elected.
UPnP may work, but again you need to assume it exist and enabled on the NAT device.
As I see it, you got only two options if you want to be generic:
1. Configure the NAT.
2. Use a proxy (the bot you mentioned).
Skype for example uses the second, but does it in a distributed manner by using every Skype client as a potential proxy (probably only if it detects it is not behind a NAT or not limited by it).
Related
I want to send small (a few bytes) data from one ESP8266 to another over long distances.
I believe the signal is too weak to connect one ESP8266 to another but maybe they can receive small messages with some delay?
How can I achieve that using arduino-esp8266 library (or whatever)?
If ESP8266 boards can not connect directly to each other because of the weak signal (approximately more than 50m between them) you can still connect them to each other (indirectly) if they are connected to the same WiFi network created by external router or internet. But you can connect them to each other without need to connect to any external network only if they are in range of each other.
It both of them are connected to same local WiFi network router you can communicate between them as if they are connected directly to each other. Just in this case router option "WiFi Client isolation" must be disabled in router settings page for boards to see each other(in case that router has this or similar named option).
Otherwise if ESP boards are connected to totally different networks, but both of these networks have access to internet, you can still connect them to each other but then you must use port forwarding on each router, so both of the ESP8266 boards are visible form internet. Google a bit about it, it is not so complicated. In this case you can have boards on any two locations in the world and make them communicate to each other as long as they are accessible from internet.
Maybe it is hard to test it when your code do not work out of the box, so I would recommend that you use some TCP or UDP terminal to test connectivity between two WiFi clients on different or same network. There are buch of such terminals available online.
For example for sending and receiving UDP packets same way as you would send/receive data from serial port, I use Docklight Scripting.
https://docklight.de/downloads/
I actually use it also for other Serial port communication so no big difference there. Don't let description text on their page fool you. It can also send/receive TCP/UDP data :)
Just when you create new project go to tools->project settings and type IP and port of another device you want to connect with (this can be another PC running docklight scripting or your ESP board). For example in my case I type in
Send/Receive Comm.: UDP:192.168.0.154:9761
Because that is the IP of my ESP board on local WiFi network and press "play button"or F5. Now you can send receive data to your board using UDP in same way that you communicate to basic serial port and in such way test basic connectivity. Or even use two laptops just to confirm connectivity between two WiFi clients on network.
And third hardest option must most versatile is needed if you do not have access to routers setup page (they are password protected) and cannot set port forwarding, then you must create server application on some server that is visible from internet and has public IP. So both of your ESP8266 boards are connected to this server as clients (for client port forwarding do not need to be set but in this case you must have access to internet by both clients) and server can forward their messages to each other. But this is way out of this topics scope. Maybe for start you use TCP/UDP terminal just to check connectivity between two WiFi clients and see if ESP8266 can communicate directly or not. Docklight scripting is just suggestion because it is my personal choice for TCP/UDP/RS232/RS485/MODBUS/USB HID, but you can use other software for this.
I hope this helps a bit about ways to connect two ESP8266 boards indirectly.
This may be a silly question but I was googling a lot last days and I've just found related but not a direct answer to this. In order to express it better let me mention a comparison:
If I setup an http server in a computer behind a router , I must set port forwarding and also that computer must have an static internal IP address in order to receive the requests on that port for that particular IP address. For instance, my router does not allow port forwarding so I could not set it up for the HTTP server. However , Bittorent works like a charm behind that same router without any static IP , nothing. How bittorrent can get requests and it can also seed? How other peers will connect to my computer while seeding? I did verify that when Bittorrent is running the port it uses for example 39031, is actually open ( using yougetsignal_dot_com). How the router can be made to open the port without using its admin settings ( we can also use Bitorrent in a public hot spot where is no way to mess with the router). I know the trackers may do some help, but there are a lot of information about a decentralized way with no trackers or basically that the peers are also trackers but not details on how it works.
Finally, why this "trick" (if any) cannot be used with an HTTP server and we must always depend on the router settings ?
BitTorrent does work behind NAT, to a certain extent. It works better if port forwarding is configured, though.
BitTorrent works reasonably well behind NAT for the following reasons:
BitTorrent is peer-to-peer, and either peer can initiate a connection; therefore, communication will succeed if either of two peers is not behind NAT or properly port forwarded;
some BitTorrent clients are able to automatically set up port forwardings using uPNP or PMP;
modern BitTorrent clients use µTP instead of TCP, and some clients are able to perform UDP hole punching.
I would like to understand networking services with a large user base a bit better so that I know how to approach a project I am busy with.
The following statements that I make may be incorrect but they still lead to the question that I want to ask...
Please consider Skype and TeamViewer clients. It seems that both keep persistent network connections open to their respective servers. They use these persistent connections to initiate additional connections. Some of these connections are created by means of Hole Punching if the clients are behind NATs. They are then used for direct Peer-to-Peer communications.
Now according to http://expandedramblings.com/index.php/skype-statistics/ there are 300 million users using Skype and 4.9 million daily active users. I would assume that most of that 4.9 million users will most probably have their client apps running most of the day. That is a lot of connections to the Skype servers that are open at any given time.
So to my question; Is this feasible or at least acceptable? I mean, wouldn't it be better to not have a network connection open while idle and aspecially when there are so many connections open to the servers at once? The only reason I can think is that it would be the only way to properly do Hole Punching. Techically, how is this achieved on the server side?
Is this feasible or at least acceptable?
Feasible it certainly is, you mention already two popular apps that do it, so it is very doable in practice.
As for acceptable, to start no internet authority (e.g. IETF) has ever said it is unacceptable to have long-lived connections even with low traffic.
Furthermore, the only components for which this matters are network elements that keep connection/flow state. These are for sure the endpoints and so-called middleboxes like NAT and firewalls. For the client this is only one connection, the server is usually fine tuned by the application developers (who made this choice) themselves, so for these it is acceptable. For middleboxes it's simple: they have no choice, they're designed to just work with all kind of flows, including long-lived persistent connections.
I mean, wouldn't it be better to not have a network connection open while idle and aspecially when there are so many connections open to the servers at once?
Not at all. First of all, that could be 'much' slower as you'd need to set up a full connection before each control-plane call. This is especially noticeable if your RTT is big or if the servers do some complicated connection proxying/redirection for load-balancing/localization purposes.
Next to that this would historically make incoming calls difficult for a huge amount of users. Many ISP's block/blocked unknown incoming connections from the internet by means of a firewall. Similar, if you are behind a NAT device that does not support UPnP or PCP you can't open a port to listen on for your public IP address. So you need it even aside from hole-punching.
The only reason I can think is that it would be the only way to
properly do Hole Punching. Techically, how is this achieved on the
server side?
Technically you can't do proper hole-punching as soon as the NAT devices maintain a full <src-ip,src-port,dest-ip,dest-port,protocol> (classical 5-tuple) flow match. Then the best you can do with 'hole punching' is set up a proxy between peers.
What hole-punching relies on is that the NAT flow lookup is only looking at <src-ip,src-port,protocol> upstream and <dest-ip,dest-port,protocol> downstream to do the translation. In that case both clients just set up a connection to the server, their ip and port gets translated and the server passes this to the other client. The other client can now start sending packets to that translated <ip,port> combination which should work because NAT ignores the server's ip/port. But even if the particular NAT would work like this, some security device (e.g. stateful firewall) might detect session hi-jacking and drop this anyway.
Nowadays you rather use UPnP to open up a port to listen on your public IP which is much easier if supported.
I need to connect to a VPN Server , I can`t use windows Connections , My Application should work independently !
I tested some Components using RAS Api , they works ! but by using windows connections .
how can i do that without any dependency to windows connections ?
The problem with this question
"VPN" stands for "Virtual Private Network". It's a way to make a private network available to your computer, possibly in a secure way, so your computer can use standard IP protocols as if it were physically connected to the private network.
The operating system needs to know about that network, so of course all VPN implementations use "windows connections". From a different perspective: When you're connected to a VPN you can open a TCP connection to an IP on the private network as if it were on your local network. Since it's the operating system's job to set up your TCP connection and route your TCP/IP packets, of course it needs to know about the VPN! If it doesn't, it'll simply forward all your requests for the given IP to it's default router and fail with a "no route to destination" message (or a "time out", if your router is not kind enough to tell your system it has no idea what the private IP is).
Can it be done?
From a theoretical point of view, of course, you can bypass Windows completely, but then you'll have to "roll your own" everything. You can't use the Windows IP services, you'll have to implement your own TCP. I'm sure there are about a million other little things that need re-implementing.
For a starting point I'd look at the Open VPN: it's Open Source and available for Windows. It uses the UDP protocol as the bases for the VPN implementation, unlike the Windows VPN (that one uses GRE - General Routing Encapsulation, protocol 47). Open VPN itself, of course, uses a "windows connection" to do it's job, because it aims to provide a useful service, but you can use the source code as the bases for your own implementation.
I personally wouldn't even think about doing this, I'm just showing you the way and proving it's possible.
What should be done
I assume you want some kind of secure communication channel to your own service. Look into simple secure connections, tunneling protocols and proxies.
If this needs to be done for one service on one server, I'd look into a simple SSL implementation. Even better, look into using HTTPS.
If you need to access many different services on possibly different servers on the given private network I'd look into proxies.
I'm looking for a tool under windows or mac that allows me to monitor (possibly in a simple way) the traffic going in and out of a computer of my network.
Long story short the residence where I live allows themselves to monitor the internet connection (and doesn't allow us to switch to another provider).
This annoys me on a personal level (I don't like the possibility of people checkin what I do without my knowledge as a general rule regardless of what I do) but also on a professional level (I sometimes work form home).
I'm using/trying out vpn providers (JAP, VyperVPN...) to avoid all this. it works fine with the http connections (if I run iptraces I end up in germany or US or UK ...) but I'm not sure for other applications such as online games, instant messaging softwares that use different ports.
So my question is how can I make sure that my internet traffic is using my vpn connection or not ?
Wireshark would do that for you on Windows and linux (Not sure about Mac). It uses WinPCap library and wraps in a nice UI for you to monitor the packets that you are interested. It allows you to listen to specific or all interfaces , so you can make sure your packets are going via the right interface
if you don't want them monitoring your internet usage, a vpn is a good solution, a vpn will encrypt all of your net traffic between your computer and the vpn gateway -- essentially you'd be surfing the web via a proxy and your landlords wouldnt be able to determine what you are doing.
assuming you are using a real vpn, and not just a browser based proxy solution, then the vpn should encrypt and tunnel all of your network traffic, this includes anything coming out of any port on your computer, not just http traffic.
when you install a vpn on your computer, the vpn creates a fake network device, and all of the vpn traffic gets tunneled to the vpn gateway. you can verify this by looking at your computer's routing tables. there are some vpns which allow for split traffic (split tunneling), e.g. traffic to certain domains gets tunneled through the vpn and others goes in the clear, but this is the rarity, most vpns will tunnel all of your traffic, which seems to be what you are looking for.
just make sure that your vpn uses an encryption protocol, there are some that don't -- this would defeat the whole purpose of your vpn.