I'm trying to get my head around traceroutes and why data travels between routers the way it does. From the research I've done I have found what seems to be a typical route (below). Would this be considered a typical route for data to travel?
Your LAN > Tier 2 ISP (your ISP) > Tier 1 ISP (your ISP’s ISP) > Tier 2 ISP (the ISP providing internet to the domain you are requesting data from) > Webserver
Yes, the route you describe is fairly typical.
First you get into the network of your ISP, then you move "up the food chain" to a bigger network, one that your ISP pays to. Then possibly one or two other large networks, and finally back "down" to the local ISP of your destination.
There are exceptions of course (like if your destination is in a small network that's directly connected to your ISP), but in general that's pretty much it.
Related
I've searched around a while and all of the IP --> Hostname things actually only end up giving an ISP. Is there something that goes beyond that? I'm only finding pay services that go further and not something that I can just tap a nice API and programmatically do it.
http://ipinfo.io/ just ends up showing ISP for many of what I've sampled. I saw that guy posts here fairly often.
whoisvisiting.com runs about $99/mnth for what my company site does but in that range I'd rather code something. I'm using the free trial right now and have the IP's logging to analytics so I'm looking at what it returns, what IIS returns as the hostname and what a couple sources like ipinfo.io show and whoisvisiting somehow actually shows what I'm looking for.
There's no way to do so. There's no central registry for which company has which address ranges. In fact, most companies will just be identifiable via their ISP.
Your paid services might be scams, by the way, or just work on very few select companies and universities that actually act as autonomous entities in the IP sense.
It is unlikely to differentiate between ISP or company IP address. Some geolocation providers will use range size or level of allocation to name ISP or business. However, this approach is not always accurate.
I would like to understand networking services with a large user base a bit better so that I know how to approach a project I am busy with.
The following statements that I make may be incorrect but they still lead to the question that I want to ask...
Please consider Skype and TeamViewer clients. It seems that both keep persistent network connections open to their respective servers. They use these persistent connections to initiate additional connections. Some of these connections are created by means of Hole Punching if the clients are behind NATs. They are then used for direct Peer-to-Peer communications.
Now according to http://expandedramblings.com/index.php/skype-statistics/ there are 300 million users using Skype and 4.9 million daily active users. I would assume that most of that 4.9 million users will most probably have their client apps running most of the day. That is a lot of connections to the Skype servers that are open at any given time.
So to my question; Is this feasible or at least acceptable? I mean, wouldn't it be better to not have a network connection open while idle and aspecially when there are so many connections open to the servers at once? The only reason I can think is that it would be the only way to properly do Hole Punching. Techically, how is this achieved on the server side?
Is this feasible or at least acceptable?
Feasible it certainly is, you mention already two popular apps that do it, so it is very doable in practice.
As for acceptable, to start no internet authority (e.g. IETF) has ever said it is unacceptable to have long-lived connections even with low traffic.
Furthermore, the only components for which this matters are network elements that keep connection/flow state. These are for sure the endpoints and so-called middleboxes like NAT and firewalls. For the client this is only one connection, the server is usually fine tuned by the application developers (who made this choice) themselves, so for these it is acceptable. For middleboxes it's simple: they have no choice, they're designed to just work with all kind of flows, including long-lived persistent connections.
I mean, wouldn't it be better to not have a network connection open while idle and aspecially when there are so many connections open to the servers at once?
Not at all. First of all, that could be 'much' slower as you'd need to set up a full connection before each control-plane call. This is especially noticeable if your RTT is big or if the servers do some complicated connection proxying/redirection for load-balancing/localization purposes.
Next to that this would historically make incoming calls difficult for a huge amount of users. Many ISP's block/blocked unknown incoming connections from the internet by means of a firewall. Similar, if you are behind a NAT device that does not support UPnP or PCP you can't open a port to listen on for your public IP address. So you need it even aside from hole-punching.
The only reason I can think is that it would be the only way to
properly do Hole Punching. Techically, how is this achieved on the
server side?
Technically you can't do proper hole-punching as soon as the NAT devices maintain a full <src-ip,src-port,dest-ip,dest-port,protocol> (classical 5-tuple) flow match. Then the best you can do with 'hole punching' is set up a proxy between peers.
What hole-punching relies on is that the NAT flow lookup is only looking at <src-ip,src-port,protocol> upstream and <dest-ip,dest-port,protocol> downstream to do the translation. In that case both clients just set up a connection to the server, their ip and port gets translated and the server passes this to the other client. The other client can now start sending packets to that translated <ip,port> combination which should work because NAT ignores the server's ip/port. But even if the particular NAT would work like this, some security device (e.g. stateful firewall) might detect session hi-jacking and drop this anyway.
Nowadays you rather use UPnP to open up a port to listen on your public IP which is much easier if supported.
Persistent connection to a mobile device is difficult. Signal conditions can change rapidly, and connectivity types can also change. For instance, I may want to stream audio to my phone as I leave my apartment (WiFi), take a bus (WiMax/LTE), transfer to the subway (intermittent CDMA, sometimes roaming on another carrier), and walk to work (WiMax/LTE and back to WiFi). On this 15-minute trip alone I use at least 4 different IP addresses/networks, and experience all sorts of connectivity issues along the way. However, there is rarely a total loss of connectivity to the Internet, and the times that the signal condition makes connectivity problematic only happen for small periods of time.
I'm looking for a protocol that allows roaming from network to network and is very tolerant of harsh network conditions, while maintaining virtual end-to-end connectivity. This protocol would enable connections between a (usually) mobile device and some sort of proxy server which would relay regular TCP/UDP connections on behalf of the mobile device, over this tolerant protocol.
This protocol would sit around layer 3, and maybe even enable creation of virtual network interfaces that are tunneled through it. Perhaps there is a VPN or SOCKS proxy solution that already meets these needs.
Does such a protocol already exist?
If not, I'm probably going to come up with one, but would rather piggy-back off of existing efforts first.
There are many efforts within the internetworking community to address precisely these "network mobility" concerns.
In particular, Mobile IP (and its IPv6 big sister, Proxy Mobile IPv6) is a broad term for efforts to make IP addresses themselves portable across networks, however I doubt these technologies have reached sufficient maturation/deployment for production use today.
To undertake such mobility without support from the network requires a means of the host announcing to you its new address in an authenticated manner; this is what the Host Identity Protocol is designed for, but it is still at the "experimental" stage of the RFC process. From the abstract of RFC 5201:
HIP allows consenting hosts to securely establish and maintain shared
IP-layer state, allowing separation of the identifier and locator
roles of IP addresses, thereby enabling continuity of communications
across IP address changes.
There are several open-source implementations that are known to interoperate. Without claiming that this is a complete list, nor vouching for any of them (they're just a few picked off a Google search for "Host Identity Protocol implementations"), there is:
OpenHIP for multiple operating systems;
HIPL for Linux;
cutehip for Java;
HIP for inter.net for *BSD/Linux.
Hi
let me make my question clear. Two people using my app are connected to the internet. Both have each other's IP and they want to chat (like Y!messanger) with each other.
I think I need to use Indy components; right? Which component should I use?
Thanks in advance
Have you looked at any of the demos on Indy's website yet?
In general, you are looking to create a "Client/Server" type application. A quick Google search for "indy client server example" pulls up lots of results, including this one: http://www.devarticles.com/c/a/Delphi-Kylix/A-Real-World-Client-Server-Application-in-Delphi/
In reality, this gets a lot more complicated when you have firewalls and NATs with private IP addresses. You will have to consider how your application will either get around or through these types of technologies.
Similar to what Scott said, I think that your biggest problem is getting them talking to each other. My computers at home go through a router, which blocks all incoming connection requests (i.e. requests to start a conversation between two computers) from the Internet. My computers can send connection requests OUT, and start a conversation that way, but unless you modify the router (port forwarding) my computers can not receive connection requests.
You need a server somewhere to which both people will connect, that can then relay messages back and forth. To get really tricky, once the connection is made to the server the two computers can then be put into direct contact, but that involves UDP packets and some clever magic.
You don't have to use Indy components, you just need anything that will handle communications over the network. Any HTTP or sockets network stack will do. Indy is the defacto standard for Delphi Win32.
To do network communications, you will need to create a listener object or service on machine A and a sender object on machine B to send a network message from A to B. To send a message from B to A, you will need a reverse path as well - 4 objects total to perform bidirectional comms. Some object wrappers hide this detail internally. I don't recall offhand whether Indy hides this or not.
It would probably be easiest if you use a common TCP/IP protocol for your machine to machine communications, such as HTTP. This will make it easier to get your connections through firewalls and proxies that frequently exist between arbitrary users. To avoid conflicting with any HTTP web services that might be running on either machine, you should use a custom port number with the IP address: 192.168.1.10:12345, not the standard HTTP web server port 80. This is what most of the IM clients do.
I need to use one logical PGM based multicast address in application while enable such application "seamlessly" running across several different geo-locations (i.e. think US/Europe/Australia).
Application is quite throughput (several million biz. messages a day) and latency demanding whith a lot of small but very frequently send messages. Classical Atom pub will not work here due some external limits of latencies.
I have come up with several options to connect those datacenters but can’t find the best one.
Options which I have considered are:
1) Forward multicast messages via VPN’s (can VPN handle such big load).
2) Translate all multicast messages to “wrapper messages” and forward them via AMQP.
3) Write specialized in-house gate which tunnels multicast messages via TCP to other two locations.
4) Any other solution
I would prefer option 1 as it does not need additional code writes from devs. but I’m afraid it will not be reliable connection.
Are there any rules to apply for such connectivity?
What the best network configuration with regard to the geographical configuration is for above constrains.
Just wanted to say hello :)
As for the topic, we have not much experience with multicasting over WAN, however, my feeling is that PGM + WAN + high volume of data would lead to retransmission storms. VPN won't make this problem disappear as all the Australian receivers would, when confronted with missing packets, send NACKS to Europe etc.
PGM specification does allow for tree structure of nodes for message delivery, so in theory you could place a single node on the receiving side that would in its turn re-multicast the data locally. However, I am not sure whether this kind of functionality is available with MS implementation of PGM. Optionally, you can place a Cisco router with PGM support on the receiving side that would handle this for you.
In any case, my preference would be to convert the data to TCP stream, pass it over the WAN and then convert it back to PGM on the other side. Some code has to be written, but no nasty surprises are to be expected.
Martin S.
at CohesiveFT we ran into a very similar problem when we designed our "VPN-Cubed" product for connecting multiple clouds up to servers behind our own firewall, in one VPN. We wanted to be able to run apps that talked to each other using multicast, but for example Amazon EC2 does not support multicast for reasons that should be fairly obvious if you consider the potential for network storms across a whole data center. We also wanted to route traffic across a wide area federation of nodes using the internet.
Without going into too much detail, the solution involved combining tunneling with standard routing protocols like BGP, and open technologies for VPNs. We used RabbitMQ AMQP to deliver messages in a pubsub style without needing physical multicast. This means you can fake multicast over wide area subnets, even across domains and firewalls, provided you are in the VPN-Cubed safe harbour. It works because it is a 'network overlay' as described in technical note here: http://blog.elasticserver.com/2008/12/vpn-cubed-technical-overview.html
I don't intend to actually offer you a specific solution, but I do hope this answer gives you confidence to try some of these approaches.
Cheers, alexis