I had a question about applications running within Docker containers and UUID generation.
Here’s our scenario:
Currently our applications are using an event driven framework.
For the events we generate the UUID’s based on mac address, pid,
time-stamp and counter.
For running containers on a distributed system like CoreOS (while a very very very low chance), there is no guarantee that all those parameters used to generate a UUID would be unique for each container as one container on one server in the cluster could generate a UUID using the same mac, pid, time-stamp and counter as another container on the cluster.
In essence if these two UUID’s were both to generate an event and send it to our messaging bus, then obviously there would be a conflict.
In our analysis, this scenario seems to boil down to the uniqueness of mac addresses on each Docker container.
So to be frank:
How unique are the mac addresses within containers?
How are mac addresses generated if they are not manually set?
From my reading of generateMacAddr function (edit: answer concerned 1.3.0-dev, but is still correct for 17.05), MAC addresses generated by docker are essentially the IPv4 address of the container's interface on the docker0 bridge: they are guaranteed to be consistent with the IP address.
The docker0 bridge's subnet you have to operate in, usually 255.255.0.0 as per this example of 172.17.42.1/16, has 65,534 routable addresses. This does reduce entropy for UUID generation, but MAC address collision isn't possible as IPs must be unique, and the scenario of identical MAC, PID, time and counter in two containers on the same docker server/CoreOS host should not be a possibility.
However two CoreOS hosts (each running one docker server) could potentially choose the same random subnet, resulting in the possibility of duplicated MACs for containers on different hosts. You could evade this by setting a fixed CIDR for the docker server on each host:
--fixed-cidr=CIDR — restrict the IP range from the docker0 subnet, using the standard CIDR notation like 172.167.1.0/28. This range must be and IPv4 range for fixed IPs (ex: 10.20.0.0/16) and must be a subset of the bridge IP range (docker0 or set using --bridge). For example with --fixed-cidr=192.168.1.0/25, IPs for your containers will be chosen from the first half of 192.168.1.0/24 subnet.
This should ensure unique MAC addresses across the cluster.
The original IEEE 802 MAC address comes from the original Xerox Ethernet addressing scheme. This 48-bit address space contains potentially 248 or 281,474,976,710,656 possible MAC addresses.
source
If you are concerned about lack of entropy (the IP to MAC mapping reduces it considerably), a better option may be to use a different mechanism for UUID generation. UUID versions 3, 4 and 5 do not take MAC address into account. Alternatively you could include the host machine's MAC in UUID generation.
Of course, whether this "considerable MAC space reduction" will have any impact of UUID generation should probably be tested before any code is changed.
Source linked to above:
// Generate a IEEE802 compliant MAC address from the given IP address.
//
// The generator is guaranteed to be consistent: the same IP will always yield the same
// MAC address. This is to avoid ARP cache issues.
func generateMacAddr(ip net.IP) net.HardwareAddr {
hw := make(net.HardwareAddr, 6)
// The first byte of the MAC address has to comply with these rules:
// 1. Unicast: Set the least-significant bit to 0.
// 2. Address is locally administered: Set the second-least-significant bit (U/L) to 1.
// 3. As "small" as possible: The veth address has to be "smaller" than the bridge address.
hw[0] = 0x02
// The first 24 bits of the MAC represent the Organizationally Unique Identifier (OUI).
// Since this address is locally administered, we can do whatever we want as long as
// it doesn't conflict with other addresses.
hw[1] = 0x42
// Insert the IP address into the last 32 bits of the MAC address.
// This is a simple way to guarantee the address will be consistent and unique.
copy(hw[2:], ip.To4())
return hw
}
Related
How do we retrieve the Local Ip address set of the NIC in NDIS 6. I will be doing some IP header modifications on the received Ethernet Frames, so will be looking for local ip of NIC card that my Filter Attached to.
It's generally a layering violation for an NDIS LWF driver (which operates at layer 2 of the OSI stack) to get involved with IP addresses (which are at layer 3 of the OSI stack).
If you have a very good reason do to this, you can query GetUnicastIpAddressTable. Keep in mind that a NIC may not have any IP address (e.g., it's used for non-IP protocols). Or it may carry IP traffic, but the OS doesn't know about any IP address (e.g., a guest VM is sending IP traffic through the host's NIC, but only the guest really knows the IP address).
In other words, NICs don't really have IP addresses. At best, you can say that the NIC may be associated with an IP interface which has some number of IP addresses.
I'm working on a machine that has multiple local network interfaces. Each interface is on a distinct network: for example, en0 might be 10.0.0.x, en1 192.168.1.x, etc.
I'd like to programatically answer the question: "Which interface is likely to be able to get to the given local IP address?"
For instance, if I wanted to talk to 10.0.0.75, that would be en0, 192.168.1.4 would be en1. I'm sure there's a smarter way to do this than manually comparing IP address fragments.
Bonus points if this works on iOS as well as Mac OS X - iOS does support multiple local interfaces, even if it's rare.
The approach I'd consider is to enumerate all the configured interfaces and get their IP-address and netmask. You can then calculate the Hostmin and Hostmax for that subnet, if your candidate falls within the Hostmax-Hostmin IP-range this host should be reachable directly on the local subnet of that interface.
You can find an online tool for these types of calculations at http://jodies.de/ipcalc. It's fairly straightforward bit-operations to do these calculations programatically.
There are IP-address functions you can use in arpa/inet.h, inet_addr for instance will give you a 32-bit value from a dotted-decimals formatted IP-address.
I would like to do a scan in a LAN network to find devices linked.
I'm developping an app in IOS for IPAD
How do I do???
Because those are mobile devices I will assume you want to find devices on a wireless network. Theoretically, since wifi uses shared medium for communication, you can passively listen for traffic flowing through the network and collect data about client without sending any packets. This is something that is commonly referred to as a promiscuous mode. In practice there is 99% chance that the network adapter driver will allow you only to get traffic destined for your MAC address. In that case you will need to resort to actively scanning the network subnet which is not 100% accurate and depending on how the network is implemented can be considered as a possible attack.
The simple way of scanning is sending ICMP requests (ping) to every IP address in the subnet and collecting data from those who send back the echo reply. This is not reliable because some hosts won't respond to ICMP echo request even if they are active. First thing you need is to find out your own IP address and the subnet mask, and calculate the range of possible addresses in your subnet. The range is obtained by using logical AND operator where operands are binary values of your IP address and subnet mask. This is an example from the program that calculates this for typical 192.168.1.1 subnet with 255.255.255.0 subnet mask (192.168.1.1/24 in CIDR notation):
Address: 192.168.1.1 11000000.10101000.00000001 .00000001
Netmask: 255.255.255.0 = 24 11111111.11111111.11111111 .00000000
Wildcard: 0.0.0.255 00000000.00000000.00000000 .11111111
Network: 192.168.1.0/24 11000000.10101000.00000001 .00000000
Broadcast: 192.168.1.255 11000000.10101000.00000001 .11111111
HostMin: 192.168.1.1 11000000.10101000.00000001 .00000001
HostMax: 192.168.1.254 11000000.10101000.00000001 .11111110
Then you would iterate through the range and ping every address. Another thing you can consider is listening for broadcast traffic such as ARP and collecting some of the information that way. I don't know what are you trying to make but you can't get many useful information this way, except for vendor of a host's network adapter.
Check my LAN Scan on Github. It does exactly what you want.
I recently used MMLANScan that was pretty good. It discovers IP, Hostname and MAC Address.
Bonjour have been around since 2002, have a look at it!
I mean, just look at their current tagline:
Bonjour, also known as zero-configuration networking, enables automatic discovery of devices and services on a local network using industry standard IP protocols. Bonjour makes it easy to discover, publish, and resolve network services with a sophisticated, yet easy-to-use, programming interface that is accessible from Cocoa, Ruby, Python, and other languages.
I am working on a distributed application in which a set of logical nodes communicate with each other.
In the initial discovery phase, each logical node starts up and sends out a UDP broadcast packet to the network to inform the rest of the nodes of its existence.
With different physical hosts, this can easily be handled by agreeing on a port number and keeping track of UDP broadcasts received from other hosts.
My problem is - I need to be able to be able to handle the case of multiple logical nodes on the same machine as well.
So in this case, it seems I cannot bind to the same port twice. How do I handle the node discovery case if there are two logical nodes on the same box ?? Thanks a lot in advance !!
Your choices are:
Create a RAW socket and listen to all packets on a particular NIC, this way ,by looking at the content of each packet, the process will identify if the packet is for destined for itself. The problem with this is tons of packets you would have to process. This is why kernels of our operating systems bind sockets to processes, so the traffic gets distributed optimally.
Create a specialized service, i.e. a daemon that will handle announcements of new processes that will be available to execute the work. When launched, the process will have to announce its port number to the service. This is usually how it is done.
Use virtual IP addresses for each process you want to run, each process binds to different IP address. If you are running on a local network, this is the simplest way.
Define range of ports and scan this range on all ip addresses you have defined.
Looking at this related SO question, I can't help but wonder about the uniqueness of MAC addresses.
How unique are MAC addresses?
I'm using them to semi-uniquely identify users. I have a website that users of virtually any device (PC, Mac, iPhone, Android phone, etc.) and any OS can hit via an HTTP request. I use a combination of IP address and MAC address to identify unique users.
I assume the following cases can exist:
A device has no MAC address (unlikely, sure, but anyway)
A device has a unique MAC address
A device has multiple unique MAC addresses
Two or more devices have the same MAC address
The first three of these cases are unique (the third because I only need a single unique MAC address). For the fourth case, how likely is this?
That is: given 100 random users (perhaps Windows users for any Windows OS), how many of them can I expect to have the same MAC address? Is it just generally because of the limited length of MAC addresses? Or is it dependent on some sort of purposeful configuration change (MAC address spoofing)?
I'm okay with MAC addresses being semi-unique, I just want some clarity on how to interpret the data.
(I'm using the C# code against .NET 2.0 in the linked question against .NET 2.0.)
This is only true on the same network.
MAC addresses are resolved locally using ARP to route local packets at a hardware level. ARP is not a routable protocol and is not resolved across subnets.
If your webserver is behind a router with port mapping and all the incoming traffic to it is coming from that router, then every connection will appear to come from the MAC address of the router, you won't 'see' the MAC address of the original machine, not unless you can pick it up with a web page somehow (i'm not a web genius so don't ask me on that one, but Im guessing you'd need some heavy lifting at the client end with Java, or some other kind of active component to interrogate the local machine, easier to use a cookie)
You can find out the Public IP address of the remote machine where the outgoing NAT took place, but once again, there could be multiple device connecting from behind a router which would limit the usefulness of this method, and it means need to look into IP packets, which I have no idea how you can do that from a web server (probably can't?).
Anyway, this is what cookies are for, a way of leaving an identifier on a remote machine so you can see where traffic came from. If people don't accept cookies, unless you start getting very very creative you aren't going to be able to uniquely identify them.
(BTW Mac address are always unique (ok, you can occasionally find a reused MAC, but its extremely rare, or at least needs to be for networking to work!), thats the purpose of them, it's just not much help in this scenario if you are not on the same network)
100 random users (perhaps Windows users for any Windows OS), how many
of them can I expect to have the same MAC address?
Zero. And when they are in the same network, they could not communicate with the same MAC-Address, since Ethernet uses them to find the Computer. They are pretty unique. Producers of network cards get ranges of addresses they may assign to their products.
But: There are ways to manipulate your MAC Address, and there are scenarios where people do just that! For example when you want to enter a network, which is restricted to certain MAC-Addresses, you can manipulate your own to match one of those (if you find out, which ones are on the whitelist). Since I don't know, what your szenario is, (what you wnat to accomplish), I can not tell you if that is relevant for yout.
You just need to understand the difference between a MAC Address (that can be changed) and the identifier of your hardware [your Network Interface Controler to be precise] (that is forever assigned by its the manufacturer).
MAC address is the name of your device when it connects to the internet (through a rooter or a switch). You can change your MAC Address, but by default, this ID uses the identifier assigned by the manufacturer of the network interface controller (NIC) (e.g. your Wifi antena)
Exemple: I've changed my MAC address for my two computers using the same MAC Address: 00:01:02:03:04:06. My computers seems to be the same when connected to the same router but the identifier of their wifi antena remain unchanged and different.