Is there a simple way to get the IP address of a connected erlang node? I'd like to initiate a SCTP connection with a few nodes and due to the way the system is engineered, the knowledge I have about them is just their node() atom.
More precisely, I am wondering if there is some API provided by Erlang (or some derived technique) that allows to obtain a node's IP address given its identifier atom().
You can use the rpc module to call the function on a foreign node
example:
rpc:call(Node,inet,getif,[])
note: this only works on nodes that are already connected via erlang distribution
I solved this problem by starting a process on the node and having the process send a message containing its IP addresses. If anyone knows of a more elegant solution, I would like to hear it.
The command I used to get the address after the process on the node had been started was: inet:getif(). Keep in mind that the result of that command includes the loop-back address.
Something to consider is that each node may have multiple IP addresses and the SCTP server may not be listening on all of them.
The other idea I thought about trying was to convert the atom returned from node() into a string, parse the string to get the hostname, and perform a DNS lookup. It might work, but I have never tried it. The result of the DNS lookup should be cached, so there might not be a network round trip. Also, I really hate assuming anything about the atom return from node().
It looks like net_kernel:nodes_info() - for all nodes - and net_kernel:node_info(Node) for a single node - have this information, and more, although it does not look like it's published in the man page. This seems like a better solution in some ways because it will also work with things like Java and C nodes that you can't send functions to.
Related
I need to build a system that consist of:
Nodes, each mode can accept one input.
The node that received the input shares it with all nodes in the network.
Each node do a computation on the input (same computation but each node has a different database so the results are different for each node).
The node that received the input consolidate each node result and apply a logic to determine the overall result.
This result is returned to the caller.
It's very similar to a map-reduce use case. Just there will be a few nodes (maybe 10~20), and solutions like hadoop seems an overkill.
Do you know of any simple framework/sdk to build:
Network (discovery, maybe gossip protocol)
Distribute a task/data to each node
Aggregate the results
Can be in any language.
Thanks very much
Regads;
fernando
Ok to begin with, there are many ways to do this. I would suggest the following if you are just starting to tackle this architecture:
Pub/Sub with Broker
Programs like RabbitMQ are meant to easily allow for variable amounts of nodes to connect and speak to one another. Most importantly, they allow for transparency and observability. You can easily ask the Broker which nodes are connected and even view messages in transit. Basically they are a 'batteries included' means of delaying with a large amount of clients.
Brokerless (Update)
I was looking for a more 'symmetric' architecture where each node is the same and do not have a centralized broker/queue manager.
You can use a brokerless Pub/Subs, but I personally avoid them. While they have tooling, it is hard to understand their registration protocols if something odd happens. I generally just use Multicast as it is very straight forward, especially if each node has just one network interface, and you can extend/modify behavior just with routing infra.
Here is how you scheme would work with Multicast:
All nodes join a known multicast address (IE: 239.1.2.3:8000)
All nodes would need to respond to a 'who's here' message
All nodes would either need to have a 'do work' api either via multicast or from consumer to node (node address grabbed from 'who's here message)
You would need to make these messages yourself, but given how short i expect them to be it should be pretty simple.
The 'who's here' message from the consumer could just be a message with a binary zero.
The 'who's here' response could just be a 1 followed by the nodes information (making it a TLV would probably be best though)
Not sure if each node has unique arguments or not so i don't know how to make your 'do work' message or responce
Can anyone tell me in the course, it is possible to override the parameters of individual box.cfg on a running instance. For example, add a replica, for several days I have been trying to deploy three replicas on three hosts via the docker service stack.
When I raise my hands on each server, everything works, through deploy they do not see each other and fall. I've tried all sorts of ways. hung up the endpoint on the target nodes, when requested, it gives the ip of the machine on which the container rises, if the ip matches one of those indicated in SEED, then substitutes the internal ip of the container instead (otherwise it cannot connect to itself).
In theory, it all works as I described, but there are suspicions that everything is not much different, I suppose that the problem is that before the declaration of box.cfg the instance does not reserve the address. Alas, I can not go inside the container because it cannot rise. I got the idea that if all three nodes are declared at the minimum settings and as soon as they rise to listen to the subnet, as soon as the node finds another, it will write it to replication and override box.cfg. Correct me please who had experience.
Some of the box.cfg parameters are dynamic. For example, the box.cfg{listen=}. You can set this one from the Lua code as you wish. In your case, if the container gets its IP address later, you need to specify only the port in listen. This way, Tarantool will listen on all possible interfaces.
The replication_source is a bit trickier. You can set it dynamically, but your first (initializing) call to box.cfg should be with the replication_source. This is because all instances that are initialized without this parameter will create their own replicaset, and it will make it impossible to join them to another replicaset.
You can read more about Tarantool replication architecture here: https://www.tarantool.io/en/doc/latest/book/replication/repl_architecture/
I am looking to dynamically set an erlang node to 'hidden' or set 'connect_all' after the node has already been created. Is this possible in erlang?
There is an undocumented net_kernel:hidden_connect_node(NodeName) function that can be used on a per-connection basis for NodeName not to share all the connection details of the caller.
There is no guarantee related to its long term support, but that's currently the only way to do things dynamically.
Thanks to #I GIVE TERRIBLE ADVICE (AND WRITE AWESOME ERLANG BOOKS) for sharing this gem of knowledge. I would also like to highlight how it has been particularly useful in my specific case :
Context :
I have several machines that host an Erlang node running my OTP application
The nodes are configured in a wireless peer-to-peer setup
For testing purposes, I would like to observe the behaviour of the cluster when multi-hop is required from a node A to another node B.
So far my best (and only) solution has been to physically move around the nodes such that they can only reach neighbours in range of their Wi-Fi antenna.
Bottom line for those that are in situations similar to what I have described, this is a very handy function for clustering nodes without completely removing the default transitive behaviour.
I have a Gatling load test that does a huge call to a webservice, and I can get response time information nicely, the problem is that all calls are done as a single user, thus cannot see the amount of concurrent users from Jennifer5 monitoring tool. I've searched but couldn't find a way to make multiple unique IP users to call this webservice, thus showing me the amount of concurrent users in the monitoring tool, however approximate that value would be.
I've found this page but this does not work at all, what should I do? I do not wish to switch to JMeter now, even though I think it does have this capability
As per the forum you are referring to
Note that, of course, you won't be able to do IP spoofing, but only use valid IP aliases.
So given your Scala code is correct you must have all the simulated IP addresses present in your operating system. Theres is a possibility to have more than one IP address on a physical network interface via IP aliasing, refer to your operating system documentation in order to learn how set additional addresses up. You may also find Using IP Spoofing to Simulate Requests from Different IP Addresses with JMeter guide interesting
So I installed and configured SNMP on lets say node A. Also lets say that node A mounts through NFS from storage node B. Finally I have monitoring node C. When node C requests information via snmp from node A, then node beautifully complies when nothing is wrong.
I've been running into this problem though. Lets say that storage node B fails. If node C requests info via SNMP from node A the snmp demon freezes on node A because it can't reach the mounting point.
This sort of action is very counter intuitive from the respect of the purpose of SNMP. SNMP is used for monitoring system stats. If something fails I want to know that soemthing fails. From node C's perspective in this case it looks as if someone turned off node A (which clearly is not what happend).
I looked for configuration settings such that the SNMP will skip over MIBS it cant read, but found nothing. Any ideas?
Turns out that nfs timeout coupled with the snmp demon is a old bug. The way to fix it is to basically skip over all the NFS mounted points by using the following option in the snmpd.conf
skipNFSInHostResources yes