I have a requirement to remove a running node from an mnesia cluster. This is a legitimate node that needs to have some maintenance performed. However, we want to keep this node running and servicing requests. I found this post. Which helps remove it from the additional nodes. However, once you re-start mnesia on the orphan node, it returns to the other nodes in the cluster.
From each of the non-orphan nodes, I run a script that does the following:
rpc:call('node_to_be_orphaned', mnesia, stop, []),
mnesia:del_table_copy(schema, 'node_to_be_orphaned'),
^^ At this point mnesia:system_info(db_nodes) shows that the node has indeed been removed.
rpc:call('node_to_be_orphaned', mnesia, start, []),
Now it's back. Ugh!
So, I then tried to flip it and remove the other nodes from the orphan first adding the following.
rpc:call(ThisNode, mnesia, stop, []),
rpc:call('node_to_be_orphaned', mnesia, del_table_copy, [schema, node()]),
rpc:call(ThisNode, mnesia, start, []),
This just creates a loop with no difference.
Is there a way to take a node out of mnesia clustering while leaving it up-and-running?
Any and all guidance is greatly appreciated
The schema is what is bothering you. You can add nodes, but removing them while keeping the table copies is, err, difficult. This is what happens when a node is connected to a distributed schema, besides receiving a new schema :
Adding a node to the list of nodes where the schema is replicated will
affect two things. First it allows other tables to be replicated to
this node. Secondly it will cause Mnesia to try to contact the node at
start-up of disc-full nodes.
This is what the documentation says about disconnecting a node from a distributed table while still keeping the schema running on the node:
The function call mnesia:del_table_copy(schema, mynode#host) deletes
the node 'mynode#host' from the Mnesia system. The call fails if
mnesia is running on 'mynode#host'. The other mnesia nodes will never
try to connect to that node again. Note, if there is a disc resident
schema on the node 'mynode#host', the entire mnesia directory should
be deleted. This can be done with mnesia:delete_schema/1. If mnesia is
started again on the the node 'mynode#host' and the directory has not
been cleared, mnesia's behaviour is undefined.
An existing distributed schema can't be kept on a disconnected node. You have to recreate one, and copy the table info.
If you wish to keep the current schema on your node, you could remove any shared table from it and use purely local tables instead.
If you really wish to remove the node from the schema, you could export the data, erase the schema and create a new, undistributed one, and import the data, for testing and development.
Here are some useful functions you could use in both cases:
Copying a mnesia table
Mnesia tables can be easily copied, like in this example I just wrote (and tested) for the sheer fun of it:
copy_table(FromTable,ToTable) ->
mnesia:create_table(ToTable, [
{attributes, mnesia:table_info(FromTable,attributes)},
{index, mnesia:table_info(FromTable,index)},
% Add other attributes to be inherited, if present
{record_name,FromTable},
{access_mode, read_write},
{disc_copies,[node()]}
]),
Keys = mnesia:dirty_all_keys(FromTable),
CopyJob = fun(Record,Counter) ->
mnesia:write(ToTable,Record,write),
Counter + 1
end,
mnesia:transaction(fun() -> mnesia:foldl(CopyJob,0,FromTable) end).
This function would copy any table (distributed or not) to a merely local onem keeping it's attributes and record definitions. You would have to use mnesia:read
Exporting/importing a mnesia table to/from a file
This other functions export a mnesia table to a file, and import it back again. They would need some minor tweaks to import them to an arbitrary named table. (you could use mnesia:ets/1 for the sheer experience of it):
export_table(Table) ->
Temp = ets:new(ignoreme,[bag,public]),
Job = fun(Key) ->
[Record] = mnesia:dirty_read(Table,Key),
ets:insert(Temp,Record) end,
Keys = mnesia:dirty_all_keys(Table),
[Job(Key) || Key <- Keys],
Path = lists:concat(["./",atom_to_list(Table),".ets"]),
ets:tab2file(Temp,Path,[{extended_info,[md5sum,object_count]}]),
ets:delete(Temp).
import_table(Table) ->
Path = lists:concat(["./",atom_to_list(Table),".ets"]),
{ok,Temp} = ets:file2tab(Path,[{verify,true}]),
{atomic,Count} = mnesia:transaction(fun() ->
ets:foldl(fun(Record,I) -> mnesia:write(Record),I+1 end
,0
,Temp)
end),
ets:delete(Temp).
Related
I have a distributed application.
In this, a Master node starts a mnesia schema with 4 tables. Some of them are replicated to other nodes, some are not.
When a node spawns, it registers at the master node and is added to the schema and the data are replicated to this node.
How can I ensure that my replication is finished?
I tried the following:
Timeout=60000,
TabList = [tab1, tab2, tab3, tab4],
mnesia:wait_for_tables(TabList, Timeout).
However, it does not take 60 seconds, not even 5 seconds until I get an error:
{{badmatch,{aborted,{no_exists,tab1}}}
Obviously it does not work..
When a new node joins a cluster, a rpc call from the master node performs the following function on the new node:
start_Mnesia(MasterNode) ->
mnesia:start(),
mnesia:change_config(extra_db_nodes, [MasterNode]),
Tabs=mnesia:system_info(tables) -- [schema],
[mnesia:add_table_copy(Tab, node(), ram_copies) || Tab <- Tabs].
Is it also waiting until it is written to ram_copies?
Thanks.
When a node joins your mnesia cluster it is already synchronised, regardless of copies of tables it, or other nodes, do or do not have.
You should see that a new node, after registering with your master and being added to the cluster, can already access all your tables. Adding a copy of a table doesn't change that, regardless of the state/stage of that copy.
When you add a copy of a table on your new node you can continue to run transactions, during and after replication, and the data that is or is not replicated to the node originating the transaction will make no difference to the correctness of the result.
So, if you are concerned just with synchronisation in terms of keeping your transactions ACID, then don't worry.
If you are concerned when when your data is actually replicated and stored safely on the other nodes, that's a different thing. In this case, I have observed that when you runmnesia:add_table_copy(Table, NewNode, disc_copies) it blocks, and returns only when NewNode has copied the data to the filesystem.
Remember though, unless you run mnesia:sync_transaction/3 all the time you don't have guarantees about data actually being on any disc after a transaction completes anyway.
Code:
mnesia:create_table(local_config,
[{ram_copies, [node()]},
{local_content, true},
{attributes, record_info(fields, local_config)}]),
mnesia:add_table_copy(local_config, node(), ram_copies),
What's the purpose of this add_table_copy ?
I found ejabberd_config read configuration later with ETS api directly. Is it relevant with this redundant copy ?
Mnesia is just wrapper around ets adding distribution, transactions, and few other features. In this case we create table only on this node, without any distribution or dist storage. So in the end we create only one ets table, only on local node.
But this does not guarantee any fault tolerance. If you look into how ets works you will see that each table is assigned to some process. If process dies, all data disperse. To protect our self's from this loss we create one more table, assigned to different process, that will act as backup for the firs one.
There is several implied question in a single one:
Why do we need to add a local copy after adding a table ?
ejabberd is design to run in cluster. Adding a local copy is required when starting ejabberd for the first time on second cluster node (and subsequent ones).
Is it relevant to create a Mnesia table if later the table is used using ETS ?
ETS is the underlying storage for Mnesia. If you really know what you are doing you can sometime manipulate data using direct access to ETS functions. This is possible for example for performance or convenience reason, especially it the content for the table is local (no content cluster-wide synchronization).
Is it redundant to add table copy after table creation if content of the table is local ? It does not seems to involve other node.
It is not redundant. It is actually needed (for Mnesia schema management). If your remove the call to mnesia:add_table_copy/3, you will get issue in production during Mnesia database operations.
ejabberd is meant to run as a cluster. If this is the first node in the cluster to execute this code, then the table will be created. If this is not the first node, then the table is already created, but we want to make a local copy of the table on this node.
One of the two statements will essentially be a no-op, depending on when the nodes of the cluster execute.
Im using neo4j to store information about maps and sensors. Every time the map or sensor layout changes I need to keep a copy. I can imagine querying and manually creating said copy but I'm wondering if it's possible to build a neo4j type query that would do this for me.
So far all I've come up with is a way to replicate the nodes in a given label:
match ( a:some_label { some_params }) with a create ( b:some_label ) set b=a,b.other_id=value;
This would allow me to put version and time stamp info on a given snap shot.
What it doesn't do is copy the edge information. Suggestions? Maybe a second (similar) query?
If I understand you correctly, you are essentially trying to maintain a history of the state of a node and the state of its incoming relationship. One way to do this is to chain the nodes in reverse chronological order.
For example, suppose the nodes in the chain are labeled Some_label and the relationships are of type SOME_TYPE. The head node of the chain is always the current (most recent) node. Unless a Some_label node is chronologically the earliest node in the chain, it will have a SOME_TYPE relationship to the previous version of the node.
Here is how you'd insert a new relationship and node (with some properties) at the head of the chain. (Just to set up this example, I assume that the first node in the chain is linked to by some node labeled HeadRef).
MATCH (x:HeadRef)-[r1:SOME_TYPE]->(a1:Some_label)
CREATE (x)-[r2:SOME_TYPE {x: "ghi"}]->(a2:Some_label {a:123, b: true})-[r:SOME_TYPE]->(a1)
SET r=r1
WITH r1
DELETE r1
Note that this approach is also much more performant than maintaining your own other_id property to link nodes together. You should always use relationships instead -- that is the graph DB way.
I'm using mongodb replica set in my rails application with 1 primary(node A) and 1 secondary node(node B).
It was working prefectly fine until i added one more node(node C) and made node C as primary. Now that primary node (node C) is having all the content but as per my observation content created on previous primary(node A) can only be read now but not edited or destroyed. As i have understood that data can only be written to primary node so i guess data from secondary(node A- earlier primary) can only be read while being accessed.
Is this a common behaviour or i'm missing something?
EDIT:
I took a db dump of replica set from the primary node(node C) and then db.dropDatabase() and mongorestore again. I found data missing in some collections. Can anyone explain what could be the issue.
In a mongodb replica set you can only write (modify, create, destroy) on the primary node. Writes are then propagated to other (secondary) nodes in the replica set. Note that this propagation may not be immediate.
However when the primary change you should be able to write on data previously written by another primary.
Note that when you add a node to a replica set, it's preferable to load the latest database backup within this node before. The replication process is based on an oplog shared between each node that indicates creation/deletion/update, however this oplog has a limited number of entries. So earlier entries may not be considered by your new primary ...
Is there a way to do local writes and and global reads ( without replication ) using mnesia. Eg: node A writes to its local DB and node B reads from node A's DB.
Node B does not have any data of its own, apart from the schema information stored locally.
According to the documentation, {local_content, true} seems like what I need to use, but I have been unsuccessful trying to get node B to read node A's data.
My schema and table configuration look like this:
On nodeA#ip1:
net_adm:ping('nodeB#ip2').
rd(user, {name, nick}).
mnesia:create_schema([node()|nodes()]).
mnesia:start().
mnesia:create_table(user, [ {local_content, true},
{disc_copies, [node()]},
{attributes,record_info(fields, user) }]).
%% insert data and list rows on nodeA
%% WORKS
On nodeB#ip2:
mnesia:start().
%% code to list rows from user table on nodeA
%% throws an ERROR saying table does not exist.
Is the configuration wrong or can this be done in any other way?
I don't think you can do it the way you mention. Another way of doing it would probably be to make an rpc call to node A and get the data that way. There is no point in using mnesia to do the read from node B because it will essentially just do an RPC anyway.
So node B should be:
rpc:call(nodeA#ip1, mnesia, read, ....).
Hope this is what you [somewhat] needs.
EDIT:
Oh and I forgot to mention that you don't need the schema on both nodes for this to work. This is assuming that Node B doesn't really care about sharing any other data with Node A it just reads it; In other words just keep all the mnesia stuff on Node A and just do RPC calls from Node B.
You may need to add node B to the schema's extra_db_nodes config thingy. You shouldn't have to do that if it's a disc based db, but in RAM it's mandatory to make it do what you want.
Not sure about the specifics, i may be confusing where to put stuff. I'm not too experienced with mnesia, but the docs indicate that you should do this.