Related
I have a SOAP node, that retrieve information from a URL in a tree structure.
Then i have a compute node to define each environment variable to each namespace variable of the SOAP retrieve.
And finally, i have a mapping node, to move the content to my message assembly structure in XML.
The error its giving me it's this (IN THE COMPUTE NODE):
I have a structure like this:
ListDocs
Description
DocType
ListTypes
Attribute
Lenght
Description
Nature
Required
ListDocs
Description
DocType
ListTypes
Attribute
Lenght
Description
Nature
Required
ListDocs
Description
DocType
ListTypes
Attribute
Lenght
Description
Nature
Required
The problem is that, when i do the definition of the variables, I do it like the code below, in the COMPUTE NODE:
WHILE I < InputRoot.SOAP.Body.ns:obterTiposDocProcessosResponse.ns:return.ns75:processo.ns75:listaTiposDocumentos
DO
SET Environment.Variables.XMLMessage.return.process.listDocs.description = InputRoot.SOAP.Body.ns75:processo.ns75:listDocs.ns75:description;
SET Environment.Variables.XMLMessage.return.process.listDocs.tipoDocumento = InputRoot.SOAP.Body.ns75:processo.ns75:listDocs.ns75:DocType;
SET Environment.Variables.XMLMessage.return.process.listDocs.listTypes.attribute = InputRoot.SOAP.Body.ns75:processo.ns75:listDocs.ns75:listTypes.ns75:atribbute;
SET Environment.Variables.XMLMessage.return.process.listDocs.listTypes.lenght = InputRoot.SOAP.Body.ns75:processo.ns75:listDocs.ns75:listTypes.ns75:lenght;
SET Environment.Variables.XMLMessage.return.process.listDocs.listTypes.description = InputRoot.SOAP.Body.ns75:processo.ns75:listDocs.ns75:listTypes.ns75:description;
SET Environment.Variables.XMLMessage.return.process.listDocs.listTypes.nature = InputRoot.SOAP.Body.ns75:processo.ns75:listDocs.ns75:listTypes.ns75:nature;
SET Environment.Variables.XMLMessage.return.process.listDocs.listTypes.required = InputRoot.SOAP.Body.ns75:processo.ns75:listDocs.ns75:listTypes.ns75:required;
SET I = I+1;
END WHILE;
BUT, in my XML final structure, it only prints the values of my first listDocs, and i want to print all of my listDocs structures.
NOTE: WITH THE WHILE LIKE THIS, IT DOESN'T EVEN WORK. I HAVE TO REMOVE THE WHILE TO PRINT THE FIRST listDocs like i said Above.
Any help?
I NEED HELP TO LOOP THE STRUCTURES, WITH A WHILE OR SOMETHING.
You should try to use the following synthax :
DECLARE I INTEGER 1;
DECLARE J INTEGER;
J = CARDINALITY(InputRoot.SOAP.Body.ns:obterTiposDocProcessosResponse.ns:return.ns75:processo.ns75:listaTiposDocumentos[])
WHILE I <= J DO
SET Environment.Variables.XMLMessage.return.process.listDocs.description = InputRoot.SOAP.Body.ns75:processo.ns75:listDocs[I].ns75:description;
....
END WHILE;
You only missed the CARDINALITY function to get the number of elements, and also the [] to define the table, and then using this [I] while accessing the elements
Note : in my sample above, the environment will be overridden at each iteration of the loop, so only the last record will be printed. You can use the [I] in the output as well if you want to construct a table in output, or you can use the following code to push each message to the output terminal (this means you have one message in input, and 3 message coming out of the output terminal)
PROPAGATE TO TERMINAL 'Out';
So for example, based on your code, if you want to generate 3 messages based on your input containing multiple element :
DECLARE I INTEGER 1;
DECLARE J INTEGER;
J = CARDINALITY(InputRoot.SOAP.Body.ns:obterTiposDocProcessosResponse.ns:return.ns75:processo.ns75:listaTiposDocumentos[])
WHILE I <= J DO
SET Environment.Variables.XMLMessage.return.process.listDocs.description = InputRoot.SOAP.Body.ns75:processo.ns75:listDocs[I].ns75:description;
SET Environment.Variables.XMLMessage.return.process.listDocs.tipoDocumento = InputRoot.SOAP.Body.ns75:processo.ns75:listDocs[I].ns75:DocType;
SET Environment.Variables.XMLMessage.return.process.listDocs.listTypes.attribute = InputRoot.SOAP.Body.ns75:processo.ns75:listDocs[I].ns75:listTypes.ns75:atribbute;
SET Environment.Variables.XMLMessage.return.process.listDocs.listTypes.lenght = InputRoot.SOAP.Body.ns75:processo.ns75:listDocs[I].ns75:listTypes.ns75:lenght;
SET Environment.Variables.XMLMessage.return.process.listDocs.listTypes.description = InputRoot.SOAP.Body.ns75:processo.ns75:listDocs[I].ns75:listTypes.ns75:description;
SET Environment.Variables.XMLMessage.return.process.listDocs.listTypes.nature = InputRoot.SOAP.Body.ns75:processo.ns75:listDocs[I].ns75:listTypes.ns75:nature;
SET Environment.Variables.XMLMessage.return.process.listDocs.listTypes.required = InputRoot.SOAP.Body.ns75:processo.ns75:listDocs[I].ns75:listTypes.ns75:required;
PROPAGATE TO TERMINAL 'Out';
END WHILE;
RETURN FALSE;
For your global information, the RETURN TRUE is the instruction "pushing" the message built in the ESQL code to the output terminal. If you use PROPAGATE instruction (same effect), you should RETURN FALSE to avoid sending an empty message after looping on your records. Another way to do it is to propagate on another terminal (i.e : 'out1'), and keep the return true. In this case, you would have all you records coming out from the out1 terminal, and a message going out of the output temrinal (due to the return true) once all the messages have been propagated (this might be useful in many situations)
So the key to understanding IIB and ESQL is that you are looking at in memory Trees built from nodes.
Each Node has pointers/REFERENCEs to PARENT, NEXTSIBLING, PREVSIBLING, FIRSTCHILD and LASTCHILD Nodes.
Nodes also have FIELDNAME, FIELDNAMESPACE, FIELDTYPE and FIELDVALUE attributes.
And last but not least that you are building Output Trees by navigating Input Trees. The Environment Tree, which you are using, is a special long lasting Tree that you can both read from and write to.
So in your code InputRoot.SOAP.Body.ns75:processo.ns75:listDocs can be thought of as shorthand for instructions to navigate to the ns75:listDocs Node. The dots '.' tell ESQL interpreter the name of the child Node of the current Node. If you were telling someone how to navigate the Nodes it would go something like this.
Start at InputRoot. InputRoot is a special Node that is automatically available to you in your ESQL Modules code.
Navigate to the first child Node of InputRoot that has the name SOAP
Navigate to the first child Node of SOAP that has the name Body
Navigate to the first child Node of Body that has the name listDocs and is in the ns75 namespace.
In the absence of a subscript ESQL assumes you want the first Node that matches the specified name ns75:listDocs and ns75:listDocs[1] both refer to the same Node.
This explains what was happening in your code. You were always navigating to the same listDocs[1] node in the InputRoot and Environment Trees.
#Jerem's code improves on what you were doing by at least navigating across the listDocs nodes in the Input tree.
For each iteration of the loop the subscript [I] gets incremented and thus it chooses a different listDocs Node. The listDocs Nodes are siblings and thus the code will access the first, second and third instance of the listDocs Nodes.
InputRoot.SOAP.Body.ns75:processo.ns75:listDocs[1] <-- Iteration I=1
InputRoot.SOAP.Body.ns75:processo.ns75:listDocs[2] <-- Iteration I=2
InputRoot.SOAP.Body.ns75:processo.ns75:listDocs[3] <-- Iteration I=3
To correct #Jerem's answer you'd need to use subscripts on the lefthand side of the statement as well. Picking the description field as an example you'd need to change your code as follows.
SET Environment.Variables.XMLMessage.return.process.listDocs[I].listTypes.description = InputRoot.SOAP.Body.ns75:processo.ns75:listDocs[I].ns75:listTypes.ns75:description;
Using subscripts is regarded as a performance no no. Imagine you had 10,000 listDocs this would result in each and every iteration of the loop walking down the tree over the InputRoot, SOAP, Body, ns75:processo Nodes and then across the listDocs sibling nodes until it found the ns75:listDocs[I] Node.
This means by the time we get round to processing ns75:listDocs[10000] it will have had to repetetively walked over all the other listDocs Nodes time and time again, In fact we can calculate it would have walked over (4 x 10,000) + ((10,000 x (10,000 + 1)) / 2) = 50,045,000 Nodes
So it's REFERENCE's to the rescue and also the answer to your question. Try a loop like this.
DECLARE ns75 NAMESPACE 'http://something.or.other.from.your.wsdl';
DECLARE InListDocsRef REFERENCE TO
InputRoot.SOAP.Body.ns75:processo.ns75:listDocs;
WHILE LASTMOVE(InListDocsRef) DO
DECLARE EnvListDocsRef REFERENCE TO Environment;
CREATE LASTCHILD OF Environment.Variables.XMLMessage.return.process AS EnvListDocsRef NAME 'listDocs';
SET EnvListDocsRef.description = InListDocsRef.ns75:description;
SET EnvListDocsRef.tipoDocumento = InListDocsRef.ns75:DocType;
SET EnvListDocsRef.listTypes.attribute = InListDocsRef.ns75:listTypes.ns75:atribbute;
SET EnvListDocsRef.listTypes.lenght = InListDocsRef.ns75:listTypes.ns75:lenght;
SET EnvListDocsRef.listTypes.description = InListDocsRef.ns75:listTypes.ns75:description;
SET EnvListDocsRef.listTypes.nature = InListDocsRef.ns75:listTypes.ns75:nature;
SET EnvListDocsRef.listTypes.required = InListDocsRef.ns75:listTypes.ns75:required;
MOVE InListDocsRef NEXTSIBLING REPEAT NAME;
END WHILE;
The code above only walks over 4 + 10,000 Nodes i.e. 10 thousand Nodes vs 50 million Nodes.
A couple of other useful things to know about setting references are:
To point to the last element you can use a subscript of [<]. So to point to the last ListItem in the aggregate MyList you would code Environment.MyList.ListItem[<]
You can use an asterisk * to set a reference to an element in the tree that you don't know the name of e.g. Environment.MyAggregate.* points to the first child of MyAggregate regardless of it's name.
You can also use asterisks * to choose an element irregardless of it's namespace InListDocsRef.*:listTypes.*:description
For anonymous namespaced elements use *:* but be very careful * and *:* are not the same thing the first means no namespace any element and the second means any namespace any element.
To process lists in reverse combine the [<] subscript with the PREVIOUSSIBLING option of MOVE.
So a chunk of code for reversing a list might go something like:
DECLARE MyReverseListItemWalkingRef REFERENCE TO Environment.MyList.ListItem[<];
WHILE LASTMOVE(MyReverseListItemWalkingRef) DO
CREATE LASTCHILD OF OuputRoot.ReversedList.Item NAME 'Description' VALUE MyReverseListItemWalkingRef.Desc;
MOVE MyReverseListItemWalkingRef PREVIOUSSIBLING REPEAT NAME;
END WHILE;
Learn how to use REFERENCES they are extremely powerful and one of your simplest options when it comes to performance.
In linked list implementation, the insertion of a new node to the linked list is usually written like this:
void push(struct node** head_ref, int new_data)
/* 1. allocate node */
struct node* new_node = (struct node*) malloc(sizeof(struct node));
/* 2. put in the data */
new_node->data = new_data;
/* 3. Make next of new node as head */
new_node->next = (*head_ref);
/* 4. move the head to point to the new node */
(*head_ref) = new_node;
(I took this code from http://quiz.geeksforgeeks.org/linked-list-set-2-inserting-a-node/)
and the node struct is:
struct node
{
int data;
struct node *next;
};
What I don't understand is the 3. and 4. of the insertion part. So you make the next pointer of new_node pointed to the head, and then the head points to the new_node? So that means the next pointer points to new_node?
It seems like a stupid question but I'm having trouble understanding it, so I hope someone can explain it to me. Thank you.
Well basically in a linked list all nodes are connected to each other. It depends upon u that where do u insert a new node either at end or start. Each time we insert a new node we will check the head pointer.
if(head == NULL) //it means that the node is empty
{
head = newNode; //so we will assign the new node to the head
}
else
{
node* temp = head; //creating a temp pointer that will go
// to the end of the linked list
while(temp -> next != NULL) { temp = temp->next; }
temp = newNode; //This will add the new node to the end
newNode->next = NULL;enter code here
}
If I understood correctly this is your scenario?
http://www.kkhsou.in/main/EVidya2/computer_science/comscience/313.gif
Your list is just a linked list with next pointer until list's last item that has null as pointer
Step 3 makes your new node to point to 2nd item that was at beginning of the list before this operation
Step 4 makes the list head to point to the new node
Hope this helps
/* 1. allocate node /
struct node new_node = (struct node*) malloc(sizeof(struct node));
/* 2. put in the data */
new_node->data = new_data;
/* 3. Make next of new node as head */
new_node->next = (*head_ref);
/* 4. move the head to point to the new node */
(*head_ref) = new_node;
In Step1 and 2, a new node is created and data is assigned to it.
When you list is empty, your *head_ref would be null.
or else if it has any elements, it would be pointing to that
Lets take an example
Now
*head_ref is null
when input is 1
newnode.data=1
newnode.next=null
*headref=newnode
now your *headref points to the latest node that is added ,this happens with step4
When you insert 2
newnode.data=2
newnode.next=*headref(to the node which is 1)
newnode=*headref
now your list is
1->2(*headref)
now if you add 3 here
it becomes
1->2->3(*headref)
Hope you understand
Rather than explaining it to you, I'm going to suggest a technique that will help you work out the answer for yourself.
Get a piece of paper, a pencil and an eraser.
Draw a box on the paper to represent each variable in your algorithm
Draw the initial linked list:
Draw a box to represent each existing node in the initial linked list.
Divide each box into sub-boxes representing the fields.
In each field write either a value, or a dot representing the "from" end of a pointer.
For each pointer, draw a line to the thing (e.g. node) that is pointed to, and put an arrowhead on the "to" end. A NULL pointer is a just a dot.
Now execute the algorithm.
Each time you allocate a new node, draw a new box.
Each time you assign something to a variable, or a field, rub out the current value and write / draw in the new value or the new dot / arrow.
If you do this carefully and systematically, you will be able to visualize exactly what the list insertion algorithm is doing.
This same technique can be used to visualize any list / tree / graph algorithm ... modulo your ability to get it all onto a sheet of paper, and the paper's ability to survive repeated rub-outs.
(This pencil and paper approach is very "old school". As in, this is what we were taught to do when I learned to program in the 1970's. A slightly more modern approach would be to use a whiteboard ...)
First of all head pointer is pointing to first node in list.
In (1) new node is created.
In (2) data is saved to new node.
In (3) The new node is pointing where the head is pointing(means to the first node)
In (4) now the head is made to point to new node, so therefore the new node is now the first node. thats it.
For example, given the Hopfield network pictured in the diagram attached..., where all nodes are originally at 0 (or "off"), is it possible for this network to relax to all 0s (that is, remain in this state)? All 0s are supposed to be a stable state for this diagram, but that doesn't make sense to me, because in this state, the input to each node would actually be dotProduct((0 0.. 0),(input1, input2 ... inputn)) = 0. But 0 >= 0, so that would make each node turn "on"/1, right? Perhaps I just don't understand how Hopfield networks relax to a stable state...
In case anyone else was interested, I found the answer...
Since a Hopfield network is a form of BAM network, it works in this way (rough pseudocode):
if(input dotted with weights > 0) node = "on"
else if(input dotted with weights < 0) node = "off"
else node remains the same
Thus, for this network, each node is receiving an input of 0, so each node will remain off (stay the same as its current state).
Disclaimer: The author is a newbie in Erlang.
Imagine, we have a graph consisting of 1M nodes, and each node has 0-4 neighbours (the edges are emanating from each node to those neighbours, so the graph is directed and connected).
Here is my choice of data structures:
To store the graph I use digraph, which is based on ETS tables. This allows fast (O(1)) access to the neighbours of a node.
For the list of unvisited nodes, I use gb_sets:take_smallest (the node is already sorted, and it is simultaneously deleted after fetching).
For the list of predecessors I use the dict structure, which allows to store the predecessors in the following way: {Node1,Node1_predecessor},{Node2,Node2_predecessor}.
For the list of visited nodes I use a simple list.
Problems:
The code becomes very hard to read and maintain when I try to update the weight of a node both in the digraph structure and in the Unvisited_nodes structure. It doesn't seem the right way to keep one 'object' with the 'fields' that need to be updated in two data structures simultaneously. What is the right way to do that?
The same question is about predecessors list. Where should I store the predecessor 'field' of a node 'object'? Maybe in the Graph (digraph structure)?
Maybe I should rethink the whole Dijkstra's algorithm in terms of processes and messages instead of objects (nodes and edges) and their fields(weights)?
UPD:
Here is the code based on the recommendations of Antonakos:
dijkstra(Graph,Start_node_name) ->
io:format("dijkstra/2: start~n"),
Paths = dict:new(),
io:format("dijkstra/2: initialized empty Paths~n"),
Unvisited = gb_sets:new(),
io:format("dijkstra/2: initialized empty Unvisited nodes priority queue~n"),
Unvisited_nodes = gb_sets:insert({0,Start_node_name,root},Unvisited),
io:format("dijkstra/2: Added start node ~w with the weight 0 to the Unvisited nodes: ~w~n", [Start_node_name, Unvisited_nodes]),
Paths_updated = loop_through_nodes(Graph,Paths,Unvisited_nodes),
io:format("dijkstra/2: Finished searching for shortest paths: ~w~n", [Paths_updated]).
loop_through_nodes(Graph,Paths,Unvisited_nodes) ->
%% We need this condition to stop looping through the Unvisited nodes if it is empty
case gb_sets:is_empty(Unvisited_nodes) of
false ->
{{Current_weight,Current_name,Previous_node}, Unvisited_nodes_updated} = gb_sets:take_smallest(Unvisited_nodes),
case dict:is_key(Current_name,Paths) of
false ->
io:format("loop_through_nodes: Found a new smallest unvisited node ~w~n",[Current_name]),
Paths_updated = dict:store(Current_name,{Previous_node,Current_weight},Paths),
io:format("loop_through_nodes: Updated Paths: ~w~n",[Paths_updated]),
Out_edges = digraph:out_edges(Graph,Current_name),
io:format("loop_through_nodes: Ready to iterate through the out edges of node ~w: ~w~n",[Current_name,Out_edges]),
Unvisited_nodes_updated_2 = loop_through_edges(Graph,Out_edges,Paths_updated,Unvisited_nodes_updated,Current_weight),
io:format("loop_through_nodes: Looped through out edges of the node ~w and updated Unvisited nodes: ~w~n",[Current_name,Unvisited_nodes_updated_2]),
loop_through_nodes(Graph,Paths_updated,Unvisited_nodes_updated_2);
true ->
loop_through_nodes(Graph,Paths,Unvisited_nodes_updated)
end;
true ->
Paths
end.
loop_through_edges(Graph,[],Paths,Unvisited_nodes,Current_weight) ->
io:format("loop_through_edges: No more out edges ~n"),
Unvisited_nodes;
loop_through_edges(Graph,Edges,Paths,Unvisited_nodes,Current_weight) ->
io:format("loop_through_edges: Start ~n"),
[Current_edge|Rest_edges] = Edges,
{Current_edge,Current_node,Neighbour_node,Edge_weight} = digraph:edge(Graph,Current_edge),
case dict:is_key(Neighbour_node,Paths) of
false ->
io:format("loop_through_edges: Inserting new neighbour node ~w into Unvisited nodes~n",[Current_node]),
Unvisited_nodes_updated = gb_sets:insert({Current_weight+Edge_weight,Neighbour_node,Current_node},Unvisited_nodes),
io:format("loop_through_edges: The unvisited nodes are: ~w~n",[Unvisited_nodes_updated]),
loop_through_edges(Graph,Rest_edges,Paths,Unvisited_nodes_updated,Current_weight);
true ->
loop_through_edges(Graph,Rest_edges,Paths,Unvisited_nodes,Current_weight)
end.
Your choice of data structures looks OK, so it is mostly a question of what to insert in them and how to keep them up to date. I'd suggest the following (I have changed the names a bit):
Result: A dict mapping Node to {Cost, Prev}, where Cost is the total cost of the path to Node and Prev is its predecessor on the path.
Open: A gb_sets structure of {Cost, Node, Prev}.
A graph with edges of the form {EdgeCost, NextNode}.
The result of the search is represented by Result and the graph isn't updated at all. There is no multiprocessing or message passing.
The algorithm goes as follows:
Insert {0, StartNode, Nil} in Open, where Nil is something that marks the end of the path.
Let {{Cost, Node, Prev}, Open1} = gb_sets:take_smallest(Open). If Node is already in Result then do nothing; otherwise add {Cost, Node, Prev} to Result, and for every edge {EdgeCost, NextNode} of Node add {Cost + EdgeCost, NextNode, Node} to Open1, but only if NextNode isn't already in Result. Continue with Open1 until the set is empty.
Dijkstra's algorithm asks for a decrease_key() operation on the Open set. Since this isn't readily supported by gb_sets we have used the workaround of inserting a tuple for NextNode even if NextNode might be present in Open already. That's why we check if the node extracted from Open is already in Result.
Extended discussion of the use of the priority queue
There are several ways of using a priority queue with Dijkstra's algorithm.
In the standard version of Wikipedia a node v is inserted only once but the position of v is updated when the cost and predecessor of v is changed.
alt := dist[u] + dist_between(u, v)
if alt < dist[v]:
dist[v] := alt
previous[v] := u
decrease-key v in Q
Implementations often simplify by replacing decrease-key v in Q with add v to Q. This means that v can be added more than once, and the algorithm must therefore check that an u extracted from the queue hasn't already been added to the result.
In my version I am replacing the entire block above with add v to Q. The queue will therefore contain even more entries, but since they are always extracted in order it doesn't affect the correctness of the algorithm. If you don't want these extra entries, you can use a dictionary to keep track of the minimum cost for each node.
in a simple MPI program I have used a column wise division of a large matrix.
How can I order the output so that each matrix appears next to the other ordered ?
I have tried this simple code the effect is quite different from the wanted:
for(int i=0;i<10;i++)
{
for(int k=0;k<numprocs;k++)
{
if (my_id==k){
for(int j=1;j<10;j++)
printf("%d",data[i][j]);
}
MPI_Barrier(com);
}
if(my_id==0)
printf("\n");
}
Seems that each process has his own stdout and so is impossible to have ordered lines output without sending all the data to one master which will print out. Is my guess true ? Or what I'm doing wrong ?
You guessed right. The MPI standard does not specify how stdout from different nodes should be collected for printing at the originating process. It is often the case that when multiple processes are doing prints the output will get merged in an unspecified way. fflush doesn't help.
If you want the output ordered in a certain way, the most portable method would be to send the data to the master process for printing.
For example, in pseudocode:
if (rank == 0) {
print_col(0);
for (i = 1; i < comm_size; i++) {
MPI_Recv(buffer, .... i, ...);
print_col(i);
}
} else {
MPI_Send(data, ..., 0, ...);
}
Another method which can sometimes work would be to use barries to lock step processes so that each process prints in turn. This of course depends on the MPI Implementation and how it handles stdout.
for(i = 0; i < comm_size; i++) {
MPI_Barrier(MPI_COMM_WORLD);
if (i == rank) {
printf(...);
}
}
Of course, in production code where the data is too large to print sensibly anyway, data is eventually combine by having each process writing to a separate file and merged separately, or using MPI I/O (defined in the MPI2 standards) to coordinate parallel writes.
I produced ordered output to a file before using the exact same method. You could try printing to a temporary file, printing the contents of said file and then deleting it.
Have the root processor do all of the printing. Use MPI_Send/MPI_Recv or MPI_Gather (or whatever) to send the data in turn from each processor to the root.
To solve this problem you can use short sleep. I use and then it works in 99%
printf("text nr 1\n");
MPI_Barrier(MPI_COMM_WORLD);
usleep(100);
printf("text nr 2\n");
It's not very elegant but works.