I have the following function for a hacky project:
% The Record variable is some known record with an associated table.
Query = qlc:q([Existing ||
Existing <- mnesia:table(Table),
ExistingFields = record_to_fields(Existing),
RecordFields = record_to_fields(Record),
ExistingFields == RecordFields
]).
The function record_to_fields/1 simply drops the record name and ID from the tuple so that I can compare the fields themselves. If anyone wants context, it's because I pre-generate a unique ID for a record before attempting to insert it into Mnesia, and I want to make sure that a record with identical fields (but different ID) does not exist.
This results in the following (redacted for clarity) stack trace:
{aborted, {{case_clause, {stuff}},
[{db, '-my_func/2-fun-1-',8, ...
Which points to the line where I declare Query, however there is no case clause in sight. What is causing this error?
(Will answer myself, but I appreciate a comment that could explain how I could achieve what I want)
EDIT: this wouldn't be necessary if I could simply mark certain fields as unique, and Mnesia had a dedicated insert/1 or create/1 function.
For your example, I think your solution is clearer anyway (although it seems you can pull the record_to_fields(Record) portion outside the comprehension so it isn't getting calculated over and over.)
Yes, list comprehensions can only have generators and assignments. But you can cheat a little by writing an assignment as a one-element generator. For instance, you can re-write your expression as this:
RecordFields = record_to_fields(Record),
Query = qlc:q([Existing ||
Existing <- mnesia:table(Table),
ExistingFields <- [record_to_fields(Existing)],
ExistingFields == RecordFields
]).
As it turns out, the QLC DSL does not allow assignments, only generators and filters; as per the documentation (emphasis mine):
Syntactically QLCs have the same parts as ordinary list
comprehensions:
[Expression || Qualifier1, Qualifier2, ...]
Expression (the template)
is any Erlang expression. Qualifiers are either filters or generators.
Filters are Erlang expressions returning boolean(). Generators have
the form Pattern <- ListExpression, where ListExpression is an
expression evaluating to a query handle or a list.
Which means we cannot variable assignments within a QLC query.
Thus my only option, insofar as I know, is to simply write out the query as:
Query = qlc:q([Existing ||
Existing <- mnesia:table(Table),
record_to_fields(Existing) == record_to_fields(Record)
]).
I want to replace the value of the 'Amount' key in a map (literal) with the sum of the existing 'Amount' value plus the new 'Amount' value such where both the 'type' and 'Price' match. The structure I have so far is:
WITH [{type:1, Orders:[{Price:10,Amount:100},{Price:11,Amount:200},{Price:12,Amount:300}]},
{type:2, Orders:[{Price:10,Amount:100},{Price:11,Amount:200},{Price:12,Amount:300}]},
{type:3, Orders:[{Price:10,Amount:100},{Price:11,Amount:200},{Price:12,Amount:300}]}] as ExistingOrders,
{type:2, Order:{Price:11,Amount:50}} as NewOrder
(I'm trying to get it to:)
RETURN [{type:1, Orders:[{Price:10,Amount:100},{Price:11,Amount:200},{Price:12,Amount:300}]},
{type:2, Orders:[{Price:10,Amount:100},{Price:11,Amount:250},{Price:12,Amount:300}]},
{type:3, Orders:[{Price:10,Amount:100},{Price:11,Amount:200},{Price:12,Amount:300}]}] as CombinedOrders
If there is no existing NewOrder.type and NewOrder.Price then it should obviously insert the new record rather than add it together.
Sorry, this is possibly really straight forward, but I'm not very good at this yet.
thanks
Edit:
I should add, that I have been able to get this working for a simpler map structure as such:
WITH [{type:1, Amount:100},{type:2, Amount:200},{type:3, Amount:300}] as ExistingOrders,
{type:2, Amount:50} as NewValue
RETURN reduce(map=filter(p in ExistingOrders where not p.type=NewValue.type),x in [(filter(p2 in ExistingOrders where p2.type=NewValue.type)[0])]|CASE x WHEN null THEN NewValue ELSE {type:x.type,Amount:x.Amount+NewValue.Amount} END+map) as CombinedOrders
But I'm struggling I think because of the Orders[array] in my first example.
I believe you are just trying to update the value of the appropriate Amount in ExistingOrders.
The following query is legal Cypher, and should normally work:
WITH ExistingOrders, NewOrder, [x IN ExistingOrders WHERE x.type = NewOrder.type | x.Orders] AS eo
FOREACH (y IN eo |
SET y.Amount = y.Amount + CASE WHEN y.Price = NewOrder.Order.Price THEN NewOrder.Order.Amount ELSE 0 END
)
However, the above query produces a (somewhat) funny ThisShouldNotHappenError error with the message:
Developer: Stefan claims that: This should be a node or a relationship
What the message is trying to say (in obtuse fashion) is that you are not using the neo4j DB in the right way. Your properties are way too complicated, and should be separated out into nodes and relationships.
So, I will a proposed data model that does just that. Here is how you can create nodes and relationships that represent the same data as ExistingOrders:
CREATE (t1:Type {id:1}), (t2:Type {id:2}), (t3:Type {id:3}),
(t1)-[:HAS_ORDER]->(:Order {Price:10,Amount:100}),
(t1)-[:HAS_ORDER]->(:Order {Price:11,Amount:200}),
(t1)-[:HAS_ORDER]->(:Order {Price:12,Amount:300}),
(t2)-[:HAS_ORDER]->(:Order {Price:10,Amount:100}),
(t2)-[:HAS_ORDER]->(:Order {Price:11,Amount:200}),
(t2)-[:HAS_ORDER]->(:Order {Price:12,Amount:300}),
(t3)-[:HAS_ORDER]->(:Order {Price:10,Amount:100}),
(t3)-[:HAS_ORDER]->(:Order {Price:11,Amount:200}),
(t3)-[:HAS_ORDER]->(:Order {Price:12,Amount:300});
And here is a query that will update the correct Amount:
WITH {type:2, Order:{Price:11,Amount:50}} as NewOrder
MATCH (t:Type)-[:HAS_ORDER]->(o:Order)
WHERE t.id = NewOrder.type AND o.Price = NewOrder.Order.Price
SET o.Amount = o.Amount + NewOrder.Order.Amount
RETURN t.id, o.Price, o.Amount;
There's two parts to your question - one with a simple answer, and a second part that doesn't make sense. Let me take the simple one first!
As far as I can tell, it seems you're asking how to concatenate a new map on to a collection of maps. So, how to add a new item in an array. Just use + like this simple example:
return [{item:1}, {item:2}] + [{item:3}];
Note that the single item we're adding at the end isn't a map, but a collection with only one item.
So for your query:
RETURN [
{type:1, Orders:[{Price:10,Amount:100},
{Price:11,Amount:200},
{Price:12,Amount:300}]},
{type:2, Orders:[{Price:10,Amount:100},
{Price:11,Amount:**250**},
{Price:12,Amount:300}]}]
+
[{type:3, Orders:[{Price:10,Amount:100},
{Price:11,Amount:200},{Price:12,Amount:300}]}]
as **CombinedOrders**
Should do the trick.
Or you could maybe do it a bit cleaner, like this:
WITH [{type:1, Orders:[{Price:10,Amount:100},{Price:11,Amount:200},{Price:12,Amount:300}]},
{type:2, Orders:[{Price:10,Amount:100},{Price:11,Amount:200},{Price:12,Amount:300}]},
{type:3, Orders:[{Price:10,Amount:100},{Price:11,Amount:200},{Price:12,Amount:300}]}] as ExistingOrders,
{type:2, Order:{Price:11,Amount:50}} as NewOrder
RETURN ExistingOrders + [NewOrder];
OK now for the part that doesn't make sense. In your example, it looks like you want to modify the map inside of the collection. But you have two {type:2} maps in there, and you're looking to merge them into something with one resulting {type:3} map in the output that you're asking for. If you need to deconflict map entries and change what the map entry ought to be, it might be that cypher isn't your best choice for that kind of query.
I figured it out:
WITH [{type:1, Orders:[{Price:10,Amount:100},{Price:11,Amount:200},Price:12,Amount:300}]},{type:2, Orders:[{Price:10,Amount:100},{Price:11,Amount:200},{Price:12,Amount:300}]},{type:3, Orders:[{Price:10,Amount:100},{Price:11,Amount:200},{Price:12,Amount:300}]}] as ExistingOrders,{type:2, Orders:[{Price:11,Amount:50}]} as NewOrder
RETURN
reduce(map=filter(p in ExistingOrders where not p.type=NewOrder.type),
x in [(filter(p2 in ExistingOrders where p2.type=NewOrder.type)[0])]|
CASE x
WHEN null THEN NewOrder
ELSE {type:x.type, Orders:[
reduce(map2=filter(p3 in x.Orders where not (p3.Price=(NewOrder.Orders[0]).Price)),
x2 in [filter(p4 in x.Orders where p4.Price=(NewOrder.Orders[0]).Price)[0]]|
CASE x2
WHEN null THEN NewOrder.Orders[0]
ELSE {Price:x2.Price, Amount:x2.Amount+(NewOrder.Orders[0]).Amount}
END+map2 )]} END+map) as CombinedOrders
...using nested Reduce functions.
So, to start with it combines a list of orders without matching type, with a list of those orders (actually, just one) with a matching type. For those latter ExistingOrders (with type that matches the NewOrder) it does a similar thing with Price in the nested reduce function and combines non-matching Prices with matching Prices, adding the Amount in the latter case.
In my graph I have data like following way.
Here a,b,c,d are nodes and r1,r2,r3,r4 are relations.
a-r1->b
b-r2->a
b-r2->c
c-r1->b
d-r3->a
a-r1->d like this.
I am using following Cypher to get path with max depth 3.
MATCH p=(n)-[r*1..3]-(m) WHERE n.id=1 and m.id=2 RETURN p
Here return p is path and I want to display path in text format like this.
Example : Suppose Path Lengh is 3.
a-r1->b-r2->c like this in text format.
Is this possible ?
Sort of. I'll give you most of the answer, but I myself can't complete the answer. Maybe another cypher wizard will come along and improve on the answer, but here's what I've got for you.
match p=(n)-[r*1..3]-(m)
WHERE id(n)=1 AND id(m)=2
WITH extract(node in nodes(p) | coalesce(node.label, "")) as nodeLabels,
extract(rel in relationships(p) | type(rel)) as relationshipLabels
WITH reduce(nodePath="", nodeLabel in nodeLabels | nodePath + nodeLabel + "-") as nodePath,
reduce(relPath="", relLabel in relationshipLabels | relPath + relLabel + "-") as relPath
RETURN nodePath, relPath
LIMIT 1;
EDIT - one small note, in your question you specify the WHERE criteria n.id=1 and m.id=2. Note that this is probably not what you want. Node IDs are usually checked with WHERE id(n)=1 AND id(m)=2. Id isn't technically a node property, so I changed that.
OK, so we're going to match the path. Then we're going to use the extract function to pull out the label property from nodes, and create a collection called nodeLabels. We'll do the same for the relationship types. What reduce does here is accumulate each of the individual strings in those collections down to a single string. So if your nodes are a, b, and c, you'd get a nodePath string that looks like a-b-c-. Similarly, your relationship string would look like r1-r2-r3-.
Now, I know you want those interleaved, and you'd prefer output like a-r1-b-r2-c. Here's the problem I see with that...
Normally, the way I'd approach that is to use FOREACH to iterate over the node label collection. Since you know there is one less relationship than nodes because of what paths are, ideally (in pseudo code) I'd want to do something like this:
buffer = ""
foreach x in range(0, length(nodeLabels)) |
buffer = buffer + nodeLabels[idx] + "-" + relLabels[idx] + "->")
This would be a way of reducing to the string that you want. You can't use the reduce function, because it doesn't provide you a way of getting which index you're at in the collection. Meaning that you can iterate over one of the collections, but not at the same time over the other. This FOREACH pseudo code will not work, because the second part of FOREACH I believe has to be a mutating operation on the graph, and you can't just use it to accumulate a string like I did here, or like the extract function does.
So as far as I can tell, you might kinda be stuck here. Hopefully someone will prove me wrong on this - I am not 100% sure.
Finally another way to go after this would be, if there was a path function that extracted node/relationship pairs, rather than just nodes() or relationships() individually as I used them above, then you could use that function to iterate over one collection, rather than shuffling two collections, as my code above attempts and fails to do. Sadly, I don't think there's any such path function, so that's just more reason why I think you might be up a creek.
Now, practically speaking, you could always execute this query in java or some other language, return the path, and then use the full power of whatever programming language you want to build up this string. But pure cypher? I'm doubtful.
Here What I ended up doing. Hope that somebody else find it useful for future.
MATCH p=(n)-[r*1..3]->(m)
WHERE n.id=1 AND m.id=4
WITH extract(rel in relationships(p) | STARTNODE(rel).name + '->' + type(rel)) as relationshipLabels, m.name as endnodename
WITH reduce(relPath="", relLabel in relationshipLabels | relPath + relLabel+ '->') as relPath , end
RETURN distinct relPath + endnodename
I have a record:
-record(bigdata, {mykey,some1,some2}).
Is doing a
mnesia:match_object({bigdata, mykey, some1,'_'})
the fastest way fetching more than 5000 rows?
Clarification:
Creating "custom" keys is an option (so I can do a read) but is doing 5000 reads fastest than match_object on one single key?
I'm curious as to the problem you are solving, how many rows are in the table, etc., without that information this might not be a relevant answer, but...
If you have a bag, then it might be better to use read/2 on the key and then traverse the list of records being returned. It would be best, if possible, to structure your data to avoid selects and match.
In general select/2 is preferred to match_object as it tends to better avoid full table scans. Also, dirty_select is going to be faster then select/2 assuming you do not need transactional support. And, if you can live with the constraints, Mensa allows you to go against the underlying ets table directly which is very fast, but look at the documentation as it is appropriate only in very rarified situations.
Mnesia is more a key-value storage system, and it will traverse all its records for getting match.
To fetch in a fast way, you should design the storage structure to directly support the query. To Make some1 as key or index. Then fetch them by read or index_read.
The statement Fastest Way to return more than 5000 rows depends on the problem in question. What is the database structure ? What do we want ? what is the record structure ? After those, then, it boils down to how you write your read functions. If we are sure about the primary key, then we use mnesia:read/1 or mnesia:read/2 if not, its better and more beautiful to use Query List comprehensions. Its more flexible to search nested records and with complex conditional queries. see usage below:
-include_lib("stdlib/include/qlc.hrl").
-record(bigdata, {mykey,some1,some2}).
%% query list comprehenshions
select(Q)->
%% to prevent against nested transactions
%% to ensure it also works whether table
%% is fragmented or not, we will use
%% mnesia:activity/4
case mnesia:is_transaction() of
false ->
F = fun(QH)-> qlc:e(QH) end,
mnesia:activity(transaction,F,[Q],mnesia_frag);
true -> qlc:e(Q)
end.
%% to read by a given field or even several
%% you use a list comprehension and pass the guards
%% to filter those records accordingly
read_by_field(some2,Value)->
QueryHandle = qlc:q([X || X <- mnesia:table(bigdata),
X#bigdata.some2 == Value]),
select(QueryHandle).
%% selecting by several conditions
read_by_several()->
%% you can pass as many guard expressions
QueryHandle = qlc:q([X || X <- mnesia:table(bigdata),
X#bigdata.some2 =< 300,
X#bigdata.some1 > 50
]),
select(QueryHandle).
%% Its possible to pass a 'fun' which will do the
%% record selection in the query list comprehension
auto_reader(ValidatorFun)->
QueryHandle = qlc:q([X || X <- mnesia:table(bigdata),
ValidatorFun(X) == true]),
select(QueryHandle).
read_using_auto()->
F = fun({bigdata,SomeKey,_,Some2}) -> true;
(_) -> false
end,
auto_reader(F).
So i think if you want fastest way, we need more clarification and problem detail. Speed depends on many factors my dear !