I have articles data indexed to elastic as follows.
{
"id": 1011,
"title": "abcd",
"author": "author1"
"status": "published"
}
Now I wanted to get all the article id grouped by status.
Result should someway look like this
{
"published": [1011, 1012, ....],
"draft": [2011],
"deleted": [3011]
}
NB: I tried normal aggs (Article.search('*',aggs: [:status], load: false).aggs) , it just giving me the count of each items in, I want ids in each item instead
#Crazy Cat
You can transform you query in this way:
sort(Inc/Dec order) your response from ES over field "status".
Only Ask ES query to return only ID Field and status.
Now the usage of sorting would be it would sort your response to like this: [1st N results of "deleted" status, then N+1 to M results to "draft" and then M+1 to K results to "published"].
Now the advantages of this technique:
You will get flagged ids field of every document over which you can apply operations in you application.
Your query would be light weight as compared to Aggs query.
This way you would also get the metdata of every document ike docId of that document.
Now the Disadvantages:
You would always have to give a high upper bound of your page size, but You can also play around with count coming in the metadata.
Might take a bit more of network size as it returns redundant status in every document.
I Hope this redesign of your query might be helpful to you.
I would like to create a set of nodes and relationships from a JSON document. Here is sample JSON:
{"records": [{
"type": "bundle",
"id": "bundle--1",
"objects": [
{
"type": "evaluation",
"id": "evaluation--12345",
"name": "Eval 1",
"goals": [
"test"
]
},
{
"type": "subject",
"id": "subject--67890",
"name": "Eval 2",
"goals": [
"execute"
]
},
{
"type": "relationship",
"id": "relationship--45678",
"relationship_type": "participated-in",
"source_ref": "subject--67890",
"target_ref": "evaluation--12345"
}
}]
}
And I would like that JSON to be represented in Neo similar to the following:
(:evaluation {properties})<-[:RELATIONSHIP]-(:subject {properties})
Ultimately I would like to have a model that represents the evaluation, subject, and relationship generated via a few cypher calls with as little outside manipulation as possible. Is it possible to use the apoc.create.* set of calls to create the necessary nodes and relationships from this JSON? I have tried something similar to the following to get this JSON to load and I can get it to create nodes of an arbitrary, in this case "object", type.
WITH "file:///C:/path/to/my/sample.json" AS json
CALL apoc.load.json(json, "$.records") YIELD value
UNWIND value.objects as object
MERGE (o:object {id: object.id, type: object.type, name: object.name})
RETURN count(*)
I have tried changing the JSONPath expression to filter different record types but it is difficult to run a Goessner path like $.records..objects[?(#.type = 'subject')] thanks to the embedded quotes. This would also lead to multiple runs (I have 15 or so different types) against the real JSON, which could be very time consuming. The LoadJSON docs have a simple filter expression and there is a blog post that shows how to parse stackoverflow but the JSON objects are keyed in a way that is easy to map in cypher. Is there a cypher trick or APOC I should be aware of that can help me solve this problem?
I would approach this as a two-pass method:
First pass: create the nodes for evaluation and subject. You could use apoc.do.case/when if helpful
Second pass: only scan for relationship and then do a MATCH to find the evaluation and subject nodes based on the source_ref and target_ref, and then MERGE or CREATE the relationship to connect them.
Like this you're not impacted by situations such as the relationship coming before the nodes it connects etc. or how many items you've got within objects
As Lju pointed out, the apoc.do.case function can be used to create a set of conditions to check, followed by a cypher statement. Combining that with another apoc call requires the returns from each apoc call to be handled properly. My answer ended up looking like the following:
WITH "file:///C:/path/to/my/sample.json" AS json
CALL apoc.load.json(json, "$.records") YIELD value as records
UNWIND records.objects as object
CALL apoc.do.case(
[object.type="evaluation", "MERGE (:Evaluation {id: object.id}) ON CREATE SET Evaluation.id = object.id, Evaluation.prop1 = object.prop1",
object.type="subject", "MERGE (:Subject {id: object.id}) ON CREATE SET Subject.id = object.id, Subject.prop1 = object.prop1",
....]
"<default case cypher query goes here>", {object:object}
)
YIELD value RETURN count(*)
Notice there are two apoc calls that YIELD. Use aliases to help the parser differentiate between objects. The documentation for the apoc.do.case is a little sparse but describes the syntax for the statement. It looks like there are other ways to accomplish this task but with smaller JSON files, and a handful of cases, this works well enough.
I am queuing and getting a report through the API and javascript, but now I want to start filtering the report. I want the results that come back to apply only to the user (other filters are needed too) who is requesting the report. What is the best way to put a filter on the initial report queue?
The way I am doing it now is adding a selected element to the report description:
...
"elements": [
{ "id": "page" },{ "id": "evar23" , "selected": ["295424","306313"]}
...
But this only seems to apply to the breakdown section of the results, not the top level count that is returned. I would expect the top level count in the below example be 66, not 68:
...
"counts":[
"68"
],
"breakdown":[
{
"name":"306313",
"url":"",
"counts":[
"43"
]
},
{
"name":"295424",
"url":"",
"counts":[
"23"
]
}
]
}
,...
I know I can just crawl through the breakdown array and total up what I need, but the more filters I apply the messier it becomes. All of a sudden I am three levels deep in a nested array, making sure that all 3 breakdown names match my conditions. There must be a better way to do this, any ideas? Many thanks.
Although there are some possible limitations to them that I am still working through, it seems that segments is what I need, not elements.
"segments": [
{
"element": "evar23","selected": ["295424","306313"]
}]
https://marketing.adobe.com/developer/forum/reporting/report-filtering-with-api
I'm currently using the example data on console.neo4j.org to write a query that outputs hierarchical JSON.
The example data is created with
create (Neo:Crew {name:'Neo'}), (Morpheus:Crew {name: 'Morpheus'}), (Trinity:Crew {name: 'Trinity'}), (Cypher:Crew:Matrix {name: 'Cypher'}), (Smith:Matrix {name: 'Agent Smith'}), (Architect:Matrix {name:'The Architect'}),
(Neo)-[:KNOWS]->(Morpheus), (Neo)-[:LOVES]->(Trinity), (Morpheus)-[:KNOWS]->(Trinity),
(Morpheus)-[:KNOWS]->(Cypher), (Cypher)-[:KNOWS]->(Smith), (Smith)-[:CODED_BY]->(Architect)
The ideal output is as follows
name:"Neo"
children: [
{
name: "Morpheus",
children: [
{name: "Trinity", children: []}
{name: "Cypher", children: [
{name: "Agent Smith", children: []}
]}
]
}
]
}
Right now, I'm using the following query
MATCH p =(:Crew { name: "Neo" })-[q:KNOWS*0..]-m
RETURN extract(n IN nodes(p)| n)
and getting this
[(0:Crew {name:"Neo"})]
[(0:Crew {name:"Neo"}), (1:Crew {name:"Morpheus"})]
[(0:Crew {name:"Neo"}), (1:Crew {name:"Morpheus"}), (2:Crew {name:"Trinity"})]
[(0:Crew {name:"Neo"}), (1:Crew {name:"Morpheus"}), (3:Crew:Matrix {name:"Cypher"})]
[(0:Crew {name:"Neo"}), (1:Crew {name:"Morpheus"}), (3:Crew:Matrix {name:"Cypher"}), (4:Matrix {name:"Agent Smith"})]
Any tips to figure this out? Thanks
In neo4j 3.x, after you install the APOC plugin on the neo4j server, you can call the apoc.convert.toTree procedure to generate similar results.
For example:
MATCH p=(n:Crew {name:'Neo'})-[:KNOWS*]->(m)
WITH COLLECT(p) AS ps
CALL apoc.convert.toTree(ps) yield value
RETURN value;
... would return a result row that looks like this:
{
"_id": 127,
"_type": "Crew",
"name": "Neo",
"knows": [
{
"_id": 128,
"_type": "Crew",
"name": "Morpheus",
"knows": [
{
"_id": 129,
"_type": "Crew",
"name": "Trinity"
},
{
"_id": 130,
"_type": "Crew:Matrix",
"name": "Cypher",
"knows": [
{
"_id": 131,
"_type": "Matrix",
"name": "Agent Smith"
}
]
}
]
}
]
}
This was such a useful thread on this important topic, I thought I'd add a few thoughts after digging into this a bit further.
First off, using the APOC "toTree" proc has some limits, or better said, dependencies. It really matters how "tree-like" your architecture is. E.g., the LOVES relation is missing in the APOC call above and I understand why – that relationship is hard to include when using "toTree" – that simple addition is a bit like adding an attribute in a hierarchy, but as a relationship. Not bad to do but confounds the simple KNOWS tree. Point being, a good question to ask is “how do I handle such challenges”. This reply is about that.
I do recommend upping ones JSON skills as this will give you much more granular control. Personally, I found my initial exploration somewhat painful. Might be because I'm an XML person :) but once you figure out all the [, {, and ('s, it is really a powerful way to efficiently pull what's best described as a report on your data. And given the JSON is something that can easily become a class, it allows for a nice way to push that back to your app.
I have found perf to also be a challenge with "toTree" vs. just asking for the JSON. I've added below a very simplistic look into what your RETURN could look like. It follows the following BN format. I'd love to see this more maturely created as the possibilities are quite varied, but this was something I'd have found useful thus I’ll post this immature version for now. As they say; “a deeper dive is left up to the readers” 😊
I've obfuscated the values, but this is an actual query on what I’ll term a very poor example of a graph architecture, whose many design “mistakes” cause some significant performance headaches when trying to access a holistic report on the graph. As in this example, the initial report query I inherited took many minutes on a server, and could not run on my laptop - using this strategy, the updated query now runs in about 5 seconds on my rather wimpy laptop on a db of about 200K nodes and .5M relationships. I added the “persons” grouping alias as a reminder that "persons" will be different in each array element, but the parent construct will be repeated over and over again. Where you put that in your hand-grown tree, will matter, but having the ability to do that is powerful.
Bottom line, a mature use of JSON in the RETURN statement, gives you a powerful control over the results in a Cypher query.
RETURN STATEMENT CONTENT:
<cypher_alias>
{.<cypher_alias_attribute>,
...,
<grouping_alias>:
(<cypher_alias>
{.<cypher_alias_attribute,
...
}
)
...
}
MATCH (j:J{uuid:'abcdef'})-[:J_S]->(s:S)<-[:N_S]-(n:N)-[:N_I]->(i:I), (j)-[:J_A]->(a:P)
WHERE i.title IN ['title1', 'title2']
WITH a,j, s, i, collect(n.description) as desc
RETURN j{.title,persons:(a{.email,.name}), s_i_note:
(s{.title, i_notes:(i{.title,desc})})}
if you know how deep your tree is, you can write something like this
MATCH p =(:Crew { name: "Neo" })-[q:KNOWS*0..]-(m)
WITH nodes(p)[0] AS a, nodes(p)[1] AS b, nodes(p)[2] AS c, nodes(p)[3] AS d, nodes(p)[4] AS e
WITH (a{.name}) AS ab, (b{.name}) AS bb, (c{.name}) AS cb, (d{.name}) AS db, (e{.name}) AS eb
WITH ab, bb, cb, db{.*,children:COLLECT(eb)} AS ra
WITH ab, bb, cb{.*,children:COLLECT(ra)} AS rb
WITH ab, bb{.*,children:COLLECT(rb)} AS rc
WITH ab{.*,children:COLLECT(rc)} AS rd
RETURN rd
Line 1 is your query. You save all paths from Neo to m in p.
In line 2 p is split into a, b, c, d and e.
Line 3 takes just the namens of the nodes. If you want all properties you can write (a{.*}) AS ab. This step is optional you can also work with nodes if you want to.
In line 4 you replace db and eb with a map containing all properties of db and the new property children containing all entries of eb for the same db.
Lines 5, 6 and 7 are basically the same. You reduce the result list by grouping.
Finally you return the tree. It looks like this:
{
"name": "Neo",
"children": [
{
"name": "Morpheus",
"children": [
{"name": "Trinity", "children": []},
{"name": "Cypher","children": [
{"name": "Agent Smith","children": []}
]
}
]
}
]
}
Unfortunately this solution only works when you know how deep your tree is and you have to add a row if your tree is one step deeper.
If someone has an idea how to solve this with dynamic tree depth, please comment.
I have merged two Fusion Tables together on the key "PID". Now I would like to do a SELECT query WHERE PID = "value'. The error comes back that no column with the name PID exists in the table. A query for another column gives this result:
"kind": "fusiontables#sqlresponse",
"columns": [
"\ufeffPID",
"Address",
"City",
"Zoning"
],
"rows": [
[
"001-374-079",
"# LOT 15 MYSTERY BEACH RD",
"No_City_Value",
"R-1"
],
It appears that the column name has been changed from "PID" to "\ufeffPID", which no matter how many attempts to get the syntax to read a GET Url, I keep getting an error.
Is there any limitation with querying on the key of a merged table? Since I cannot seem to get the name correct for the column a work around would be to use the Column ID but that does not seem to be an option either. Here is the URL:
https://www.googleapis.com/fusiontables/v1/query?sql=SELECT 'PID','Address','City','Zoning' FROM 1JanYNl3T45kFFxqAmGS0BRgkopj4AS207qnLVQI WHERE '\ufeffPID' = 001-493-078&key=myKey
Cheers
I have no explanation for \ufeff in there; that's the Unicode character 'ZERO WIDTH NO-BREAK SPACE', so it's conceivable that it's actually there in the column name because it would be invisible in the UI. So, first off I would recommend changing the name in the base tables and see if that works.
Column IDs for merge tables have a different form than for base tables. An easy way to get them is to add the filters of interest to one of your tabs (any type will do) and then do Tools > Publish. The top text ("Send a link in email or IM") has a query URL that has what you need. Run it through a URL decoder such as http://meyerweb.com/eric/tools/dencoder/ and you'll see the column ID for PID is col0>>0.
Rod