I am queuing and getting a report through the API and javascript, but now I want to start filtering the report. I want the results that come back to apply only to the user (other filters are needed too) who is requesting the report. What is the best way to put a filter on the initial report queue?
The way I am doing it now is adding a selected element to the report description:
...
"elements": [
{ "id": "page" },{ "id": "evar23" , "selected": ["295424","306313"]}
...
But this only seems to apply to the breakdown section of the results, not the top level count that is returned. I would expect the top level count in the below example be 66, not 68:
...
"counts":[
"68"
],
"breakdown":[
{
"name":"306313",
"url":"",
"counts":[
"43"
]
},
{
"name":"295424",
"url":"",
"counts":[
"23"
]
}
]
}
,...
I know I can just crawl through the breakdown array and total up what I need, but the more filters I apply the messier it becomes. All of a sudden I am three levels deep in a nested array, making sure that all 3 breakdown names match my conditions. There must be a better way to do this, any ideas? Many thanks.
Although there are some possible limitations to them that I am still working through, it seems that segments is what I need, not elements.
"segments": [
{
"element": "evar23","selected": ["295424","306313"]
}]
https://marketing.adobe.com/developer/forum/reporting/report-filtering-with-api
Related
I have articles data indexed to elastic as follows.
{
"id": 1011,
"title": "abcd",
"author": "author1"
"status": "published"
}
Now I wanted to get all the article id grouped by status.
Result should someway look like this
{
"published": [1011, 1012, ....],
"draft": [2011],
"deleted": [3011]
}
NB: I tried normal aggs (Article.search('*',aggs: [:status], load: false).aggs) , it just giving me the count of each items in, I want ids in each item instead
#Crazy Cat
You can transform you query in this way:
sort(Inc/Dec order) your response from ES over field "status".
Only Ask ES query to return only ID Field and status.
Now the usage of sorting would be it would sort your response to like this: [1st N results of "deleted" status, then N+1 to M results to "draft" and then M+1 to K results to "published"].
Now the advantages of this technique:
You will get flagged ids field of every document over which you can apply operations in you application.
Your query would be light weight as compared to Aggs query.
This way you would also get the metdata of every document ike docId of that document.
Now the Disadvantages:
You would always have to give a high upper bound of your page size, but You can also play around with count coming in the metadata.
Might take a bit more of network size as it returns redundant status in every document.
I Hope this redesign of your query might be helpful to you.
new to neo4j, I want to load a JSON with the following structure into my neo4j DB:
{
"nodes": [
{
"last_update": 1629022369,
"pub_key": "pub1",
"alias": "alias1"
},
{
"last_update": 1618162974,
"pub_key": "pub2",
"alias": "alias2"
},
{
"last_update": 1634745976,
"pub_key": "pub3",
"alias": "alias3"
}
],
"edges": [
{
"node1_pub": "pub1",
"node2_pub": "pub2",
"capacity": "37200"
},
{
"node1_pub": "pub2",
"node2_pub": "pub3",
"capacity": "37200"
},
{
"node1_pub": "pub3",
"node2_pub": "pub1",
"capacity": "37200"
}
]
}
I load nodes and edges in separate queries:
WITH "file:///graph.json" AS graph
CALL apoc.load.json(graph) YIELD value
FOREACH (nodeObject in value.nodes | CREATE (node:Node {pubKey:nodeObject.pub_key}))
WITH "file:///graph.json" AS graph
CALL apoc.load.json(graph) YIELD value
UNWIND value.edges as edgeObject
MATCH (node1:Node {pubKey: edgeObject.node1_pub})
MATCH (node2:Node {pubKey: edgeObject.node2_pub})
CREATE (node1)-[:IS_CONNECTED {capacity: edgeObject.capacity}]->(node2)
This works fine with a small number of edges, but I have a ~100mb file with plenty of edges in there. In the latter case, the query does not return. I'm running it from the neo4j web interface. neo4j is running in docker and the max heap size is set to 3g, which should be more than enough.
I have not grasped all of the concepts of Cypher, so probably there is some better way to do it anyways. Also maybe in one query, so that the file does not need to be loaded twice.
Thanks a lot!
You can load the json file by batch using txBatchSize parameter. See the documentation below:
https://neo4j.com/labs/apoc/4.1/import/load-json/#load-json-available-procedures-apoc.import.json
WITH "file:///graph.json" as graph
CALL apoc.load.json(graph, '[0:10000]') YIELD value
RETURN value
Where it will return 10000 rows.
okay, after trying out the batching suggested by #jose_bacoy, I saw that even 1000 rows took around 20s.
Obviously, the MATCH operation is quite CPU intensive. After I created an index the import of 80k edges worked like a charm.
CREATE INDEX FOR (n:Node) ON (n.pubKey)
In a question on CouchDB I asked previously (Can you implement document joins using CouchDB 2.0 'Mango'?), the answer mentioned creating domain objects instead of storing relational data in Couch.
My use case, however, is not necessarily to store relational data in Couch but to flatten relational data. For example, I have the entity of Invoice that I collect from several suppliers. So I have two different schemas for that entity.
So I might end up with 2 docs in Couch that look like this:
{
"type": "Invoice",
"subType": "supplier B",
"total": 22.5,
"date": "10 Jan 2017",
"customerName": "me"
}
{
"type": "Invoice",
"subType": "supplier A",
"InvoiceTotal": 10.2,
"OrderDate": <some other date format>,
"customerName": "me"
}
I also have a doc like this:
{
"type": "Customer",
"name": "me",
"details": "etc..."
}
My intention then is to 'flatten' the Invoice entities, and then join on the reduce function. So, the map function looks like this:
function(doc) {
switch(doc.type) {
case 'Customer':
emit(doc.customerName, { doc information ..., type: "Customer" });
break;
case 'Invoice':
switch (doc.subType) {
case 'supplier B':
emit (doc.customerName, { total: doc.total, date: doc.date, type: "Invoice"});
break;
case 'supplier A':
emit (doc.customerName, { total: doc.InvoiceTotal, date: doc.OrderDate, type: "Invoice"});
break;
}
break;
}
}
Then I would use the reduce function to compare docs with the same customerName (i.e. a join).
Is this advisable using CouchDB? If not, why?
First of all apologizes for getting back to you late, I thought I'd look at it directly but I haven't been on SO since we exchanged the other day.
Reduce functions should only be used to reduce scalar values, not to aggregate data. So you wouldn't use them to achieve things such as doing joins, or removing duplicates, but you would for example use them to compute the number of invoices per customer - you see the idea. The reason is you can only make weak assumptions with regards to the calls made to your reduce functions (order in which records are passed, rereduce parameter, etc...) so you can easily end up with serious performance problems.
But this is by design since the intended usage of reduce functions is to reduce scalar values. An easy way to think about it is to say that no filtering should ever happen in a reduce function, filtering and things such as checking keys should be done in map.
If you just want to compare docs with the same customer name you do not need a reduce function at all, you can query your view the following parameters:
startkey=["customerName"]
endkey=["customerName", {}]
Otherwise you may want to create a separate view to filter on customers first, and return their names and then use these names to query your view in a bulk manner using the keys view parameter. Startkey/endkey is good if you only want to filter one customer at a time, and/or need to match complex keys in a partial way.
If what you are after are the numbers, you may want to do :
if(doc.type == "Invoice") {
emit([doc.customerName, doc.supplierName, doc.date], doc.amount)
}
And then use the _stats built-in reduce function to get statistics on the amount (sum, min, max,)
So that to get the amount spent with a supplier, you'd just need to make a reduce query to your view, and use the parameter group_level=2 to aggregate by the first 2 elements of the key. You can combine this with startkey and endkey to filter specific values of this key :
startkey=["name1", "supplierA"]
endkey=["name1", "supplierA", {}]
You can then build from this example to do things such as :
if(doc.type == "Invoice") {
emit(["BY_DATE", doc.customerName, doc.date], doc.amount);
emit(["BY_SUPPLIER", doc.customerName, doc.supplierName], doc.amount);
emit(["BY_SUPPLIER_AND_DATE", doc.customerName, doc.supplierName, doc.date], doc.amount);
}
Hope this helps
It is totally ok to "normalize" your different schemas (or subTypes) via a view. You cannot create views based on those normalized schemas, though, and on the long run, it might be hard to manage different schemas.
The better solution might be to normalize the documents before writing them to CouchDB. If you still need the documents in their original schema, you can add a sub-property original where you store your documents in their original form. This would make working on data much easier:
{
"type": "Invoice",
"total": 22.5,
"date": "2017-01-10T00:00:00.000Z",
"customerName": "me",
"original": {
"supplier": "supplier B",
"total": 22.5,
"date": "10 Jan 2017",
"customerName": "me"
}
},
{
"type": "Invoice",
"total": 10.2,
"date": "2017-01-12T00:00:00:00.000Z,
"customerName": "me",
"original": {
"subType": "supplier A",
"InvoiceTotal": 10.2,
"OrderDate": <some other date format>,
"customerName": "me"
}
}
I d' also convert the date to ISO format because it parses well with new Date(), sorts correctly and is human-readable. You can easily emit invoices grouped by year, month, day and whatever with that.
Use reduce preferably only with built-in functions, because reduces have to be re-executed on queries, and executing JavaScript on many documents is a complex and time-intensive operation, even if the database has not changed at all. You find more information about the reduce process in the CouchDB process. It makes more sense to preprocess the documents as much as you can before storing them in CouchDB.
I am trying to use JIRA REST API[1] to list all the groups in JIRA. I am currently using JIRA version 6.01 .
I tried /rest/api/2/groups/picker[2] in JIRA REST API 6.01 but could not find a way to specify the parameter "query" as the way I need.
If I use a whole group name in parameter "query", I receive the correct group like this.
Request 1:
GET /jira/rest/api/2/groups/picker?query=jira-users
Response 1
{
"header": "Showing 1 of 1 matching groups",
"total": 1,
"groups": [ {
"name": "jira-users",
"html": "<b>jira-users<\/b>"
}]
}
But if I use a part of the group name in "query" parameter, it does not give expected results.
Request 2
GET /jira/rest/api/2/groups/picker?query=j
According to the method spec [2] I hope to receive all groups that name contains "j" but I do not receive any result.
Response 2
{
"header": "Showing 0 of 0 matching groups",
"total": 0,
"groups": []
}
Can anyone please let me know the right way to give parameters?
Thank you
[1] https://developer.atlassian.com/static/rest/jira/6.0.1.html
[2] https://developer.atlassian.com/static/rest/jira/6.0.1.html#id150432
We're using JIRA 6.0.7 and can do:
/rest/api/2/groups/picker?maxResults=10000
Which will show you all groups up to a max of 10000 results. The the response is important part as it shows the total number of groups, this may require you to adjust the maxResults query parameter that you pass to it if you have too small of a value to show all results:
{
"header":"Showing 5014 of 5014 matching groups",
"total":5014,
"groups":{
...
}
}
If you omit the maxResults it just returns the first 20 out of 5014. However, for us doing:
/rest/api/2/groups/picker?query=j
Will result in all groups containing the letter j to show up. Maybe it wasn't properly implemented in your version. If you are unable to get the query part working properly, you could try and get all results and then do your own filter by analyzing the name for each group object returned.
I am building a site with a database of users. I am using arbor.js to build a graph for each user. The graph is a tree-like structure with edges and nodes that looks something like this (I had an image ready to go but apparently don't have enough reputation yet):
vehicle
/ \
/ \
car truck
/
/
sedan
and is represented by the following JSON:
{
"nodes":{
"vehicle":{
"color":"black",
"label":"vehicle"
},
"car":{
"color":"orange",
"label":"car"
},
"truck":{
"color":"red",
"label":"truck"
},
"sedan":{
"color":"red",
"label":"sedan"
}
},
"edges":{
"vehicle":{
"car":{
"weight":5,
"directed":true,
"color":"orange"
},
"truck":{
"weight":5,
"directed":true,
"color":"red"
}
},
"car":{
"sedan":{
"weight":2,
"directed":true,
"color":"orange"
}
}
}
}
Each graph will always have a nodes and edges object with dynamic nodes and edges. Their respective attributes (color, label, weight etc.) will be fixed.
I am trying to figure out how best to model this data for each user. I am using Rails with MongoDB (Mongoid), because I understand that MongoDB can save objects as documents in the database. I'm pretty sure each user will have a graph model which I can define, but beyond that I'm not sure how to handle the nodes and edges.
I'm guessing the solution will involve has_many, embeds_many, or possibly serialize, but I'm unclear on how to use these with a mix of fixed and dynamic data.
Also, it would be nice to retrieve the data exactly the way it looks above so I can easily create the graph when loading it from disk.
Any help would be appreciated!
In case all you need is to perform graph operations only per user. You can follow this model.
{
"nodes": [{"type": "vehicle", "color":"black", "label": "vehicle"},
{"type": "car", "color":"orange", "label": "car"},
{"type":"truck", "color":"red", "label":"truck"},
{"type": "sedan", "color":"red", "label":"sedan"}
],
"edges": {
"vehicle": [
{"type": "car", "weight": 5, "color": "orange"},
{"type": "truck", "weight": 5, "color": "red"}
],
"car": [
{"type": "sedan", "weight": 2, "color": "orange"}
],
"sedan": [],
"truck":: []
}
It is like you are storing a multimap for edges. Also it is self suggestive whether its a bi-directional or not. For individual user's graph to be processed independently, it is a pretty natural model you can go with.
Tell me if it meets your requirement. Also, until you specify what kind of queries you want to perform over your collection, its not possible to suggest a model.
Also if you are starting your project you can explore some graph databases as well like neo4j