Same key, different values: nested dicts of dicts - grep

Borrowing an MWE from this question, I have a set of nested dicts of dicts:
{
"type": "A"
"a": "aaa",
"payload": {"another":{"dict":"value", "login":"user1"}},
"actor": {"dict":"value", "login":"user2"}
}
{
"type": "B"
"a": "aaa",
"payload": {"another":{"dict":"value", "login":"user3"}},
"actor": {"dict":"value", "login":"user4"}
}
}
{
"type": "A"
"a": "aaa",
"b": "bbb",
"payload": {"another":{"dict":"value", "login":"user5"}},
"actor": {"dict":"value", "login":"user6"}
}
}
{
"type": "A"
"a": "aaa",
"b": "bbb",
"payload": {"login":"user5"},
"actor": {"login":"user6"}
}
}
For dictionaries that have "type":"A", I want to get the username from the payload dict and the username from actor dict. The same username can appear multiple times. I would like to store a txt file with a list of actor (ID1) and a list of payload (ID2) like this:
ID1 ID2
user2 user1
user6 user5
user6 user5
Right now, I have a start:
zgrep "A" | zgrep -o 'login":"[^"]*"' | zgrep -o 'payload":"[^"]*" > usernames_list.txt
But of course this won't work, because I need to find login within the payload dict and login within the actor dict for each dict of type A.
Any thoughts?

I am assuming you have the payload and actor dictionaries for all entries of type A.
Parse out the user name from the payload entries and redirect them
to a file named payload.txt
Parse out the user name from actor entries and redirect them to a
different file named actor.txt
Use paste command to join the entries and output them the way you want it

Related

JMESPath to extract key where values match

Given source which looks like:
{
"Name": "sandbox-config",
"VersionList": {
"version-2": [ "STAGING" ],
"version-1": [ "CURRENT", "NEXT" ],
"version-0": [ "ANCIENT" ]
}
}
I'm looking for a jmespath query which would give me:
{
"Name": "sandbox-config",
"Version": "version-1"
}
where version-1 is the first key where the value array contains "CURRENT".
So, a query like,
{ Name:Name, Version:VersionList.*[?#==`CURRENT`] | [] | [0]}
gives me:
{
"Name": "sandbox-config",
"Version": "CURRENT"
}
which isn't what I'm after. Similarly:
{Name:Name, Version:VersionList.keys(#)}
which gives me:
{
"Name": "sandbox-config",
"Version": [
"version-2",
"version-1",
"version-0"
]
}
Any suggestions? I feel like I'm circling around a solution and not quite getting there.
(Context for this: I'm trying to process the output of aws secretsmanager list-secrets, which has SecretVersionsToStages with ARN values as keys with an array containing "AWSCURRENT".)
https://docs.aws.amazon.com/secretsmanager/latest/apireference/API_DescribeSecret.html#API_DescribeSecret_ResponseSyntax
if you want just get the version number for one secret with stage [AWSCURRENT], I recommend you use describe secret rather than list secret.
And one thing I need to call out is that SecretVersionsToStages with versoin number as keys with an array containing "AWSCURRENT".

Create relationship with multipl values

how can I create an relationship in NEO4J from one node to another node which has multiple vales.
The first node has unique values for the identifier. For example:
Data of the first NodeA:
{
"c": "11037",
"b": 15.4,
"a": 10.0,
"id": 11137100
}
The second NodeB look like this:
{
"text": "some text",
"prio": 1,
"id": 11137100,
"value": 0.1
}
But here we have data which has the same id like here:
{
"text": "some other text",
"prio": 2,
"id": 11137100,
"value": 2.1
}
Now want to create a relationship between both nodes. But if I do things like:
MATCH (p:NodeA),(h:NodeB)
WHERE h.id = p.id
CREATE (p)-[dr:Contains{prio:h.prio}]->(h)
RETURN (dr)
I get multiple relationships. I want one NodeA with two Outputs to NodeB.
How can I do it?
The CREATE statement will create a new node/relationship, irrespective if one already exists.
If the intent is to only create a relationship if one does not already exist, I would suggest you do a pre-filter query first, e.g.
MATCH (p:NodeA), (h:NodeB)
WHERE h.id = p.id AND NOT (p)-[:Contains{prio:h.prio}]->(h)
//continue your query here

Importing data from JSON

I'm facing an issue importing a JSON file using the apoc.load.json procedure.
The expected relationship I'm trying to capture:
University --child--> Class --child--> Student
Output:
Neo.ClientError.Statement.SyntaxError: Variable `value` not defined (line 1, column 8 (offset: 7))
"UNWIND value.university AS university"
Here is the performed command sequence:
CALL apoc.load.json("FILE:///C:/tmp/input.json") YIELD value
UNWIND value.university AS university
UNWIND university.class AS class
UNWIND class.student AS student
MERGE (u:UniversityCategory {name:university.name})
MERGE (c:ClassCategory {name:class.name})
MERGE (s:StudentCategory {instr:student.name})
ON CREATE SET i.ID = instructions.ID
ON CREATE SET i.GPA = instructions.GPA
MERGE (u)-[:CHILD]->(c)
MERGE (c)-[:CHILD]->(s)
Here is the JSON file structure:
{
"university": [{
"name": "universityA",
"class": [{
"name": "class_1",
"student": [{
"name": "student_1",
"ID": "1234",
"GPA": "3.8"
},
{
"name": "student_2",
"ID": "12345",
"GPA": "3.4"
}
]
},
{
"name": "class_2",
"student": [{
"name": "student_3",
"ID": "14",
"GPA": "3.0"
}]
}
]
}]
}
My apoc.load.json command appears to work because I see the structured JSON file in the browser window. The next steps are suspect but I think I'm close to defining the relationships.
Resolved my issue.
The commands I expressed above are all correct but the load.apoc.json command has to be run in the SAME query as the rest versus sequentially.

Rails PSQL query JSON for nested array and objects

So I have a json (in text field) and I'm using postgresql and I need to query the field but it's nested a bit deep. Here's the format:
[
{
"name":"First Things",
"items":[
{
"name":"Foo Bar Item 1",
"price":"10.00"
},
{
"name":"Foo Item 2",
"price":"20.00"
}
]
},
{
"name":"Second Things",
"items": [
{
"name":"Bar Item 3",
"price":"15.00"
}
]
}
]
And I need to query the name INSIDE the items node. I have tried some queries but to no avail, like:
.where('this_json::JSON #> [{"items": [{"name": ?}]}]', "%#{name}%"). How should I go about here?
I can query normal JSON format like this_json::JSON -> 'key' = ? but need help with this bit.
Here you need to use json_array_elements() twice, as your top level document contains array of json, than items key has array of sub documents. Sample psql query may be the following:
SELECT
item->>'name' AS item_name,
item->>'price' AS item_price
FROM t,
json_array_elements(t.v) js_val,
json_array_elements(js_val->'items') item;
where t - is the name of your table, v - name of your JSON column.

Emit Tuples From Erlang Views In CouchDB

CouchDB, version 0.10.0, using native erlang views.
I have a simple document of the form:
{
"_id": "user-1",
"_rev": "1-9ccf63b66b62d15d75daa211c5a7fb0d",
"type": "user",
"identifiers": [
"ABC",
"DEF",
"123"
],
"username": "monkey",
"name": "Monkey Man"
}
And a basic javascript design document:
{
"_id": "_design/user",
"_rev": "1-94bd8a0dbce5e2efd699d17acea1db0b",
"language": "javascript",
"views": {
"find_by_identifier": {
"map": "function(doc) {
if (doc.type == 'user') {
doc.identifiers.forEach(function(identifier) {
emit(identifier, {\"username\":doc.username,\"name\":doc.name});
});
}
}"
}
}
}
which emits:
{"total_rows":3,"offset":0,"rows":[
{"id":"user-1","key":"ABC","value":{"username":"monkey","name":"Monkey Man"}},
{"id":"user-1","key":"DEF","value":{"username":"monkey","name":"Monkey Man"}},
{"id":"user-1","key":"123","value":{"username":"monkey","name":"Monkey Man"}}
]}
I'm looking into building an Erlang view that does the same thing. Best attempt so far is:
%% Map Function
fun({Doc}) ->
case proplists:get_value(<<"type">>, Doc) of
undefined ->
ok;
Type ->
Identifiers = proplists:get_value(<<"identifiers">>, Doc),
ID = proplists:get_value(<<"_id">>, Doc),
Username = proplists:get_value(<<"username">>, Doc),
Name = proplists:get_value(<<"name">>, Doc),
lists:foreach(fun(Identifier) -> Emit(Identifier, [ID, Username, Name]) end, Identifiers);
_ ->
ok
end
end.
which emits:
{"total_rows":3,"offset":0,"rows":[
{"id":"user-1","key":"ABC","value":["monkey","Monkey Man"]},
{"id":"user-1","key":"DEF","value":["monkey","Monkey Man"]},
{"id":"user-1","key":"123","value":["monkey","Monkey Man"]}
]}
The question is - how can I get those values out as tuples, instead of as arrays? I don't imagine I can (or would want to) use records, but using atoms in a tuple doesn't seem to work.
lists:foreach(fun(Identifier) -> Emit(Identifier, {id, ID, username, Username, name, Name}) end, Identifiers);
Fails with the following error:
{"error":"json_encode","reason":"{bad_term,{<<\"user-1\">>,<<\"monkey\">>,<<\"Monkey Man\">>}}"}
Thoughts? I know that Erlang sucks for this specific kind of thing (named access) and that I can do it by convention (id at first position, username next, real name last), but that makes the client side code pretty ugly.
The JSON object {"foo":"bar","baz":1} is {[{<<"foo">>,<<"bar">>},{<<"baz">>,1}]}
In Erlang lingua it is a proplist wrapped in a tuple.
It's not pretty, but very efficient :)
To get a feel for it you can play with the JSON lib that ships with CouchDB:
Start CouchDB with the -i
(interactive) flag
On the resulting erlang shell, type: couch_util:json_decode(<<"{\"foo\":\"bar\"}">>).
Profit
// in later versions of CouchDB, this is ejson:decode()
For test_suite_reports bd, that has tests field:
[
{
"name": "basics",
"status": "success",
"duration": 21795
},
{
"name": "all_docs",
"status": "success",
"duration": 385
} ...
I have wrote this to get name and status:
fun({Doc}) ->
Name = fun(L) -> proplists:get_value(<<"name">>, L, null) end,
Status = fun(L) -> proplists:get_value(<<"status">>, L, null) end,
Tests = proplists:get_value(<<"tests">>, Doc, null),
lists:foreach(fun({L}) -> Emit(Name(L), Status(L)) end, Tests)
end.
If you like experimental features (that still work...), you might want to have a look to Erlang exprecs.
I found it extremely helpful in creating a sort of dynamic records for Erlang.

Resources