Emit Tuples From Erlang Views In CouchDB - erlang

CouchDB, version 0.10.0, using native erlang views.
I have a simple document of the form:
{
"_id": "user-1",
"_rev": "1-9ccf63b66b62d15d75daa211c5a7fb0d",
"type": "user",
"identifiers": [
"ABC",
"DEF",
"123"
],
"username": "monkey",
"name": "Monkey Man"
}
And a basic javascript design document:
{
"_id": "_design/user",
"_rev": "1-94bd8a0dbce5e2efd699d17acea1db0b",
"language": "javascript",
"views": {
"find_by_identifier": {
"map": "function(doc) {
if (doc.type == 'user') {
doc.identifiers.forEach(function(identifier) {
emit(identifier, {\"username\":doc.username,\"name\":doc.name});
});
}
}"
}
}
}
which emits:
{"total_rows":3,"offset":0,"rows":[
{"id":"user-1","key":"ABC","value":{"username":"monkey","name":"Monkey Man"}},
{"id":"user-1","key":"DEF","value":{"username":"monkey","name":"Monkey Man"}},
{"id":"user-1","key":"123","value":{"username":"monkey","name":"Monkey Man"}}
]}
I'm looking into building an Erlang view that does the same thing. Best attempt so far is:
%% Map Function
fun({Doc}) ->
case proplists:get_value(<<"type">>, Doc) of
undefined ->
ok;
Type ->
Identifiers = proplists:get_value(<<"identifiers">>, Doc),
ID = proplists:get_value(<<"_id">>, Doc),
Username = proplists:get_value(<<"username">>, Doc),
Name = proplists:get_value(<<"name">>, Doc),
lists:foreach(fun(Identifier) -> Emit(Identifier, [ID, Username, Name]) end, Identifiers);
_ ->
ok
end
end.
which emits:
{"total_rows":3,"offset":0,"rows":[
{"id":"user-1","key":"ABC","value":["monkey","Monkey Man"]},
{"id":"user-1","key":"DEF","value":["monkey","Monkey Man"]},
{"id":"user-1","key":"123","value":["monkey","Monkey Man"]}
]}
The question is - how can I get those values out as tuples, instead of as arrays? I don't imagine I can (or would want to) use records, but using atoms in a tuple doesn't seem to work.
lists:foreach(fun(Identifier) -> Emit(Identifier, {id, ID, username, Username, name, Name}) end, Identifiers);
Fails with the following error:
{"error":"json_encode","reason":"{bad_term,{<<\"user-1\">>,<<\"monkey\">>,<<\"Monkey Man\">>}}"}
Thoughts? I know that Erlang sucks for this specific kind of thing (named access) and that I can do it by convention (id at first position, username next, real name last), but that makes the client side code pretty ugly.

The JSON object {"foo":"bar","baz":1} is {[{<<"foo">>,<<"bar">>},{<<"baz">>,1}]}
In Erlang lingua it is a proplist wrapped in a tuple.
It's not pretty, but very efficient :)
To get a feel for it you can play with the JSON lib that ships with CouchDB:
Start CouchDB with the -i
(interactive) flag
On the resulting erlang shell, type: couch_util:json_decode(<<"{\"foo\":\"bar\"}">>).
Profit
// in later versions of CouchDB, this is ejson:decode()

For test_suite_reports bd, that has tests field:
[
{
"name": "basics",
"status": "success",
"duration": 21795
},
{
"name": "all_docs",
"status": "success",
"duration": 385
} ...
I have wrote this to get name and status:
fun({Doc}) ->
Name = fun(L) -> proplists:get_value(<<"name">>, L, null) end,
Status = fun(L) -> proplists:get_value(<<"status">>, L, null) end,
Tests = proplists:get_value(<<"tests">>, Doc, null),
lists:foreach(fun({L}) -> Emit(Name(L), Status(L)) end, Tests)
end.

If you like experimental features (that still work...), you might want to have a look to Erlang exprecs.
I found it extremely helpful in creating a sort of dynamic records for Erlang.

Related

Returning Neo4j map projection with WebFlux

I have the nodes user and game with some relationships between them.
My REST API should return all relationships between the games and 1 user.
The cypher query i use is:
MATCH (u:User {id: '1234'} ) -[rel]- (game:Game) return game{.*, relationships: collect(DISTINCT rel)}
In my Neo4j Browser, everything works as expected and i see all properties i need.
But the GetMapping retuns everything except the relationship properties.
Neo4j Browser
{
"relationships": [
{
"identity": 54,
"start": 9,
"end": 8,
"type": "OWNED",
"properties": {
"ownedDate": "2021-07-03"
}
},
{
"identity": 45,
"start": 9,
"end": 8,
"type": "PLAYED",
"properties": {
"times": 5
}
}
],
"name": "Blood Rage",
"state": "ACTIVE",
"id": "1c152c91-4044-41f0-9208-0c436d6f6480",
"gameUrl": "https://asmodee.de/blood-rage"
}
GetMapping result (As you can see, the relationships are empty, but i have more empty JsonObjects, when there are more Relationships
{
"game": {
"relationships": [
{},
{}
],
"name": "Blood Rage",
"gameUrl": "https://asmodee.de/blood-rage",
"state": "ACTIVE",
"id": "1c152c91-4044-41f0-9208-0c436d6f6480"
}
}
The GetMapping is:
...
final ReactiveNeo4jClient client;
...
...
...
#GetMapping(value = { "/{id}/games"})
#RolesAllowed({"user", "admin"})
Flux<Map<String, Object>> findGamesByUser(#PathVariable String id){
String query = "MATCH (uuser:User {id: '" + id + "'} ) -[rel]- (game:Game) return game{.*, relationships: collect(DISTINCT rel)}";
return client.query(query).fetch().all();
}
A RelationshipProperty-Example
#RelationshipProperties
#Data
#Builder
public class PlayedGame {
#Id
#GeneratedValue
private Long relationshipId;
#Property
int times = 0;
#TargetNode private GameEntity game;
public int addPlay(){
this.times = this.times + 1;
return this.times;
}
}
What do i have to change in my GetMapping to show the relationship-properties?
Thank you,
Kevin
You need to return the actual nodes and relationships, otherwise you're missing the id-mapping.
There should be examples in the SDN docs.
Best if you have a small reproducible example (e.g. with the default movies graph).
Not sure if there is something off in your SDN setup, in general for such simple queries you should be able to just use a repository and not need to write cypher queries by hand.
The general information given by Michael is correct but there is more in you question:
First of all the meta domain model is completely ignored if you are using the Neo4jClient. It does not automatically map anything back but uses the driver's types.
As a result you will end up with an (current state of this answer) InternalRelationship which does not have any getter-methods.
I assume that you are serializing the result in the application with Jackson. This is the reason why you see objects that represent the relationships but without any content within.
If you want to get things mapped for you, create also the domain objects properly and use (at least) the Neo4jTemplate with your query.
If you model User, Game, and the relationship properties like PlayedGame correctly, a
neo4jTemplate.findAll("MATCH (u:User)<-[rel]-(g:Game) return u, collect(rel), collect(g)", User.class)
will map the results properly. Also if this is all you have, you could also skip the custom query at all and use
neo4jTemplate.findAll(User.class)
or
neo4jTemplate.findById(useId, User.class)

elasticsearch mongoid filter on has_many ids

I'm using ES 1.4, Rails 5, and Mongoid 6. I'm also the mongoid-elasticsearch gem, which I don't think is relevant, but including it in case I'm wrong.
I have a Case model. When I run this query, everything works great, here's the query:
GET _search
{
"query":{
"filtered":{
"query":{
"query_string":{
"query":"behemoth"
}
}
}
}
}
Here's a result, notice the organization_id:
hits": [
{
"_index": "cases",
"_type": "case",
"_id": "57d5e583a46100386987d7f4",
"_score": 0.13424811,
"_source": {
"basic info": {
"first name": "Joe",
"last name": "Smith",
"narrative": "behemoth"
},
"organization_id": {
"$oid": "57d4bc2fa461003841507f83"
},
"case_type_id": {
"$oid": "57d4bc7aa461002854f88441"
}
}
}
See how there's that "$oid" for organzation id? That's because in my as_indexed_json method for Case, I have:
["organization_id"] = self.case_type.organization.id
I think that the filter doesn't work b/c Mongoid somehow adds that subkey $oid. So my next thought was, I'll just make it a string:
["organization_id"] = self.case_type.organization.id.to_s
But that throws an error:
{"error":"MapperParsingException[object mapping for [case] tried to parse as object, but got EOF, has a concrete value been provided to it?]","status":400}
Anyone have any idea how to A) either use a mongo id as a filter, or B) get ES the info it needs so it doesn't complain as above?
Thanks for any help,
Kevin
Turns out that this is because there was already an existing index. When I nuked the index and re-indexed the data, this works fine (really bad error syntax with ES).

Neocons Cypher tquery and accessing values from keys

After calling neo4j with neocons (cy/tquery conn node-query {:_nodeid _nodeid}), how do you perform accessing functions to get property values from the keys that are returned from the neo4j datastore response?
For example if this object was the response from the neo4j datastore, what neocons syntax do I use to access the value stored in key "attributes"?
[ {
"id": "letter-a",
"name": "Letter A",
"attributes": [ ... ]
}]
Currently I can only get so far as (first _response) but (get-in (first _response) [:attributes]) is giving me back nil
******************* EDIT *******************************
Here is the cypher query string I use as an argument to invoke the tquery function:
(def node-query "MATCH (n)-[attributelist:RELATIONSHIPTYPE]->(target)
RETURN n.id AS id,
n.name AS name,
COLLECT({
target : target.id
}) AS attributes;")
I don't understand what type of variable tquery returns? It looks like this object when the client displays it all the way in the browser:
[
{
"id": "node-999990a0a0a0sa0",
"name": "Node Name",
"attributes": [
{
"target": "node-id-one"
},
{
"target": "node-id-two"
},
{
"target": "node-id-two"
},
{
"target": "node-id-two"
},
{
"target": "node-id-three"
}
]
}
]
But I want to intercept what is returned from the tquery before the clojure server delivers it to the client and I want to manipulate the array value of the key "attributes" so I can run a reduce (tally) report before I deliver a rebuilt object response to the client.
{
...
"attributes" : {
"node-id-one" : 1,
"node-id-two" : 3,
"node-id-three" : 1
}
}
But I am having trouble because I do not know the syntax to access the "attributes" key from the object that is returned from the tquery
Sorry I don't understand your question. Which query did you run with tquery?
Usually you return the data you're interested in directly from the query.
e.g
MATCH (p:Person)-[:ACTED_IN]->(m)
WHERE p.name = "Tom Hanks"
RETURN m.title, m.released`
Otherwise you'd have to requery using a label+unique-property
MATCH (m:Movie)
WHERE m.title = "The Matrix"
RETURN m.title, m.released`
or node-id match.
MATCH (m:Movie)
WHERE id(m) = 123
RETURN m.title, m.released`
You usually would use parameters instead of literal values, i.e. {name}, {title}, {id}.
Update
I think for intercepting you would have to look into the neocons implementation.
Note: there is no clojure server, it's a Neo4j server with an http endpoint.
You should be able to do what you want (almost) in cypher.
MATCH (n)-[:RELATIONSHIPTYPE]->(target)
WITH n, target.id as target, count(*) as c
RETURN n.id as id, n.name as name, collect([target,c]) as targets;
Unfortunately right now there are no dynamic map-keys in Cypher, so a tuple-collection will have to do.
PS
You should use a label at least for your n (and optionally target) nodes.

JSON-LD normalization - ignore JSON nesting

I'm working on JSON-LD serialization, and ideally I would like to have a #context which I can add to the existing GeoJSON output (together with some #ids and #types), so that both the Turtle output and the JSON-LD output will normalize to the same triples.
Data is organized as follows: each object/feature has an ID and a name, and data on one or more layers. Per layer, there is a data field, which contains a JSON object.
Example GeoJSON output:
{
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"properties": {
"id": "admr.nl.appingedam",
"name": "Appingedam",
"layers": {
"cbs": {
"data": {
"name": "Appingedam",
"population": 1092
}
},
"admr": {
"data": {
"name": "Appingedam",
"gme_code": 4654,
"admn_level": 3
}
}
}
},
"geometry": {…}
}
]
}
Example Turtle output:
<admr.nl.appingedam>
a :Node ;
dc:title "Appingedam" ;
:createdOnLayer <layer/admr> ;
:layerData <admr.nl.appingedam/admr> ;
:layerData <admr.nl.appingedam/cbs> .
<admr.nl.appingedam/admr>
a :LayerData ;
:definedOnLayer <layer/admr> ;
<layer/admr/name> "Appingedam" ;
<layer/admr/gme_code> "4654" .
<layer/admr/admn_level> "3" .
<admr.nl.appingedam/cbs>
a :LayerData ;
:definedOnLayer <layer/cbs> ;
<layer/cbs/name> "Appingedam" ;
<layer/cbs/population> "1092" ;
The properties object does not have its own URI. Is there a way to create a JSON-LD context which takes the contents of the properties into account, but further 'ignores' its precence?
Answered by Gregg Kellogg on JSON-LD mailing list:
This is something that keeps coming up: having a transparent layer,
that basically folds properties up a level. This was discussed during
the development of JSON-LD, but ultimately it was rejected.
I don't see any prospects for doing something in the short-term, but
it could be revisited in a possible future WG chartered with revising
the spec. Feedback like this is quite useful.
In the mean time, you can play with different JSON-LD encodings that
match your RDF though tools like http://json-ld.org/playground and my
own http://rdf.greggkellogg.net/distiller.
Gregg

dust js: aliasing an object not working

Template
{#person alias=root}{alias.value}: {name}, {age}{/person}
data:
{
"root": {value:"MR."},
"person": {
"name": "Larry",
"age": 45
}
}
Expected output:
MR. Larry, 45
Actual output:
: Larry, 45
I'm trying to alias an object like shown above. But its not working. Please have a look into this fiddle http://jsfiddle.net/G86mu/1/.
If i replace {value:"MR."} with a string say "root":"Mr." and change my template to
{#person alias=root}{alias}: {name}, {age}{/person}
the output is as expected. Please let me know how do i alias an object
The reason this isn't working is because the context within Dust is not the same as the JSON you pass in to dust.render. Internally, Dust wraps your JSON so that it can include params, globals, and blocks in the context.
So, you are not adding alias to the current context, as you might assume. Instead, you are adding alias one level above your current context. Although the representation isn't exactly accurate, it should be helpful for explanation purposes:
// Incorrect:
{
"root": {
"value": "MR."
},
"person": {
// Current context
"alias": {
"value": "MR."
},
"name": "Larry",
"age": "45"
}
}
// (more) correct:
{
"root": {
"value": "MR."
},
"alias": {
"value": "MR."
"person": {
// Current context
"name": "Larry",
"age": "45"
}
}
}
When the context is viewed in this way, it makes sense why {#person alias=root}{alias.value}: {name}, {age}{/person} will not work. When using the dot-notation inside of a reference (as in {alias.value}, dust starts in the current context and goes down. Since there is no "alias" object inside of the current context, dust gives up, and you get an empty string.
However, if when you don't use the dot-notation, dust starts at the current context and searches up. The first time it finds a match, it will use that match. So, for your example, you could use the following to get your expected output.
{#person alias=root}{#alias}{value}{/alias}: {name}, {age}{/person}
Alternatively, if you could use:
{#person aliasVal=root.value}{aliasVal}: {name}, {age}{/person}

Resources