elasticsearch mongoid filter on has_many ids - ruby-on-rails

I'm using ES 1.4, Rails 5, and Mongoid 6. I'm also the mongoid-elasticsearch gem, which I don't think is relevant, but including it in case I'm wrong.
I have a Case model. When I run this query, everything works great, here's the query:
GET _search
{
"query":{
"filtered":{
"query":{
"query_string":{
"query":"behemoth"
}
}
}
}
}
Here's a result, notice the organization_id:
hits": [
{
"_index": "cases",
"_type": "case",
"_id": "57d5e583a46100386987d7f4",
"_score": 0.13424811,
"_source": {
"basic info": {
"first name": "Joe",
"last name": "Smith",
"narrative": "behemoth"
},
"organization_id": {
"$oid": "57d4bc2fa461003841507f83"
},
"case_type_id": {
"$oid": "57d4bc7aa461002854f88441"
}
}
}
See how there's that "$oid" for organzation id? That's because in my as_indexed_json method for Case, I have:
["organization_id"] = self.case_type.organization.id
I think that the filter doesn't work b/c Mongoid somehow adds that subkey $oid. So my next thought was, I'll just make it a string:
["organization_id"] = self.case_type.organization.id.to_s
But that throws an error:
{"error":"MapperParsingException[object mapping for [case] tried to parse as object, but got EOF, has a concrete value been provided to it?]","status":400}
Anyone have any idea how to A) either use a mongo id as a filter, or B) get ES the info it needs so it doesn't complain as above?
Thanks for any help,
Kevin

Turns out that this is because there was already an existing index. When I nuked the index and re-indexed the data, this works fine (really bad error syntax with ES).

Related

Elasticsearch saves document as string of array, not array of strings

I am trying to contain array as a document value.
I succeed it in "tags" field as below;
This document contains array of strings.
curl -XGET localhost:9200/MY_INDEX/_doc/132328908
#=> {
"_index":"MY_INDEX",
"_type":"_doc",
"_id":"132328908",
"found":true,
"_source": {
"tags": ["food"]
}
}
However, when I am putting items in the same way as above,
the document is SOMETIMES like that;
curl -XGET localhost:9200/MY_INDEX/_doc/328098989
#=> {
"_index":"MY_INDEX",
"_type":"_doc",
"_id":"328098989",
"found":true,
"_source": {
"tags": "[\"food\"]"
}
}
This is string of array, not array of strings, which I expected.
"tags": "[\"food\"]"
It seems that this situation happens randomly and I could not predict it.
How could it happen?
Note:
・I use elasticsearch-ruby client to index a document.
This is my actual code;
es_client = Elasticsearch::Client.new url: MY_ENDPOINT
es_client.index(
index: MY_INDEX,
id: random_id, # defined elsewhere
body: {
doc: {
"tags": ["food"]
},
}
)
Thank you in advance.

Returning Neo4j map projection with WebFlux

I have the nodes user and game with some relationships between them.
My REST API should return all relationships between the games and 1 user.
The cypher query i use is:
MATCH (u:User {id: '1234'} ) -[rel]- (game:Game) return game{.*, relationships: collect(DISTINCT rel)}
In my Neo4j Browser, everything works as expected and i see all properties i need.
But the GetMapping retuns everything except the relationship properties.
Neo4j Browser
{
"relationships": [
{
"identity": 54,
"start": 9,
"end": 8,
"type": "OWNED",
"properties": {
"ownedDate": "2021-07-03"
}
},
{
"identity": 45,
"start": 9,
"end": 8,
"type": "PLAYED",
"properties": {
"times": 5
}
}
],
"name": "Blood Rage",
"state": "ACTIVE",
"id": "1c152c91-4044-41f0-9208-0c436d6f6480",
"gameUrl": "https://asmodee.de/blood-rage"
}
GetMapping result (As you can see, the relationships are empty, but i have more empty JsonObjects, when there are more Relationships
{
"game": {
"relationships": [
{},
{}
],
"name": "Blood Rage",
"gameUrl": "https://asmodee.de/blood-rage",
"state": "ACTIVE",
"id": "1c152c91-4044-41f0-9208-0c436d6f6480"
}
}
The GetMapping is:
...
final ReactiveNeo4jClient client;
...
...
...
#GetMapping(value = { "/{id}/games"})
#RolesAllowed({"user", "admin"})
Flux<Map<String, Object>> findGamesByUser(#PathVariable String id){
String query = "MATCH (uuser:User {id: '" + id + "'} ) -[rel]- (game:Game) return game{.*, relationships: collect(DISTINCT rel)}";
return client.query(query).fetch().all();
}
A RelationshipProperty-Example
#RelationshipProperties
#Data
#Builder
public class PlayedGame {
#Id
#GeneratedValue
private Long relationshipId;
#Property
int times = 0;
#TargetNode private GameEntity game;
public int addPlay(){
this.times = this.times + 1;
return this.times;
}
}
What do i have to change in my GetMapping to show the relationship-properties?
Thank you,
Kevin
You need to return the actual nodes and relationships, otherwise you're missing the id-mapping.
There should be examples in the SDN docs.
Best if you have a small reproducible example (e.g. with the default movies graph).
Not sure if there is something off in your SDN setup, in general for such simple queries you should be able to just use a repository and not need to write cypher queries by hand.
The general information given by Michael is correct but there is more in you question:
First of all the meta domain model is completely ignored if you are using the Neo4jClient. It does not automatically map anything back but uses the driver's types.
As a result you will end up with an (current state of this answer) InternalRelationship which does not have any getter-methods.
I assume that you are serializing the result in the application with Jackson. This is the reason why you see objects that represent the relationships but without any content within.
If you want to get things mapped for you, create also the domain objects properly and use (at least) the Neo4jTemplate with your query.
If you model User, Game, and the relationship properties like PlayedGame correctly, a
neo4jTemplate.findAll("MATCH (u:User)<-[rel]-(g:Game) return u, collect(rel), collect(g)", User.class)
will map the results properly. Also if this is all you have, you could also skip the custom query at all and use
neo4jTemplate.findAll(User.class)
or
neo4jTemplate.findById(useId, User.class)

Mongodb - search without key

I'd like to search documents in collection only by value. Let's say my collection contains documents like below:
[
{
"_id": "57a443c74d854d192afcc451",
"somekey": "123",
"otherkey": "zxc"
},
{
"_id": "57a443ca4d854d192afcc452",
"key": "123",
"otherkey": "123zxcvbnm"
}
]
and now I want to get all documents where value of any key contains 123.
I tried to do something like (written in Ruby and using mongoid):
new_search_query = { /.*/ => /#{v}/ }
collection.find(new_search_query)
but it looks like it is not suported becuase I get:
BSON::InvalidKey (Regexp instances are not allowed as keys in a BSON document.):
Is there any other manner or some workaround to do it?
Try full_text_search of mongoid for rails app.

Elasticsearch – combine ids query with must_not clause

I’d like to combine ids query with must_not clause. I’m using (re)tire gem with rails application.
I’m doing something like this:
query do
ids ["foo", "bar", "test"]
end
Which works. However, when I’m trying to combine it with must_not clause, I’m getting error – must_not works only withing boolean scope, which does not include ids query. Is there any way to combine that two queries?
The ES query that you need to use is this one:
GET /some_index/some_type/_search
{
"query": {
"bool": {
"must": [
{
"ids": {
"values": ["foo", "bar", "test"]
}
}
],
"must_not": [
{
"match": {
"name": "whatever"
}
}
]
}
}
}
So, you need boolean scope, a must with your ids and a must_not with your must not match queries.

Emit Tuples From Erlang Views In CouchDB

CouchDB, version 0.10.0, using native erlang views.
I have a simple document of the form:
{
"_id": "user-1",
"_rev": "1-9ccf63b66b62d15d75daa211c5a7fb0d",
"type": "user",
"identifiers": [
"ABC",
"DEF",
"123"
],
"username": "monkey",
"name": "Monkey Man"
}
And a basic javascript design document:
{
"_id": "_design/user",
"_rev": "1-94bd8a0dbce5e2efd699d17acea1db0b",
"language": "javascript",
"views": {
"find_by_identifier": {
"map": "function(doc) {
if (doc.type == 'user') {
doc.identifiers.forEach(function(identifier) {
emit(identifier, {\"username\":doc.username,\"name\":doc.name});
});
}
}"
}
}
}
which emits:
{"total_rows":3,"offset":0,"rows":[
{"id":"user-1","key":"ABC","value":{"username":"monkey","name":"Monkey Man"}},
{"id":"user-1","key":"DEF","value":{"username":"monkey","name":"Monkey Man"}},
{"id":"user-1","key":"123","value":{"username":"monkey","name":"Monkey Man"}}
]}
I'm looking into building an Erlang view that does the same thing. Best attempt so far is:
%% Map Function
fun({Doc}) ->
case proplists:get_value(<<"type">>, Doc) of
undefined ->
ok;
Type ->
Identifiers = proplists:get_value(<<"identifiers">>, Doc),
ID = proplists:get_value(<<"_id">>, Doc),
Username = proplists:get_value(<<"username">>, Doc),
Name = proplists:get_value(<<"name">>, Doc),
lists:foreach(fun(Identifier) -> Emit(Identifier, [ID, Username, Name]) end, Identifiers);
_ ->
ok
end
end.
which emits:
{"total_rows":3,"offset":0,"rows":[
{"id":"user-1","key":"ABC","value":["monkey","Monkey Man"]},
{"id":"user-1","key":"DEF","value":["monkey","Monkey Man"]},
{"id":"user-1","key":"123","value":["monkey","Monkey Man"]}
]}
The question is - how can I get those values out as tuples, instead of as arrays? I don't imagine I can (or would want to) use records, but using atoms in a tuple doesn't seem to work.
lists:foreach(fun(Identifier) -> Emit(Identifier, {id, ID, username, Username, name, Name}) end, Identifiers);
Fails with the following error:
{"error":"json_encode","reason":"{bad_term,{<<\"user-1\">>,<<\"monkey\">>,<<\"Monkey Man\">>}}"}
Thoughts? I know that Erlang sucks for this specific kind of thing (named access) and that I can do it by convention (id at first position, username next, real name last), but that makes the client side code pretty ugly.
The JSON object {"foo":"bar","baz":1} is {[{<<"foo">>,<<"bar">>},{<<"baz">>,1}]}
In Erlang lingua it is a proplist wrapped in a tuple.
It's not pretty, but very efficient :)
To get a feel for it you can play with the JSON lib that ships with CouchDB:
Start CouchDB with the -i
(interactive) flag
On the resulting erlang shell, type: couch_util:json_decode(<<"{\"foo\":\"bar\"}">>).
Profit
// in later versions of CouchDB, this is ejson:decode()
For test_suite_reports bd, that has tests field:
[
{
"name": "basics",
"status": "success",
"duration": 21795
},
{
"name": "all_docs",
"status": "success",
"duration": 385
} ...
I have wrote this to get name and status:
fun({Doc}) ->
Name = fun(L) -> proplists:get_value(<<"name">>, L, null) end,
Status = fun(L) -> proplists:get_value(<<"status">>, L, null) end,
Tests = proplists:get_value(<<"tests">>, Doc, null),
lists:foreach(fun({L}) -> Emit(Name(L), Status(L)) end, Tests)
end.
If you like experimental features (that still work...), you might want to have a look to Erlang exprecs.
I found it extremely helpful in creating a sort of dynamic records for Erlang.

Resources