I am trying to perform multiple aggregations at the same level with ElasticSearch using the elasticsearch-rails and elasticsearch-model gems.
In the query hash that I am generating, I have the following -
def query_hash(params, current_person = nil, manager_id = nil)
aggs = {}
aggs[:home_country_id] = {terms: {field: "home_country_id"}}
aggs[:home_region_id] = {terms: {field: "home_region_id"}}
{
sort: [ { created_at: { order: "desc" } }, id: { order: "desc" } ],
aggs: aggs
}
end
The response I stored in an object es_response.
When I search for both the aggregations, I can only find the last one in the response.
es_response.response["aggregrations"] only has the response of the latest aggregation object, home_region_id.
I couldn't find much documentation on the ES Reference on structuring multiple aggregations on the same level although there was a lot about nesting aggregations.
How can I fix this?
My ES version is 5.1
Related
I have tried mostly everything. Checked all docs and stack questions and shopify community like:
Shopify API how to do a search query with like
https://community.shopify.com/c/Shopify-APIs-SDKs/Shopify-api-search-products-by-title/td-p/341866
How to search products by title using Shopify product search API?
https://github.com/Shopify/shopify_app
https://community.shopify.com/c/Shopify-APIs-SDKs/Search-product-from-title-handle-and-description/td-p/469156
and found out that
#search = "2018";
#products = ShopifyAPI::Product.find(:all, params: { limit: 10,title:#search })
but this is returning empty array although I have may records containing this in title. https://prnt.sc/sx37o6
I want to get records according to #search
I have tried Product.search too but it causes: undefined method `search' for ShopifyAPI::Product:Class
Using the RestAPI I failed doing filtering (with wildcards for example) as well. But with the GraphQL-API the search functionalities (see here https://shopify.dev/concepts/about-apis/search-syntax) are pretty solid.
This is an example including auth, filtering by title including wildcard-support (for part matching) and mapping results to a simpel array of hashes:
#responses = []
shopify_session = ShopifyAPI::Session.temp(
domain: shop.shopify_domain,
token: shop.shopify_token,
api_version: ShopifyApp.configuration.api_version
) do
client = ShopifyAPI::GraphQL.client
ql_query = <<-GRAPHQL
{
products(first: 10, query: "title:*#{query}*") {
edges {
node {
id
title
handle
}
}
}
}
GRAPHQL
query_result = client.query(client.parse(ql_query))
query_result.data.products.edges.each do |result|
#responses << {
id: result.node.id,
title: result.node.title,
handle: result.node.handle
}
end
end
#responses
For the life of me I can't find any reference to using the ElasticSearch scroll api from within Ruby on Rails and the elastisearch-model (or rails or dsl) gem.
The only thing they do reference in the docs is calling scroll directly on the client, which kind of defeats the purpose. Also, it does not use the client or any client settings you've already set in your Rails app.
I want to do something like this.
Here is the ElasticSearch query that works from within the Kibana Dev Tools:
GET model_index/_search?scroll=1m
{
"size": 100,
"query": {
"match": {
"tenant_id": 3196
}
},
"_source": "id"
}
I would have thought that I could call something like
MyModel.search scroll: '1m', ...
but instead it seems like I need to do:
# First create a client by hand
client = Elasticssearch::Client.new
result = client.search index: 'model_index',
scroll: '1m',
body: { query: { match: { tenant_id: 3196 } }, sort: '_id' }
Does anyone have any more user-friendly examples?
As per elasticsearch guide -
We no longer recommend using the scroll API for deep pagination. If you need to preserve the index state while paging through more than 10,000 hits, use the search_after parameter with a point in time (PIT).
Ref - https://www.elastic.co/guide/en/elasticsearch/reference/7.x/scroll-api.html
Further edit for above question -
To scroll on document need to use scroll_id from result, to get next set of result.
body = { query: { match: { tenant_id: 3196 } }, sort: '_id' }
response = Elasticsearch::Client.new.search(
index: 'model_index',
scroll: "1m",
body: body,
size: 3000
)
loop do
hits = response.dig('hits', 'hits')
break if hits.empty?
hits.each do |hit|
# do something
end
response = Elasticsearch::Client.new.scroll(
:body => { :scroll_id => response['_scroll_id'] },
:scroll => '1m'
)
end
This is the index of my model in the elastic search
{
"_index":"cars",
"_type":"car",
"_id":"3275",
"_version":4,
"_score":1,
"_source":{
"category_id": 6,
"car_branches":[
{
"id":32,
"name":"Type1"
},
{
"id":33,
"name":"Type2"
},
{
"id":36,
"name":"Type3"
}
],
}
}
I can query category_id with
Car.__elasticsearch__.search query:{match:{category_id: 6}}
How do I query for car_branches? I tried this
response = Car.__elasticsearch__.search query:{match:{car_branches:[id: 32]}}
I am getting Elasticsearch::Transport::Transport::Errors::BadRequest: [400]
You first need to delete your index first and recreate it. Before doing so, you need to change your mapping and make the car_branches field nested, like this:
indexes :car_branches, type: 'nested' do
indexes :id
indexes :name
Then you'll be able to make the query your want like this:
response = Car.__elasticsearch__.search query:{nested:{path: 'car_branches', query:{term:{'car_branches.id':[32]}}}}
I used gem elasticsearch-rails and elasticsearch-model and I have difficult to write this query in elasticsearch-rails.
SELECT "news".* FROM "news"
WHERE "news"."is_active" = 'true' AND
((priority is not null AND created_at > '2014-07-08 08:55:52.888587') OR
(created_at > '2014-07-08 08:55:52.888820' AND is_persisted = 't') )
ORDER BY "news"."priority" ASC, "news"."created_at" DESC
LIMIT 10 OFFSET 0
In my previous project I used "Tire Model", I used something like this:
filter :bool, must: {and: [{term: {field with options}}, {term: {field with options}}]}, It works in tire model
But if I use something like this in elasticsearch-rails, it throws missing filtered error
I write something like this for filtering active record:
def self.news_index(page = 1)
query =:
response=self.elasticsearch.search query: { match: { is_active: true }}
response=response.page(page)
end
In the above method, I want to add combined filter with bool option. Can anyone guide me?
Elasticsearch-ruby is far closer to the elasticsearch DSL when it comes to querying. Most of the time you'll be passing in hashes (or hash-like objects) to the query method.
Something like this should get you close:
self.search query: {
filtered: {
query: { match: { is_active: true }},
filter: {
bool: {
must: {
and: [{term: {field with options}}, {term: {field with options}}]
}
}
}
}
}
The difference is that instead of filter being a method call in the tire query which took arguments :bool and the filter. You now need to specify a :filter key with a hash value which then contains a :bool key with your existing filter as the value.
I'm having an issue with mongoid any_of. I'm trying to find objects that have either one field > 0, or another one > 0. My query is :
Model.any_of(best_friend_method.gt => 0, method.gt => 0).desc(best_friend_method, method)
It is "translated" in :
#<Mongoid::Criteria
selector: {"$or"=>[{:best_friends_lc_sum=>{"$gt"=>0}, :lc_sum=>{"$gt"=>0}}]},
options: {:sort=>[[:best_friends_lc_sum, :desc], [:lc_sum, :desc]]},
class: FbAlbum,
embedded: false>
As I understand it, this is what I want. But it only returns me 6 results. Model.where(:best_friends_lc_sum.gt => 0).count returns me 6 results too, but Model.where(:lc_sum.gt => 0).count returns me ~850 objects.
I expect my query to return the union of those two : is a mongoid/mongodb error, or am I doing something wrong ?
FYI : mongoid 2.4.5, mongodb 2.0.2, rails 3.1.3
Thanks for your time!
It's because you pass only one args and not 2 args. So it's like you have no $or usage.
Try :
Model.any_of({best_friend_method.gt => 0}, {method.gt => 0}).desc(best_friend_method, method)
In this case the Criteria become :
#<Mongoid::Criteria
selector: {"$or"=>[{:best_friends_lc_sum=>{"$gt"=>0}}, {:lc_sum=>{"$gt"=>0}}]},
options: {:sort=>[[:best_friends_lc_sum, :desc], [:lc_sum, :desc]]},
class: FbAlbum,
embedded: false>
Sometime the usage of {} is mandatory to separate different hash.
In case this helps someone...In Mongoid 3, the Origin gem provides the syntax for querying. Here is the list of methods that you can use to write your Mongoid 3 queries. Among those methods is the or method which allows you to perform an $or query:
# Mongoid query:
Model.or(
{ name: "Martin" }, { name: "Dave" }
)
# resulting MongoDB query:
{
"$or" => [
{ "name" => "Martin" }, { "name" => "Dave" }
]
}
Using the OP's original example, it can be rewritten as:
Model.or(
{ best_friend_method.gt => 0 },
{ method.gt => 0 }
).order_by(
best_friend_method,
method
)
At least one of the hashes passed to the or method must match in order for a record to be returned.