I am using Tire for rails to integrate elasticsearch. This bit is quite confusing and I want to make sure I'm doing this right.
Is this how I apply multiple filters? I'm basically trying to check 'mixtape_id IS NULL AND artist_id IS NOT NULL'
def self.search(query)
tire.search() do
query { string query }
filter :exists, { field: 'artist_id' }
filter :not, { exists: { field: 'mixtape_id' } }
end
end
Here is my second attempt, still doesnt appear to work
def self.search(query)
tire.search(load: true) do
query { string query }
filter :and, [
{ exists: { field: 'artist_id' } },
{ not: { exists: { field: 'mixtape_id' } } }
]
end
end
Thanks
I mostly had it working the whole time, I stupidly forgot to force reindexing each time though sigh. Here is some cleaned up code that takes advantage of the missing filter.
def self.search(query)
tire.search load: { include: { artist: :attachments } } do
query { string query }
filter :and, [
{ exists: { field: 'artist_id' } },
{ missing: { field: 'mixtape_id' } }
]
end
end
Related
I'm using the mongoid 6.1.0 aggregation framework in my Rails 5 project. I need to add a $match pipeline if the value of a search field (text or select field) is not empty. Otherwise, it should be ignored and don't filter results. Something like:
#messages = Message.collection.aggregate([
{ '$match' => {'month': {'$gte' => #fr_mnth, '$lte' => #to_mnth}}},
{ '$group' => {'_id': '$mmsi'} },
{ '$lookup' => {'from': 'ships', 'localField': "_id", 'foreignField': "mmsi", as: "ship"}},
{ '$match' => {"ship.n2": params[:n2] if !params[:n2].blank? }}
]).allow_disk_use(true)
or to be more clear:
if not params[:n2].blank? { '$match' => {"ship.n2": params[:n2] }}
The problem is that if !params[:n2].blank? cannot be included in the aggregation framework. Is there any other alternative solution?
I don't know ruby, but maybe I understand your problem.
Pseudo-code
# DON'T DO SO! SEE UPDATE BELOW
if your_condition is true:
filter = { field: 'some value' }
else:
filter = { # always true condition
$or: [
{ field: { $exists: true } },
{ field: { $exists: false } }
]
}
Message.collection.aggregate([
# ...
{
"$match": filter
}
])
UPDATE:
As Aboozar Rajabi noted, if condition is true then we can just add $match stage to pipeline:
pipeline = [
# stages
];
if condition is true:
pipeline.push({
$match: {
# filter
}
});
The above pseudo-code (Kan A's answer) is translated to Ruby and Mongoid aggregation framework as its syntax might be a bit confusing and there're a few Mongoid aggregation examples online:
if not params[:field].blank?
filter = { "db_field_name": params[:field] }
else
filter = {
'$or' => [
{ "db_field_name" => { '$exists' => true } },
{ "db_field_name" => { '$exists' => false } }
]
}
end
I hope it would help the others who will see this page later. Also, this solution and the code in the question would be an example of using MongoDB aggregation framework in a Rails or Ruby project.
So i have the below module in an ElasticSearch concern for my Model in rails.
This is working, but how do I make each of the bool query(must, must_not, filter) accept nil or empty parameters?
Say if I pass an empty query_string it would get all the documents.
Then when I pass an empty size parameter it will return all sizes.
module ClassMethods
def home_page_search(query_string, size, start_date, end_date)
search({
query: {
bool: {
must: [
{
multi_match: {
query: query_string,
fields: [:brand, :name, :notes, :size_notes]
}
}
],
must_not: [
range: {
unavailable_dates: { gte: start_date, lte: end_date }
}
],
filter: [
{ term: { size: size } }
]
}
}
})
end
end
I solved a similar problem by constructing the query string on more of an as-needed basis, so I only included a clause if there was a search term for it. The query I sent to Elasticsearch only included the terms that were actually set by the user. For example:
if size.present?
query[:query][:bool][:filter] = { term: { size: size } }
end
(assuming the correct representation of the query, etc.)
I'm using elastic search in Rails. I am trying to sort a list of customers by their total dollars spent descending. This is my ruby code:
query = {
bool: {
filter: {
term: { store_id: store.id } # Limits customers by current store
}
}
}
sort = {
sort: { "total_spent": { order: "desc" }}
}
response = Contact.search(query: query, sort: sort)
This returns with an error of sort option [total_spent] not supported I've tried with other fields to make sure it wasn't just something wrong with the total_spent field. Thanks.
I'm not really sure, but I think this may be related to incorrect usage of the ES::DSL.
What happens when you try this:
query = {
bool: {
filter: {
term: { store_id: store.id } # Limits customers by current store
}
}
}
sort = {
sort: [{ "total_spent": { order: "desc" }}] #https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-sort.html
}
response = Contact.search(query, sort)
We can sort specific to the field, refer https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-sort.html.
so we can use like,
query = {
bool: {
filter: {
term: { store_id: store.id } # Limits customers by current store
}
},
sort: { total_spent: { order: :desc }}
}
response = Contact.search(query)
There is the following Post model:
class Post < ActiveRecord::Base
include Elasticsearch::Model
include Elasticsearch::Model::Callbacks
def self.search query
__elasticsearch__.search(
{
query: {
multi_match: {
query: query,
fields: ['title']
}
},
filter: {
and: [
{ term: { deleted: false } },
{ term: { enabled: true } }
]
}
}
)
end
settings index: { number_of_shards: 1 } do
mappings dynamic: 'false' do
indexes :title, analyzer: 'english'
end
end
end
Post.import
I have one Post with 'Amsterdam' title. When I execute Post.search('Amsterdam') I will get one record, all is good. But if I execute Post.search('Amster') I will get no records. What do I wrong? How can I fix it? Thanks!
OS - OS X, ElasticSearch I installed using Homebrew
You will have to use nGram tokenizer, in order to create a partial text search. A very good example of how to do this can be found here. That said, I would be very careful with nGram, as it can often turn up unrelated results.
This is because the substring "mon" is contained within all of the strings: "monkey", "money", and "monday". All of which are unrelated.
Alternatively (What I would do.)
You could try making it a fuzzy search. However, the max distance with fuzzy search is only two, which still doesn't return anything in your example. However, it tends to return relevant results.
The example I found: How to use Fuzzy Search
# Perform a fuzzy search!
POST /fuzzy_products/product/_search
{
"query": {
"match": {
"name": {
"query": "Vacuummm",
"fuzziness": 2,
"prefix_length": 1
}
}
}
}
I've been knocking my head against a wall with Elasticsearch today, trying to fix a failing test case.
I am using Rails 3.2.14, Ruby 1.9.3, the Tire gem and ElasticSearch 0.90.2
The objective is to have the query return matching results EXCLUDING the item where
vid == "ABC123xyz"
The Ruby code in the Video model looks like this:
def related_videos(count)
Video.search load: true do
size(count)
filter :term, :category_id => self.category_id
filter :term, :live => true
filter :term, :public => true
filter :not, {:term => {:vid => self.vid}}
query do
boolean do
should { text(:_all, self.title, boost: 2) }
should { text(:_all, self.description) }
should { terms(:tags, self.tag_list, minimum_match: 1) }
end
end
end
end
The resulting search query generated by Tire looks like this:
{
"query":{
"bool":{
"should":[
{
"text":{
"_all":{
"query":"Top Gun","boost":2
}
}
},
{
"text":{
"_all":{
"query":"The macho students of an elite US Flying school for advanced fighter pilots compete to be best in the class, and one romances the teacher."
}
}
},
{
"terms":{
"tags":["top-gun","80s"],
"minimum_match":1
}
}
]
}
},
"filter":{
"and":[
{
"term":{
"category_id":1
}
},
{
"term":{
"live":true
}
},
{
"term":{
"public":true
}
},
{
"not":{
"term":{
"vid":"ABC123xyz"
}
}
}
]
},
"size":10
}
The resulting JSON from ElasticSearch:
{
"took": 7,
"timed_out": false,
"_shards":{
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total":1,
"max_score":0.2667512,
"hits":[
{
"_index":"test_videos",
"_type":"video",
"_id":"8",
"_score":0.2667512,
"_source":{
"vid":"ABC123xyz",
"title":"Top Gun",
"description":"The macho students of an elite US Flying school for advanced fighter pilots compete to be best in the class, and one romances the teacher.",
"tags":["top-gun","80s"],
"category_id":1,
"live":true,
"public":true,
"featured":false,
"last_video_view_count":0,
"boost_factor":0.583013698630137,
"created_at":"2013-08-28T14:24:47Z"
}
}
]
}
}
Could somebody help! The docs for Elasticsearch are sparse around this topic and I'm running out of ideas.
Thanks
Using a top-level filter the way you are doesn't filter the results of your query - it just filters results out of things like facet counts. There's a fuller description in the elasticsearch documentation for filter.
You need to do a filtered query which is slightly different and filters the results of your query clauses:
Video.search load: true do
query do
filtered do
boolean do
should { text(:_all, self.title, boost: 2) }
should { text(:_all, self.description) }
should { terms(:tags, self.tag_list, minimum_match: 1) }
end
filter :term, :category_id => self.category_id
filter :term, :live => true
filter :term, :public => true
filter :not, {:term => {:vid => self.vid}}
end
end
end