A little background....I'm converting an existing API to leverage elastic search for performance gains.
It's a rails app that's using active record. Here is the query where condition that we're trying to do in elasticsearch:
where("start_date BETWEEN :start_at AND :end_at
OR end_date BETWEEN :start_at AND :end_at
OR :start_at BETWEEN start_date AND end_date
OR :end_at BETWEEN start_date and end_date", start_at: start_at.to_date, end_at: end_at.to_date)
The first half of that where clause is easy to replicate. Here's how that part looks:
Model.search(
query: {
bool: {
should: [
{
range: {
start_date: {
gte: '2015-12-01',
lte: '2015-12-25'
}
}
},
{
range: {
end_date: {
gte: '2015-12-01',
lte: '2015-12-25'
}
}
}
],
minimum_should_match: 1
}
}
)
I'm just not sure how to implement the 2nd part of that where clause. I've attempted using scripting but I'm having issues converting the dates to the proper formats so that they can be compared.
For example here was something I tried:
filter: {
script: {
script: {
inline: "doc['start_date'] > start_on",
params: {
start_on: Date.parse('2015-12-01')
}
}
}
}
When I do this I get this error:
Cannot compare org.elasticsearch.index.fielddata.ScriptDocValues$Longs with value '[1449446400000]' and java.lang.String with value '2015-12-01'"
I've also tried this:
script: "Date.parse('yyyy-MM-dd', '2015-12-01').getTime() >= doc['start_date']"
I get this error:
org.elasticsearch.index.fielddata.ScriptDocValues$Longs cannot be cast to java.lang.Long
I'm just not sure how to get the data types to match so that I can compare the dates.
I wish I could do something like this, but its not possible of course:
range: {
'2015-12-01': {
gte: start_date
}
}
Any help on this would be greatly appreciated. Thanks!
Try this:
filter: {
script: {
script: {
inline: "new java.util.Date(doc['start_date'].value) > new java.util.Date(2015-12-01)"
}
}
}
A co-worker sent me this: https://stackoverflow.com/a/325964, so I ended up with a much simpler approach:
Model.search(
query: {
bool: {
must: [
{
range: {
start_date: {
lte: '2015-12-25'
}
}
},
{
range: {
end_date: {
gte: '2015-12-01',
}
}
}
]
}
}
)
Related
So i have the below module in an ElasticSearch concern for my Model in rails.
This is working, but how do I make each of the bool query(must, must_not, filter) accept nil or empty parameters?
Say if I pass an empty query_string it would get all the documents.
Then when I pass an empty size parameter it will return all sizes.
module ClassMethods
def home_page_search(query_string, size, start_date, end_date)
search({
query: {
bool: {
must: [
{
multi_match: {
query: query_string,
fields: [:brand, :name, :notes, :size_notes]
}
}
],
must_not: [
range: {
unavailable_dates: { gte: start_date, lte: end_date }
}
],
filter: [
{ term: { size: size } }
]
}
}
})
end
end
I solved a similar problem by constructing the query string on more of an as-needed basis, so I only included a clause if there was a search term for it. The query I sent to Elasticsearch only included the terms that were actually set by the user. For example:
if size.present?
query[:query][:bool][:filter] = { term: { size: size } }
end
(assuming the correct representation of the query, etc.)
I'm using this to search my outfits:
def self.search(query, purchasedlow, purchasedhigh)
__elasticsearch__.search(
{
query: {
function_score: {
query: {
bool: {
filter: [
{
multi_match: {
query: query,
fields: ['name','description','material']
}
},
{
range: {
purchased: { lte: purchasedhigh.to_i, gte: purchasedlow.to_i},
},
}
]
}
}
}
}
}
)
end
But I don't know how to add this code:
field_value_factor: {
field: "likes",
factor: "100"
}
I know that I'm supposed to put it after the function score, so that the calculated score is then multiplied by the amount of likes to make the final score, but when I put the code after the function_score, I get the following error:
[400] {"error":{"root_cause":[{"type":"parsing_exception","reason":"[function_score] malformed query, expected [END_OBJECT] but found [FIELD_NAME]","line":1,"col":232}],"type":"parsing_exception","reason":"[function_score] malformed query, expected [END_OBJECT] but found [FIELD_NAME]","line":1,"col":232},"status":400}
Where do I need to put the field value factor so that it works correctly?
I used field_value_factor in one of my queries like this:
#products = Product.search(
query:{
function_score:{
query:{
bool:{
must:{
multi_match:{
fields: ['brand^5', '_all'],
query: "#{query}",
fuzziness: "AUTO"
}
},
filter:{
bool:{
must:[
{term: {"brand":"NordicTrack"}},
{term: {"product_type":"Treadmill"}}
]
}
}
}
},
field_value_factor:{
field: "popularity",
modifier: "log1p"
}
}
})
I'm using elastic search in Rails. I am trying to sort a list of customers by their total dollars spent descending. This is my ruby code:
query = {
bool: {
filter: {
term: { store_id: store.id } # Limits customers by current store
}
}
}
sort = {
sort: { "total_spent": { order: "desc" }}
}
response = Contact.search(query: query, sort: sort)
This returns with an error of sort option [total_spent] not supported I've tried with other fields to make sure it wasn't just something wrong with the total_spent field. Thanks.
I'm not really sure, but I think this may be related to incorrect usage of the ES::DSL.
What happens when you try this:
query = {
bool: {
filter: {
term: { store_id: store.id } # Limits customers by current store
}
}
}
sort = {
sort: [{ "total_spent": { order: "desc" }}] #https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-sort.html
}
response = Contact.search(query, sort)
We can sort specific to the field, refer https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-sort.html.
so we can use like,
query = {
bool: {
filter: {
term: { store_id: store.id } # Limits customers by current store
}
},
sort: { total_spent: { order: :desc }}
}
response = Contact.search(query)
I'd like make a multi query on Elasticsearch through Tire but with raw JSON
I can to a single request like this
#search = Tire.search('questions', query: {
function_score: {
query: {
bool: {
must: [
{
terms: {
interests: [2943,5106,3540,1443,3639]
}
}
]
}
},
random_score: {}
}
})
But for multiple I can't.
I'd like somthing like this, but it's not correct for now...
#search = Tire.multi_search 'questions' do
search :level2 do
query: {
function_score: {
query: {
bool: {
must: [{
terms: {
interests: [5090,2938,3062]
}}]
}
},
random_score: {}
}
}
end
end
Do you now how I could do to make it work?
Thank you
I found the solution.
Actually, in my case Search method is requiring :payload key in options params
#search = Tire.multi_search 'questions' do
search( :level1, :payload => {
query: {
function_score: {
query: {
bool: {
must: [
{
terms: {
interests: [2943,5106,3540,1443,3639]
}
},{
term: {
difficulty: 1
}
}
]
}
},
random_score: {}
}
}})
search( :level2, :payload => {
query: {
function_score: {
query: {
bool: {
must: [
{
terms: {
interests: [5160,2938,3062]
}
},{
term: {
difficulty: 2
}
}
]
}
},
random_score: {}
}
}})
end
I am using Tire for rails to integrate elasticsearch. This bit is quite confusing and I want to make sure I'm doing this right.
Is this how I apply multiple filters? I'm basically trying to check 'mixtape_id IS NULL AND artist_id IS NOT NULL'
def self.search(query)
tire.search() do
query { string query }
filter :exists, { field: 'artist_id' }
filter :not, { exists: { field: 'mixtape_id' } }
end
end
Here is my second attempt, still doesnt appear to work
def self.search(query)
tire.search(load: true) do
query { string query }
filter :and, [
{ exists: { field: 'artist_id' } },
{ not: { exists: { field: 'mixtape_id' } } }
]
end
end
Thanks
I mostly had it working the whole time, I stupidly forgot to force reindexing each time though sigh. Here is some cleaned up code that takes advantage of the missing filter.
def self.search(query)
tire.search load: { include: { artist: :attachments } } do
query { string query }
filter :and, [
{ exists: { field: 'artist_id' } },
{ missing: { field: 'mixtape_id' } }
]
end
end