What I want to do:
I have a model 'Item' with 2 fields I want elasticsearch to search on: title and description.
I want the search to find partial words, ex: bicycl should match against bicycle, bicycles, etc...
Current situation:
The search only shows perfect matches
Here is what I have right now in my Item model:
include Tire::Model::Search
include Tire::Model::Callbacks
class << self
def search_index
Tire.index(Item.index_name)
end
end
settings :analysis => {
:filter => {
:my_ngram => {
"type" => "nGram",
"max_gram" => 10,
"min_gram" => 3 }
},
:analyzer => {
:my_analyzer => {
"type" => "custom",
"tokenizer" => "standard",
"filter" => ["my_ngram"]
}
}
} do
mapping do
indexes :title, boost: 10, analyzer: 'my_analyzer'
indexes :description, boost: 5, analyzer: 'my_analyzer'
end
end
def self.search(query_string)
tire.search(load: true) do
if query_string.present?
query do
string query_string, default_operator: "AND"
end
end
end
end
When you do...
string query_string, default_operator: "AND"
... you're actually searching the magic _all field.
I'm pretty sure that you need to specifically search for the field analyzed with the ngram filter for this to work.
should { string "title:#{query_string}", default_operator: "OR" }
should { string "description:#{query_string}", default_operator: "OR" }
for instance.
Related
I put tire search in my model:
class Name < ActiveRecord::Base
include Tire::Model::Search
include Tire::Model::Callbacks
mapping do
indexes :name, type: 'string', analyzer: 'snowball'
indexes :lang, type: 'string'
indexes :private, type: 'boolean'
indexes :id, index: :not_analyzed, type: 'integer'
end
end
Then, when i perform:
txt = params[:search]
Name.tire.search page: page, per_page: PER_PAGE do
string txt
end
If works well, but how do i chain more search conditions like:
Name.where(private: false, lang: ['ru', 'en'], id: [1,2,3,4])
I tried to do:
#results = Name.tire.search per_page: per, page: page do
query do
boolean do
must { string txt }
must { term 'names.id', ids } unless ids.blank?
must { term 'names.private', false }
must { term 'names.lang', lang }
end
end
end
But it not returning any results..
try with:
Name.tire.search per_paga: per, page: page do
query {string txt}
filter :boolean, private: false
filter :array, lang: ['ru', 'en'] #here i'm not sure if is array or string
end
Finally found the solution.
Name.tire.search per_pag: per, page: page do
query {string 'text'}
filter :term, private: false
filter :terms, lang: ['ru', 'en']
filter :terms, id: [1,2,3,4]
end
Note the difference in "term" and "terms"(for array desired)
I am trying to match a search query against two fields, as well as filter by facets if selected from dropdowns on the page.
When the user enters keywords it should match if found in two database fields: Title and Description. The dropdowns filter by a status, and a type.
Here is my Tire search configuration:
def self.search(params)
tire.search(load: true, page: params[:page], per_page: 25) do
query do
boolean do
should { string "title:#{params[:query]}", default_operator: "OR" } if params[:query].present?
should { string "description:#{params[:query]}", default_operator: "OR" } if params[:query].present?
must { term :status_id, params[:status_id] } if params[:status_id].present?
must { term :type_id, params[:type_id] } if params[:type_id].present?
end
end
sort { by :updated_at, "desc" } if params[:query].blank?
facet "status" do
terms :status_id
end
facet "type" do
terms :type_id
end
end
end
Indexing settings:
settings :analysis => {
:filter => {
:my_ngram => {
"type" => "nGram",
"max_gram" => 10,
"min_gram" => 3}
},
:analyzer => {
:my_analyzer => {
"type" => "custom",
"tokenizer" => "lowercase",
"filter" => ["my_ngram"]
}
}
} do
mapping do
indexes :title, boost: 10, analyzer: 'my_analyzer'
indexes :description, boost: 5, analyzer: 'my_analyzer'
indexes :status_id, :type => 'integer'
indexes :type_id, :type => 'integer'
end
end
I originally only had the title and description fields, which was working fine. I am now trying to add the ability to filter by status and type.
What is the proper way to configure this? If status is selected, it should only return records with that status. The same follows for type, and if both are selected.
Any help is appreciated.
It's not that errors occur, but the results no longer filter at all by either keywords or facets:
curl -X GET 'http://localhost:9200/projects/project/_search?load=true&size=25&pretty' -d '{"query":{"bool":{"should":[{"query_string":{"query":"title:test","default_operator":"OR"}},{"query_string":{"query":"description:test","default_operator":"OR"}}],"must":[{"term":{"status_id":{"term":"1"}}},{"term":{"type_id":{"term":"1"}}}]}},"facets":{"status":{"terms":{"field":"status_id","size":10,"all_terms":false}},"type":{"terms":{"field":"type_id","size":10,"all_terms":false}}},"size":25}'
# 2013-08-16 12:08:34:791 [200] (31 msec)
#
# {"took":31,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":0,"max_score":null,"hits":[]},"facets":{"status":{"_type":"terms","missing":0,"total":0,"other":0,"terms":[]},"type":{"_type":"terms","missing":0,"total":0,"other":0,"terms":[]}}}
If you could create the equivalent of this I think you would get the desired results. (Please excuse the lack of quotes on the JSON keys!)
{
query: {
multi_match: {
query: "test",
fields: ["title", "description"]
}
},
filter: {
and: [
{
term: { status_id: 123 }
},
{
term: { type_id: 456 }
}
]
},
facets: {
type: {
terms: {
field: "type_id",
size: 10
}
},
status: {
terms: {
field: "status_id",
size: 10
}
}
}
}
Update
I don't know tire but will try to write something!
def self.search(params)
tire.search(load: true, page: params[:page], per_page: 25) do
query do
boolean do
should { match :title params[:query] } if params[:query].present?
should { match :description params[:query] } if params[:query].present?
end
end
sort { by :updated_at, "desc" } if params[:query].blank?
filter :and, { :term => { :status_id => params[:status_id] } } if params[:status_id].present?
{ :term => { :type_id => params[:type_id] } } if params[:type_id].present?
end
end
You will probably have to fix the ruby, but a few things to note. Match queries are the recommended default string search, they are faster than query_string ones (though you have slightly less control). Also
I am using tire and we face a search result problem.
We are searching for 'pencil'.
'red pencil' OK
'electronic pencil sharpener' NOT OK should not be included in the result set.
This is the tire settings on the model:
settings :analysis => {
:analyzer => {
:my_analyzer => {
"tokenizer" => "lowercase",
# "filter" => ["synonym", "porterStem", "phonetic"]
"filter" => ["synonym", "porterStem"]
}
},
:filter => {
:synonym => {
"type" => "synonym",
"synonyms_path" => "#{Synonym.path}"
}
}
} do
mapping do
indexes :commodity_code
indexes :commodity_name
indexes :long_description, analyzer: 'my_analyzer'
indexes :short_description, boost: 10, analyzer: 'my_analyzer'
The query electronic pencil sharpener will be translated to electronic OR pencil OR sharpener by default.
If you want to exclude documents containing sharpener, use a query like this: electronic OR pencil NOT sharpener or +electronic +pencil -sharpener.
Have look at bool and match queries to express conditions like these in the Query DSL; https://github.com/karmi/tire/tree/master/test/integration
I am trying build a search function in rails based on elasticsearch+tire enabling search for Persons with filtering for associated Objects and their Values. A Person has_many Objects, and an Object has_many Values.
I have managed to get the filtering on the object name (params[:object]) to work, but not for object+value. How should I construct the range filter for the values and the mapping so that the value is dependent on the object?
Person controller
mapping do
indexes :objects do
indexes :_id
indexes :object_values do
indexes :value
end
end
indexes :name, type: 'string', analyzer: 'snowball'
end
def self.search(params)
tire.search do
query do
boolean do
must { string params[:query]} if params[:query].present?
end
end
filter :term, {"objects._id" => params[:object]} if params[:object].present?
filter :range, “objects.object_values.value” => {from: params[:value] } if params[:value].present?
end
end
def to_indexed_json
{
:name => name,
:objects => objects.map { |o| {
:_type => 'object',
:_id => o.id,
:object_values => o.object_values.map {|ov| {
:_type => 'object_value',
:_id => ov.id,
:value => ov.value } },
} }
}.to_json
end
Use gt or gte rather than from to specify the lower bounds of your range
filter :range, “objects.object_values.value” => {from: params[:value] }
When adding a term to my query instead of a filter I am getting 0 facets. FYI I am using the tire gem with Ruby.
Here is my model code with its mapping:
class Property < ActiveRecord::Base
include Tire::Model::Search
include Tire::Model::Callbacks
has_and_belongs_to_many :tags
mapping do
indexes :id, type: 'integer'
indexes :status
indexes :refno, type: 'integer'
indexes :name, :analyzer => 'snowball', :boost => 100
indexes :description
indexes :tags, type: 'object',
properties: {
name: { type: 'multi_field',
fields: {
name: { type: 'string', analyzer: 'snowball' },
exact: { type: 'string', index: 'not_analyzed' }
}
}
}
end
def to_indexed_json
to_json( include: {
tags: { only: [:name] },
})
end
Then here is the search method
def self.search(params={})
tire.search(page: params[:page], per_page: 2, load: true) do
query do
boolean do
must { string params[:name], default_operator: "AND" } if params[:name].present?
must { term :status, 'live' }
must { term :refno, params[:refno]} if params[:refno].present?
# must { term :tag, params[:tag]} if params[:tag].present? ## does not work either
must { term 'tags.name.exact', params[:tag]} if params[:tag].present?
end
end
facet "tags" do
terms 'tags.name.exact'
end
raise to_json
# raise to_curl
end
end
I get 0 Facets. But if I move facets to a filter ie below I get full facets.
def self.search(params={})
tire.search(page: params[:page], per_page: 2, load: true) do
query do
boolean do
must { string params[:name], default_operator: "AND" } if params[:name].present?
must { term :status, 'live' }
must { term :refno, params[:refno]} if params[:refno].present?
end
end
filter :term, 'tags.name.exact' => params[:tag] if params[:tag].present?
facet "tags" do
terms 'tags.name.exact'
end
raise to_json
# raise to_curl
end
end
While this is ok it's not want, When a facet filter is clicked I want to remove non available tags from my facet filter and update the new facet count.
If it helps here is the json for the query which works and does not.
## No Factes
{
"query":{
"bool":{
"must":[
{
"query_string":{
"query":"England",
"default_operator":"AND"
}
},
{
"term":{
"status":"live"
}
},
{
"term":{
"tags.name.exact":[
"Pet Friendly"
]
}
}
]
}
},
"facets":{
"tags":{
"terms":{
"field":"tags.name.exact",
"size":10,
"all_terms":false
}
}
},
"size":2
}
## Facets working
{
"query":{
"bool":{
"must":[
{
"query_string":{
"query":"England",
"default_operator":"AND"
}
},
{
"term":{
"status":"live"
}
}
]
}
},
"facets":{
"tags":{
"terms":{
"field":"tags.name.exact",
"size":10,
"all_terms":false
}
}
},
"filter":{
"term":{
"tags.name.exact":[
"Pet Friendly"
]
}
},
"size":2
}
Really hope someone can advise. Starting to pull my hair out on this one.
You should use filtered query for facet seach to get exact result:
query do
filtered do
query { <search keywords> }
filter <your filter> (pass in facet values)
end
end
<facet>
...
<facet>
I was actually very close. As my tags can have multiple I needed to use terms not term ie,
def self.search(params={})
tire.search(page: params[:page], per_page: 2, load: true) do
query do
boolean do
must { string params[:name], default_operator: "AND" } if
must { term :status, 'live' }
must { term :refno, params[:refno]} if params[:refno].present?
must { terms 'tags.name.exact', params[:tag]} if params[:tag].present?
end
end
facet "tags" do
terms 'tags.name.exact'
end
# raise to_json
# raise to_curl
end
end
Thank you for your advise though imotov, Hoang.
A search request usually consists of two parts: a query and a filter. If a search request contains only a query part, facets are calculated based on the complete search result. In other words if a search result contains 10 records with the tag "Pet Friendly" and 5 records with the tag "No Pets Allowed", the facet response will contain two facets: "Pet Friendly" and "No Pets Allowed". Now let's assume a user limits results by selecting the "Pet Friendly" tag. If the "Pet Friendly" clause is added to the query part of the request, the search result will be limited to 10 records with the "Pet Friendly" tag, and only one facet will be returned: "Pet Friendly". However, if the "Pet Friendly" clause is added as a filter, the search result will be still limited to 10 records, but two facets will be returned. It happens because facets are calculated based only on the query portion of the search request and query portion didn't change - it still produces search results with 15 records with two different facets.
To answer your question, if a query returns no results (for example, user selected both "Pet Friendly" and "No Pets Allowed" tags) then results have no facets in them, so no facets are returned.