Facet Troubles with Elasticsearch on Query - ruby-on-rails

When adding a term to my query instead of a filter I am getting 0 facets. FYI I am using the tire gem with Ruby.
Here is my model code with its mapping:
class Property < ActiveRecord::Base
include Tire::Model::Search
include Tire::Model::Callbacks
has_and_belongs_to_many :tags
mapping do
indexes :id, type: 'integer'
indexes :status
indexes :refno, type: 'integer'
indexes :name, :analyzer => 'snowball', :boost => 100
indexes :description
indexes :tags, type: 'object',
properties: {
name: { type: 'multi_field',
fields: {
name: { type: 'string', analyzer: 'snowball' },
exact: { type: 'string', index: 'not_analyzed' }
}
}
}
end
def to_indexed_json
to_json( include: {
tags: { only: [:name] },
})
end
Then here is the search method
def self.search(params={})
tire.search(page: params[:page], per_page: 2, load: true) do
query do
boolean do
must { string params[:name], default_operator: "AND" } if params[:name].present?
must { term :status, 'live' }
must { term :refno, params[:refno]} if params[:refno].present?
# must { term :tag, params[:tag]} if params[:tag].present? ## does not work either
must { term 'tags.name.exact', params[:tag]} if params[:tag].present?
end
end
facet "tags" do
terms 'tags.name.exact'
end
raise to_json
# raise to_curl
end
end
I get 0 Facets. But if I move facets to a filter ie below I get full facets.
def self.search(params={})
tire.search(page: params[:page], per_page: 2, load: true) do
query do
boolean do
must { string params[:name], default_operator: "AND" } if params[:name].present?
must { term :status, 'live' }
must { term :refno, params[:refno]} if params[:refno].present?
end
end
filter :term, 'tags.name.exact' => params[:tag] if params[:tag].present?
facet "tags" do
terms 'tags.name.exact'
end
raise to_json
# raise to_curl
end
end
While this is ok it's not want, When a facet filter is clicked I want to remove non available tags from my facet filter and update the new facet count.
If it helps here is the json for the query which works and does not.
## No Factes
{
"query":{
"bool":{
"must":[
{
"query_string":{
"query":"England",
"default_operator":"AND"
}
},
{
"term":{
"status":"live"
}
},
{
"term":{
"tags.name.exact":[
"Pet Friendly"
]
}
}
]
}
},
"facets":{
"tags":{
"terms":{
"field":"tags.name.exact",
"size":10,
"all_terms":false
}
}
},
"size":2
}
## Facets working
{
"query":{
"bool":{
"must":[
{
"query_string":{
"query":"England",
"default_operator":"AND"
}
},
{
"term":{
"status":"live"
}
}
]
}
},
"facets":{
"tags":{
"terms":{
"field":"tags.name.exact",
"size":10,
"all_terms":false
}
}
},
"filter":{
"term":{
"tags.name.exact":[
"Pet Friendly"
]
}
},
"size":2
}
Really hope someone can advise. Starting to pull my hair out on this one.

You should use filtered query for facet seach to get exact result:
query do
filtered do
query { <search keywords> }
filter <your filter> (pass in facet values)
end
end
<facet>
...
<facet>

I was actually very close. As my tags can have multiple I needed to use terms not term ie,
def self.search(params={})
tire.search(page: params[:page], per_page: 2, load: true) do
query do
boolean do
must { string params[:name], default_operator: "AND" } if
must { term :status, 'live' }
must { term :refno, params[:refno]} if params[:refno].present?
must { terms 'tags.name.exact', params[:tag]} if params[:tag].present?
end
end
facet "tags" do
terms 'tags.name.exact'
end
# raise to_json
# raise to_curl
end
end
Thank you for your advise though imotov, Hoang.

A search request usually consists of two parts: a query and a filter. If a search request contains only a query part, facets are calculated based on the complete search result. In other words if a search result contains 10 records with the tag "Pet Friendly" and 5 records with the tag "No Pets Allowed", the facet response will contain two facets: "Pet Friendly" and "No Pets Allowed". Now let's assume a user limits results by selecting the "Pet Friendly" tag. If the "Pet Friendly" clause is added to the query part of the request, the search result will be limited to 10 records with the "Pet Friendly" tag, and only one facet will be returned: "Pet Friendly". However, if the "Pet Friendly" clause is added as a filter, the search result will be still limited to 10 records, but two facets will be returned. It happens because facets are calculated based only on the query portion of the search request and query portion didn't change - it still produces search results with 15 records with two different facets.
To answer your question, if a query returns no results (for example, user selected both "Pet Friendly" and "No Pets Allowed" tags) then results have no facets in them, so no facets are returned.

Related

Exposing a new table and column to rails with elastic search & searchkick

I have added a new field to searchkick, and can get the results to return for that model but I am trying to find the parent model as a result.
People can have many tags. I can search for People just fine by any attribute that exist on that schema. I can search for Tags just fine and searchkick and elasticsearch will return the result.
I want to search by the tag name and then return the people associated with that result
search_query = if #q.present?
{
query: {
multi_match: {
query: #q.strip.downcase,
fields: %w(first_name last_name email_address phone_number address_1 name),
type: 'cross_fields',
operator: 'AND'
}
}
}
else
{
query: {
query_string: {
query: ('*'),
default_operator: 'AND'
}
},
# order: { signup_at: :desc },
# page: search_params[:page],
# per_page: (Kaminari.config.default_per_page unless request.format == :csv)
}
end
#results = Person.search search_query
#people = #results.records.where(active: true).order("people.#{sort_column}" + ' ' + sort_direction).page(search_params[:page])
This currently works fine for searching people only attributes. If I replace
#results = Tag.search search_query When inputting a tag name I get the resulting tag.
An older query was in place that worked fine, but had to be changed to allow full name searching. The old query was
query: {
query_string: {
query: (#q.strip.downcase),
default_operator: 'AND'
}
},
And that returned the associated tags with the rest of the code remaining unchanged.
Here is the search_data method that exist on the person model` # For Searchkick
def search_data
attributes.
each { |_k, v| v.downcase if v.is_a? String }.
merge({
tag: tags.map { |t| t.name.downcase },
trait: traits.map { |t| t.name.downcase },
trait_value: person_traits.map { |pt| pt.value.downcase },
question: questions.map(&:id),
answer: answers.map { |a| a.value.downcase },
last_participated: last_participation_date.to_s,
signup_at: signup_at.to_s
})
end
Please let me know if I can provide other information to help.
Since you have tags in the search_data method on Person, you can do:
Person.search("sometag", fields: [:tag])
Make sure your search_data method returns correct data with:
Person.first.search_data
And make sure you have reindexed.
Person.reindex

ActiveRelation where() to tire search

I put tire search in my model:
class Name < ActiveRecord::Base
include Tire::Model::Search
include Tire::Model::Callbacks
mapping do
indexes :name, type: 'string', analyzer: 'snowball'
indexes :lang, type: 'string'
indexes :private, type: 'boolean'
indexes :id, index: :not_analyzed, type: 'integer'
end
end
Then, when i perform:
txt = params[:search]
Name.tire.search page: page, per_page: PER_PAGE do
string txt
end
If works well, but how do i chain more search conditions like:
Name.where(private: false, lang: ['ru', 'en'], id: [1,2,3,4])
I tried to do:
#results = Name.tire.search per_page: per, page: page do
query do
boolean do
must { string txt }
must { term 'names.id', ids } unless ids.blank?
must { term 'names.private', false }
must { term 'names.lang', lang }
end
end
end
But it not returning any results..
try with:
Name.tire.search per_paga: per, page: page do
query {string txt}
filter :boolean, private: false
filter :array, lang: ['ru', 'en'] #here i'm not sure if is array or string
end
Finally found the solution.
Name.tire.search per_pag: per, page: page do
query {string 'text'}
filter :term, private: false
filter :terms, lang: ['ru', 'en']
filter :terms, id: [1,2,3,4]
end
Note the difference in "term" and "terms"(for array desired)

Mixing conditional operators with facets in Elasticsearch

I am trying to match a search query against two fields, as well as filter by facets if selected from dropdowns on the page.
When the user enters keywords it should match if found in two database fields: Title and Description. The dropdowns filter by a status, and a type.
Here is my Tire search configuration:
def self.search(params)
tire.search(load: true, page: params[:page], per_page: 25) do
query do
boolean do
should { string "title:#{params[:query]}", default_operator: "OR" } if params[:query].present?
should { string "description:#{params[:query]}", default_operator: "OR" } if params[:query].present?
must { term :status_id, params[:status_id] } if params[:status_id].present?
must { term :type_id, params[:type_id] } if params[:type_id].present?
end
end
sort { by :updated_at, "desc" } if params[:query].blank?
facet "status" do
terms :status_id
end
facet "type" do
terms :type_id
end
end
end
Indexing settings:
settings :analysis => {
:filter => {
:my_ngram => {
"type" => "nGram",
"max_gram" => 10,
"min_gram" => 3}
},
:analyzer => {
:my_analyzer => {
"type" => "custom",
"tokenizer" => "lowercase",
"filter" => ["my_ngram"]
}
}
} do
mapping do
indexes :title, boost: 10, analyzer: 'my_analyzer'
indexes :description, boost: 5, analyzer: 'my_analyzer'
indexes :status_id, :type => 'integer'
indexes :type_id, :type => 'integer'
end
end
I originally only had the title and description fields, which was working fine. I am now trying to add the ability to filter by status and type.
What is the proper way to configure this? If status is selected, it should only return records with that status. The same follows for type, and if both are selected.
Any help is appreciated.
It's not that errors occur, but the results no longer filter at all by either keywords or facets:
curl -X GET 'http://localhost:9200/projects/project/_search?load=true&size=25&pretty' -d '{"query":{"bool":{"should":[{"query_string":{"query":"title:test","default_operator":"OR"}},{"query_string":{"query":"description:test","default_operator":"OR"}}],"must":[{"term":{"status_id":{"term":"1"}}},{"term":{"type_id":{"term":"1"}}}]}},"facets":{"status":{"terms":{"field":"status_id","size":10,"all_terms":false}},"type":{"terms":{"field":"type_id","size":10,"all_terms":false}}},"size":25}'
# 2013-08-16 12:08:34:791 [200] (31 msec)
#
# {"took":31,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":0,"max_score":null,"hits":[]},"facets":{"status":{"_type":"terms","missing":0,"total":0,"other":0,"terms":[]},"type":{"_type":"terms","missing":0,"total":0,"other":0,"terms":[]}}}
If you could create the equivalent of this I think you would get the desired results. (Please excuse the lack of quotes on the JSON keys!)
{
query: {
multi_match: {
query: "test",
fields: ["title", "description"]
}
},
filter: {
and: [
{
term: { status_id: 123 }
},
{
term: { type_id: 456 }
}
]
},
facets: {
type: {
terms: {
field: "type_id",
size: 10
}
},
status: {
terms: {
field: "status_id",
size: 10
}
}
}
}
Update
I don't know tire but will try to write something!
def self.search(params)
tire.search(load: true, page: params[:page], per_page: 25) do
query do
boolean do
should { match :title params[:query] } if params[:query].present?
should { match :description params[:query] } if params[:query].present?
end
end
sort { by :updated_at, "desc" } if params[:query].blank?
filter :and, { :term => { :status_id => params[:status_id] } } if params[:status_id].present?
{ :term => { :type_id => params[:type_id] } } if params[:type_id].present?
end
end
You will probably have to fix the ruby, but a few things to note. Match queries are the recommended default string search, they are faster than query_string ones (though you have slightly less control). Also

How to filter search by attribute only if it exists using ElasticSearch and Tire?

Right now I wrote
Tire.search INDEX_NAME do
query do
filtered do
query { string term }
filter :or, { missing: { field: :app_id } },
{ terms: { app_id: app_ids } }
end
end
end.results.to_a
Well returning items that either have no app_id or one that matches your terms sounds like a job for an or filter - I'd try
filter :or, [
{:not => {:exists => {:field => :app_id}}},
{:terms => {:app_id => app_ids}}
]

Tire does not find partial word (search on 2 fields)

What I want to do:
I have a model 'Item' with 2 fields I want elasticsearch to search on: title and description.
I want the search to find partial words, ex: bicycl should match against bicycle, bicycles, etc...
Current situation:
The search only shows perfect matches
Here is what I have right now in my Item model:
include Tire::Model::Search
include Tire::Model::Callbacks
class << self
def search_index
Tire.index(Item.index_name)
end
end
settings :analysis => {
:filter => {
:my_ngram => {
"type" => "nGram",
"max_gram" => 10,
"min_gram" => 3 }
},
:analyzer => {
:my_analyzer => {
"type" => "custom",
"tokenizer" => "standard",
"filter" => ["my_ngram"]
}
}
} do
mapping do
indexes :title, boost: 10, analyzer: 'my_analyzer'
indexes :description, boost: 5, analyzer: 'my_analyzer'
end
end
def self.search(query_string)
tire.search(load: true) do
if query_string.present?
query do
string query_string, default_operator: "AND"
end
end
end
end
When you do...
string query_string, default_operator: "AND"
... you're actually searching the magic _all field.
I'm pretty sure that you need to specifically search for the field analyzed with the ngram filter for this to work.
should { string "title:#{query_string}", default_operator: "OR" }
should { string "description:#{query_string}", default_operator: "OR" }
for instance.

Resources