Mixing conditional operators with facets in Elasticsearch - ruby-on-rails

I am trying to match a search query against two fields, as well as filter by facets if selected from dropdowns on the page.
When the user enters keywords it should match if found in two database fields: Title and Description. The dropdowns filter by a status, and a type.
Here is my Tire search configuration:
def self.search(params)
tire.search(load: true, page: params[:page], per_page: 25) do
query do
boolean do
should { string "title:#{params[:query]}", default_operator: "OR" } if params[:query].present?
should { string "description:#{params[:query]}", default_operator: "OR" } if params[:query].present?
must { term :status_id, params[:status_id] } if params[:status_id].present?
must { term :type_id, params[:type_id] } if params[:type_id].present?
end
end
sort { by :updated_at, "desc" } if params[:query].blank?
facet "status" do
terms :status_id
end
facet "type" do
terms :type_id
end
end
end
Indexing settings:
settings :analysis => {
:filter => {
:my_ngram => {
"type" => "nGram",
"max_gram" => 10,
"min_gram" => 3}
},
:analyzer => {
:my_analyzer => {
"type" => "custom",
"tokenizer" => "lowercase",
"filter" => ["my_ngram"]
}
}
} do
mapping do
indexes :title, boost: 10, analyzer: 'my_analyzer'
indexes :description, boost: 5, analyzer: 'my_analyzer'
indexes :status_id, :type => 'integer'
indexes :type_id, :type => 'integer'
end
end
I originally only had the title and description fields, which was working fine. I am now trying to add the ability to filter by status and type.
What is the proper way to configure this? If status is selected, it should only return records with that status. The same follows for type, and if both are selected.
Any help is appreciated.
It's not that errors occur, but the results no longer filter at all by either keywords or facets:
curl -X GET 'http://localhost:9200/projects/project/_search?load=true&size=25&pretty' -d '{"query":{"bool":{"should":[{"query_string":{"query":"title:test","default_operator":"OR"}},{"query_string":{"query":"description:test","default_operator":"OR"}}],"must":[{"term":{"status_id":{"term":"1"}}},{"term":{"type_id":{"term":"1"}}}]}},"facets":{"status":{"terms":{"field":"status_id","size":10,"all_terms":false}},"type":{"terms":{"field":"type_id","size":10,"all_terms":false}}},"size":25}'
# 2013-08-16 12:08:34:791 [200] (31 msec)
#
# {"took":31,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":0,"max_score":null,"hits":[]},"facets":{"status":{"_type":"terms","missing":0,"total":0,"other":0,"terms":[]},"type":{"_type":"terms","missing":0,"total":0,"other":0,"terms":[]}}}

If you could create the equivalent of this I think you would get the desired results. (Please excuse the lack of quotes on the JSON keys!)
{
query: {
multi_match: {
query: "test",
fields: ["title", "description"]
}
},
filter: {
and: [
{
term: { status_id: 123 }
},
{
term: { type_id: 456 }
}
]
},
facets: {
type: {
terms: {
field: "type_id",
size: 10
}
},
status: {
terms: {
field: "status_id",
size: 10
}
}
}
}
Update
I don't know tire but will try to write something!
def self.search(params)
tire.search(load: true, page: params[:page], per_page: 25) do
query do
boolean do
should { match :title params[:query] } if params[:query].present?
should { match :description params[:query] } if params[:query].present?
end
end
sort { by :updated_at, "desc" } if params[:query].blank?
filter :and, { :term => { :status_id => params[:status_id] } } if params[:status_id].present?
{ :term => { :type_id => params[:type_id] } } if params[:type_id].present?
end
end
You will probably have to fix the ruby, but a few things to note. Match queries are the recommended default string search, they are faster than query_string ones (though you have slightly less control). Also

Related

Trouble highlighting in elasticsearch using rails

After reading several sites (including elasticsearch's documentation) and experimenting around a lot, I'm having trouble getting highlights. I can do the basic keyword search, but it's clear I'm not grasping something. Here's my code.
Gems:
gem 'elasticsearch-model'
gem 'elasticsearch-rails'
Controller:
class TermsController < ApplicationController
def search
#terms = Term.search(params[:query]).results
end
end
Model:
require 'elasticsearch/model'
class Term < ActiveRecord::Base
include Elasticsearch::Model
include Elasticsearch::Model::Callbacks
settings index: { number_of_shards: 1, number_of_replicas: 0 } do
mappings dynamic: 'false' do
indexes :id, index: :not_analyzed
indexes :name, analyzer: 'spanish'
indexes :gender, index: :not_analyzed
indexes :part_of_speech, index: :not_analyzed
indexes :definition
indexes :etymology1
indexes :etymology2
indexes :uses
indexes :romance_cognates
indexes :notes1
indexes :notes2
indexes :quote1, analyzer: 'spanish'
indexes :quote2, analyzer: 'spanish'
end
end
def as_indexed_json(options = {})
as_json(
only: [:name, :gender, :part_of_speech, :definition, :etymology1, :etymology2, :uses, :romance_cognates, :notes1, :notes2, :quote1, :quote2]
)
end
def self.search(query)
__elasticsearch__.search(
{
query: {
multi_match: {
query: query,
fields: ['name', 'definition', 'etymology1', 'etymology2', 'uses', 'romance_cognates', 'notes1', 'notes2', 'quote1', 'quote2']
}
},
highlight: {
tags_schema: 'styled',
fields: {
:'*' => {}
}
}
}
)
end
end
# Delete the previous terms index in Elasticsearch
Term.__elasticsearch__.client.indices.delete index: Term.index_name rescue nil
# Create the new index with the new mapping
Term.__elasticsearch__.client.indices.create \
index: Term.index_name,
body: { settings: Term.settings.to_hash, mappings: Term.mappings.to_hash }
# Index all term records from the db to Elasticsearch
Term.import(force: true)
I also tried:
{
query: {
multi_match: {
query: query,
fields: ['name', 'definition', 'etymology1', 'etymology2', 'uses', 'romance_cognates', 'notes1', 'notes2', 'quote1', 'quote2']
}
},
highlight: {
fields: {
content: {'force_source': true}
}
}
}
and
{
query: {
multi_match: {
query: query,
fields: ['name', 'definition', 'etymology1', 'etymology2', 'uses', 'romance_cognates', 'notes1^5', 'notes2', 'quote1', 'quote2']
}
},
highlight: {
fields: {
content: {type: 'plain'}
}
}
}
and
{
query: {
multi_match: {
query: query,
fields: ['name', 'definition', 'etymology1', 'etymology2', 'uses', 'romance_cognates', 'notes1^5', 'notes2', 'quote1', 'quote2']
}
},
highlight: {
pre_tags: ['<tag1>']
post_tags: ['</tag1>']
fields: {
_all: {}
}
}
}
...Along with many other attempts I can't remember
It appears the key that I was missing as illustrated here is that I needed the try() method in my view template. I'm sure there's a more concise way of writing this, but a sample of my view syntax looks like this:
<%= term.try(:highlight).try(:definition) ? term.highlight.definition[0].html_safe : term.definition.html_safe %>
<%= term.try(:highlight).try(:etymology1) ? term.highlight.etymology1[0].html_safe : term.etymology1.html_safe %>

How to query nested indexes using Retire

I've got an Article model:
class Article < ActiveRecord::Base
include Tire::Model::Search
include Tire::Model::Callbacks
settings default_options do
mapping do
indexes :id, index: :not_analyzed
indexes :roles do
indexes :machine_name, analyzer: 'keyword'
end
indexes :published_at, type: 'date', include_in_all: false
end
end
end
where the default_options is:
index: { store: { type: Rails.env.test? ? :memory : :niofs },
analysis: {
analyzer: {
default: {
tokenizer: "standard",
filter: ["asciifolding", "lowercase", "snowball"],
char_filter: ["html_strip"]
}
}
}
I'm simply trying to search articles while filtering roles, but I don't have any idea how to do so. I've been trying something like that without success:
Tire.search("article") do
query { string 'foo bar baz' }
filter :nested, { path:'roles',
query: {
filtered: {
query: {
match_all: {}
},
filter: {
term:{'roles.machine_name' => ['da']}
}
}
}
}
end
This give me that error:
QueryParsingException[[development-oaciq::application-article] [nested] nested object under path [roles] is not of nested type];
After finding this question, it seems the nested filter wasn't required, it could be done like this:
Tire.search("article") do
query do
string 'foo bar baz'
term 'roles.machine_name', 'test'
end
end

How to filter search by attribute only if it exists using ElasticSearch and Tire?

Right now I wrote
Tire.search INDEX_NAME do
query do
filtered do
query { string term }
filter :or, { missing: { field: :app_id } },
{ terms: { app_id: app_ids } }
end
end
end.results.to_a
Well returning items that either have no app_id or one that matches your terms sounds like a job for an or filter - I'd try
filter :or, [
{:not => {:exists => {:field => :app_id}}},
{:terms => {:app_id => app_ids}}
]

Tire does not find partial word (search on 2 fields)

What I want to do:
I have a model 'Item' with 2 fields I want elasticsearch to search on: title and description.
I want the search to find partial words, ex: bicycl should match against bicycle, bicycles, etc...
Current situation:
The search only shows perfect matches
Here is what I have right now in my Item model:
include Tire::Model::Search
include Tire::Model::Callbacks
class << self
def search_index
Tire.index(Item.index_name)
end
end
settings :analysis => {
:filter => {
:my_ngram => {
"type" => "nGram",
"max_gram" => 10,
"min_gram" => 3 }
},
:analyzer => {
:my_analyzer => {
"type" => "custom",
"tokenizer" => "standard",
"filter" => ["my_ngram"]
}
}
} do
mapping do
indexes :title, boost: 10, analyzer: 'my_analyzer'
indexes :description, boost: 5, analyzer: 'my_analyzer'
end
end
def self.search(query_string)
tire.search(load: true) do
if query_string.present?
query do
string query_string, default_operator: "AND"
end
end
end
end
When you do...
string query_string, default_operator: "AND"
... you're actually searching the magic _all field.
I'm pretty sure that you need to specifically search for the field analyzed with the ngram filter for this to work.
should { string "title:#{query_string}", default_operator: "OR" }
should { string "description:#{query_string}", default_operator: "OR" }
for instance.

Facet Troubles with Elasticsearch on Query

When adding a term to my query instead of a filter I am getting 0 facets. FYI I am using the tire gem with Ruby.
Here is my model code with its mapping:
class Property < ActiveRecord::Base
include Tire::Model::Search
include Tire::Model::Callbacks
has_and_belongs_to_many :tags
mapping do
indexes :id, type: 'integer'
indexes :status
indexes :refno, type: 'integer'
indexes :name, :analyzer => 'snowball', :boost => 100
indexes :description
indexes :tags, type: 'object',
properties: {
name: { type: 'multi_field',
fields: {
name: { type: 'string', analyzer: 'snowball' },
exact: { type: 'string', index: 'not_analyzed' }
}
}
}
end
def to_indexed_json
to_json( include: {
tags: { only: [:name] },
})
end
Then here is the search method
def self.search(params={})
tire.search(page: params[:page], per_page: 2, load: true) do
query do
boolean do
must { string params[:name], default_operator: "AND" } if params[:name].present?
must { term :status, 'live' }
must { term :refno, params[:refno]} if params[:refno].present?
# must { term :tag, params[:tag]} if params[:tag].present? ## does not work either
must { term 'tags.name.exact', params[:tag]} if params[:tag].present?
end
end
facet "tags" do
terms 'tags.name.exact'
end
raise to_json
# raise to_curl
end
end
I get 0 Facets. But if I move facets to a filter ie below I get full facets.
def self.search(params={})
tire.search(page: params[:page], per_page: 2, load: true) do
query do
boolean do
must { string params[:name], default_operator: "AND" } if params[:name].present?
must { term :status, 'live' }
must { term :refno, params[:refno]} if params[:refno].present?
end
end
filter :term, 'tags.name.exact' => params[:tag] if params[:tag].present?
facet "tags" do
terms 'tags.name.exact'
end
raise to_json
# raise to_curl
end
end
While this is ok it's not want, When a facet filter is clicked I want to remove non available tags from my facet filter and update the new facet count.
If it helps here is the json for the query which works and does not.
## No Factes
{
"query":{
"bool":{
"must":[
{
"query_string":{
"query":"England",
"default_operator":"AND"
}
},
{
"term":{
"status":"live"
}
},
{
"term":{
"tags.name.exact":[
"Pet Friendly"
]
}
}
]
}
},
"facets":{
"tags":{
"terms":{
"field":"tags.name.exact",
"size":10,
"all_terms":false
}
}
},
"size":2
}
## Facets working
{
"query":{
"bool":{
"must":[
{
"query_string":{
"query":"England",
"default_operator":"AND"
}
},
{
"term":{
"status":"live"
}
}
]
}
},
"facets":{
"tags":{
"terms":{
"field":"tags.name.exact",
"size":10,
"all_terms":false
}
}
},
"filter":{
"term":{
"tags.name.exact":[
"Pet Friendly"
]
}
},
"size":2
}
Really hope someone can advise. Starting to pull my hair out on this one.
You should use filtered query for facet seach to get exact result:
query do
filtered do
query { <search keywords> }
filter <your filter> (pass in facet values)
end
end
<facet>
...
<facet>
I was actually very close. As my tags can have multiple I needed to use terms not term ie,
def self.search(params={})
tire.search(page: params[:page], per_page: 2, load: true) do
query do
boolean do
must { string params[:name], default_operator: "AND" } if
must { term :status, 'live' }
must { term :refno, params[:refno]} if params[:refno].present?
must { terms 'tags.name.exact', params[:tag]} if params[:tag].present?
end
end
facet "tags" do
terms 'tags.name.exact'
end
# raise to_json
# raise to_curl
end
end
Thank you for your advise though imotov, Hoang.
A search request usually consists of two parts: a query and a filter. If a search request contains only a query part, facets are calculated based on the complete search result. In other words if a search result contains 10 records with the tag "Pet Friendly" and 5 records with the tag "No Pets Allowed", the facet response will contain two facets: "Pet Friendly" and "No Pets Allowed". Now let's assume a user limits results by selecting the "Pet Friendly" tag. If the "Pet Friendly" clause is added to the query part of the request, the search result will be limited to 10 records with the "Pet Friendly" tag, and only one facet will be returned: "Pet Friendly". However, if the "Pet Friendly" clause is added as a filter, the search result will be still limited to 10 records, but two facets will be returned. It happens because facets are calculated based only on the query portion of the search request and query portion didn't change - it still produces search results with 15 records with two different facets.
To answer your question, if a query returns no results (for example, user selected both "Pet Friendly" and "No Pets Allowed" tags) then results have no facets in them, so no facets are returned.

Resources