I am using elasticsearch-rails and elasticsearch-model gem for searching words in my rails app.
I want to make my search case insensitive and must be independent of pluralization. I researched a lot on google but got a hunch on how to do it using analyzer but not success. So I had to post a new question.
Here is my model where I want to search to take place
require 'elasticsearch/model'
class Article < ActiveRecord::Base
include Elasticsearch::Model
include Elasticsearch::Model::Callbacks
settings index: { number_of_shards: 1 } do
mappings dynamic: 'false' do
indexes :title, analyzer: 'english', index_options: 'offsets'
indexes :text, analyzer: 'english'
end
end
def self.search(query)
__elasticsearch__.search(
{
query: {
multi_match: {
query: query,
fields: ['title^10', 'text']
}
},
highlight: {
pre_tags: ['<em>'],
post_tags: ['</em>'],
fields: {
title: {},
text: {}
}
}
}
)
end
end
# Delete the previous articles index in Elasticsearch
Article.__elasticsearch__.client.indices.delete index: Article.index_name rescue nil
# Create the new index with the new mapping
Article.__elasticsearch__.client.indices.create \
index: Article.index_name,
body: { settings: Article.settings.to_hash, mappings: Article.mappings.to_hash }
# Index all article records from the DB to Elasticsearch
Article.import
#Article.import force: true
My questions are how do I do search word? tshirt, T-shirt, Tshirts should all match.
Any links for further research is also helpful.
Thank you
One of the method for searching in a table is:
User.where("firstname ILIKE ? OR lastname ILIKE ?", "%#{params[q]}%", "%#{params[q]}%" );
where
User is Model
params[q] is search query parameter.
Related
So let's say I enter the text "Ebook" for searching and get the following results:
Ebook for Rails
Elastic Ebook
Booking for ebook.
Now I select "Elastic Ebook" from the list. Let's say that several other users input Ebook for searching and most of them select "Elastic Ebook" from the list. This clearly indicates the popularity for Elastic Ebook for the key word "Ebook". Hence in further searches, I would want Elastic Ebook to have higher priority and appear at the top.
Elastic Ebook
Ebook for Rails
Booking for an ebook.
Is there any way to achieve this.
My code document.rb :
require 'elasticsearch/model'
class Document < ApplicationRecord
include Elasticsearch::Model
include Elasticsearch::Model::Callbacks
belongs_to :user
Document.import force: true
def self.search(query)
__elasticsearch__.search({
query: {
multi_match: {
query: query,
fields: ['name^10', 'service'],
fuzziness: "AUTO"
}
}
})
end
settings index: {
"number_of_shards": 1,
analysis: {
analyzer: {
edge_ngram_analyzer: { type: "custom", tokenizer: "standard", filter:
["lowercase", "edge_ngram_filter", "stop", "kstem" ] },
}
},
filter: {
edge_ngram_filter: { type: "edgeNGram", min_gram: "3",
max_gram: "20" }
}
} do
mapping do
indexes :name, type: "string", analyzer: "edge_ngram_analyzer"
indexes :service, type: "string", analyzer: "edge_ngram_analyzer"
end
end
end
search controller code:
def search
if params[:query].nil?
#documents = []
else
#documents = Document.search params[:query]
end
end
There is no simple solution for this. Elasticsearch does not know what your users choose in the application.
In the simplest approach it would have to be your responsibility to tell elasticesearch what is popular (just after user searched and clicked on one of the results from the list), and store this information in your documents by updating them. Technically updating means deleting and inserting the document again but it can be done with Update API and scripts.
Having this, you can use function_score queries to search with boosting-by-popularity:
{
"query": {
"function_score": {
"query": {
"multi_match": {
"query": "Rental",
"fields": [ "name"]
}
},
"field_value_factor": {
"field": "popularity",
"modifier": "..."
}
}
}
}
where popularity is the field you update.
It is also possible to store the information about popluratity in separate documents and extend the query to combine these two sources of information.
There are also plugins for Learning to Rank integration but I've never used them so it's hard to tell how complicated it is to integrate.
i am using elasticsearch with my rails application(railscasts #306,#307) now i want to know how i can get id of a product on click of any autosuggestion, so that i can find that product and can show user only that perticular product record. i am using tire gem and my model query is as follow:
include Tire::Model::Search
include Tire::Model::Callbacks
settings index: {
number_of_shards: 1,
number_of_replicas: 0,
} do
mapping do
indexes :name, :type => 'string'
end
end
def self.search(params)
if params[:query].present?
tire.search(load: true) do
query { string "#{params[:query]}*"}
end
else
tire.search(load: true) do
query { string "*#{params["term"]}*"}
end
end
end
Please help me as soon as possible and thanks in advance.
This seems like a really easy issue, but everything I've tried from other solutions and websites is not working. I have three fields I do not want indexed or queried--:p_s, :gender, and :part_of_speech--but elasticsearch is still returning values from those fields even though I don't specify that they should be indexed or queried. About halfway down, this article says to say no to indexing, but they don't indicate where this would occur.
Term Controller:
def search
#terms = Term.search(params[:query]).page(params[:page])
end
Model:
require 'elasticsearch/model'
class Term < ActiveRecord::Base
include Elasticsearch::Model
include Elasticsearch::Model::Callbacks
settings index: { number_of_shards: 1, number_of_replicas: 0 },
do
mappings dynamic: 'false' do
indexes :id, index: :not_analyzed
indexes :name, analyzer: :spanish_analyzer
indexes :definition, analyzer: :combined_analyzer
indexes :etymology1, analyzer: :combined_analyzer
indexes :etymology2, analyzer: :combined_analyzer
indexes :uses, analyzer: :combined_analyzer
indexes :notes1, analyzer: :combined_analyzer
indexes :notes2, analyzer: :combined_analyzer
end
end
def self.search(query)
__elasticsearch__.search(
{
query: {
multi_match: {
query: query,
fields: ['name^7', 'definition^6', 'etymology1^5', 'etymology2^4', 'uses^3', 'notes1^2', 'notes2^1'],
operator: 'and'
}
}
}
)
end
end
# Delete the previous term index in Elasticsearch
Term.__elasticsearch__.client.indices.delete index: Term.index_name rescue nil
# Create the new index with the new mapping
Term.__elasticsearch__.client.indices.create \
index: Term.index_name,
body: { settings: Term.settings.to_hash, mappings: Term.mappings.to_hash }
# Index all term records from the DB to Elasticsearch
Term.import(force: true)
To mark a field as non-indexed use this:
mappings dynamic: 'false' do
...
indexes :p_s, index: :no
indexes :gender, index: :no
indexes :part_of_speech, index: :no
...
end
By default elasticsearch returns all document fields under "_source" key. To only get specific fields you can either specify fields array on the top query level like this
def self.search(query)
__elasticsearch__.search(
{
query: {
multi_match: {
query: query,
fields: ['name^7', 'definition^6', 'etymology1^5', 'etymology2^4', 'uses^3', 'notes1^2', 'notes2^1'],
operator: 'and'
}
},
fields: ['name', 'definition', 'etymology1', 'etymology2', 'uses', 'notes1', 'notes2']
}
)
end
or filter "_source"
def self.search(query)
__elasticsearch__.search(
{
query: {
multi_match: {
query: query,
fields: ['name^7', 'definition^6', 'etymology1^5', 'etymology2^4', 'uses^3', 'notes1^2', 'notes2^1'],
operator: 'and'
}
},
'_source': ['name', 'definition', 'etymology1', 'etymology2', 'uses', 'notes1', 'notes2']
}
)
end
See Elasticsearch source filtering docs for more.
When using multi_match clause, the inner fields element specifies the fields to run the search on and, optionally, the boost like in your example. The outer fields or '_source' clause in turn determines which fields to return and this is the one you're after.
To have a better visibility into what's going on while debugging elasticsearch queries, use a tool like Sense. When you get the result you want it may be much easier to transfer the query to ruby code than vice versa.
I think using the included elasticsearch methods makes a lot of sense. However, in my own case, in my model I did something like this, modified for your own case:
def as_indexed_json
as_json(only: [:id, :name, :definition, :etymology1, :etymology2, :uses, :notes1, :notes2])
end
This should work because by default Elasticsearch would call the as_indexed_json method in your model to get the data it needs to index.
The data structure is a Post which has_many Post_text. Following a great example at https://github.com/elasticsearch/elasticsearch-rails/blob/master/elasticsearch-model/examples/activerecord_associations.rb. I have defined the mapping as the following:
include SearchableModule
mapping do
indexes :country
indexes :post_texts do
indexes :subject, type: 'string', boost: 10, analyzer: 'snowball'
indexes :description, type: 'string', analyzer: 'snowball'
end
end
And of course, in the searchable_module.rb I just copy what's in the example with some changes in as_index_json():
def as_indexed_json(options={})
self.as_json(
include: { post_texts: { only: [:subject, :description]}
})
end
And things seems ok. I have re-import the data:
Post.import
Post.__elasticsearch__.
Then I try to check the result of SQL's LIKE and Elasticsearch by:
SQL LIKE:
PostText.where("subject LIKE '%Testing%' OR description LIKE '%Testing%'").each do |r|
puts r.post_id
end
There are 12 unique post_id with this approach.
Elasticsearch:
Post.search("Testing").results.count
=> 10
Is there anything I have missed? Thank you!!!!
you could try Post.search("Testing").total which should return summary number of results, in case with results.count you just count number of returned records suppose limited per_page
I have the following code and i'm trying to use ElasticSearch to query it.
It is working when i do Book.search(:q=>'Foo') but it doesn't work when i do Book.search(:author=>'Doctor'). In my database I have a entry with a name like "Barz, Foo, Doctor"
I'm not sure if I should use terms or term, in my query, because i'm breaking the name using snowball. I tried with terms and then I get an error. With term I get no results.
class Author < ActiveRecord::Base
has_many :books
end
class Book < ActiveRecord::Base
belongs_to :author
include Tire::Model::Search
include Tire::Model::Callbacks
mapping do
indexes :title,
indexes :description
indexes :author,type: 'object', properties: {
name: { type: 'multi_field',
fields: { name: { type: 'string', analyzer: 'snowball' },
exact: { type: 'string', index: 'not_analyzed' }
}
} }
end
def to_indexed_json
to_json(:include=>{:author=>{:only=>[:name]}} )
end
def self.search(params = {})
tire.search(load:true) do
query do
boolean do
should { string params[:q] } if params[:q].present?
should { term params[:author] } if params[:author].present?
end
end
filter :term, :active=>true
end
end
end
You can do like this
should { terms :author, [params[:author]]} if params[:author].present?
OR
should { term :author, params[:author]} if params[:author].present?
OR
should { string "author:#{params[:author]}"} if params[:author].present?
As #Karmi stated enter link description here
Hi, yeah, your approach seems one. Couple of things:
* unless you want to use Lucene query syntax (boosting, ranges, etc), it's maybe best to use the text query,
* yes, filters are more performant then queries, an the active=true in your example is a good fit for filters. Beware of the interplay between queries, filters and facets, though.
Your definition of the term query is incorrect, though -- it should be:
term :author, params[:author]