I' trying to figure out what would be the best way to do a multi table search with elastic.co.
In particular, I was wondering if I could add more indexes to this search method.
Chapter.rb
def self.search(params)
fields = [:title, :description, :content ]
**tables** = [Chapter.index_name, Book.index_name]
tire.search(**tables**, {load: true,page: params[:page], per_page: 5}) do
query do
boolean do
must { string params[:q], default_operator: "AND" } if params[:q].present?
end
end
highlight *fields, :options => { :tag => '<strong>' }
end
The above example works without the Tables. How to make it work with the tables ?
If you're adding more indexes then you are moving away from it being a model-centric search. That's probably fine as I guess you'll be handling the search results differently on account of them being from different indexes.
In which case I think you can do:
Tire.search([Chapter.index_name, Book.index_name],
page: params[:page],
... etc ...
) do
query do
... etc ...
end
end
It does mean that you won't be able to do stuff like load: true because you've moved outside of knowing what model to load the results for.
From digging around in the code (here) it looks like you might be able to specify multiple indexes even for a model-centric search. Something like:
tire.search({
index: [Chapter.index_name, Book.index_name],
load: true,
... etc ...
I haven't tried it though and I'm doubtful as to whether it will work - again because of not being able to load the results into a specific model once multiple indexes are involved.
Related
I am trying to implement solr search into one of my rails project. My problem statement is to be able to search upon my models and show relevant results along with auto suggest. Could someone help me to complete it properly. At the moment I am trying to use sunspot solr although its not working for me as expected. I can see some indexing has been created in my project but the search is not working. Also there is no gem for auto suggest. Below are some snippets of my code. And yes I have gone through other links for solr and it did not solve my problem.
Category Model
class Category < ActiveRecord::Base
attr_accessible :name, :parent_category_id, :image, :image_file_name, :image_content_type, :image_file_size
has_and_belongs_to_many :events, :join_table => :categories_events
has_attached_file :image
searchable do
text :name
end
end
My Controller
if params.has_key?(:category)
puts "Inside Index Search"
puts params[:category]
#search = Category.ransack do
fulltext params[:category]
end
#category = #search.result.first
I am getting the results into #category and using it in my view to display.
Thanks in advance. I really appreciate your help. :-)
What does the method Category.ransack do ? I assume that you are using sunspot gem to integrate Solr into your project. If so use Category.search to search you index. Then use
#search.results
not
#search.result
So your code should look like this:
#search = Category.search do
fulltext params[:category]
end
#category = #search.results.first
You can also write it shorter:
#category = Category.search do
fulltext params[:category]
end.results.first
Remember that before you can use index in Solr, you have run:
rake sunspot:solr:reindex
If you are looking for information about implementing autocomplete on Solr, please read this article: http://olgagr.github.io/ruby/solr/how-to-implement-autocomplete-with-solr-and-ruby-on-rails
Some time ago I also struggled with this topic.
I solved it. There was a conflict between Solr and Ransack gem and that's why I could not get the resultset. It needed the below changes and its working now.
#search = Sunspot.search(Category) do
fulltext params[:search]
end
#category = #search.results
EDIT
My answer might not seem complete so just adding the code snippet that worked for me.
My Model
searchable do
text :name, :as => :name_textp
text :description, :as => :description_textp
text :category_strings, :as => :category_strings_textp
string :get_valid_dates, :multiple => true
integer :category_ids, :multiple => true
boolean :active
boolean :company_display
end
My Controller
search_results = Sunspot.search(Event) do
fulltext current_search
with(:category_ids, [ id ])
with(:active, true)
with(:company_display, true)
paginate(:page => #current_page, :per_page => page_size)
end
Hope it helps.
It conflicts of Solr & Ransack method search.
So please user solr_search instead of search
ex:
search = Product.solr_search do
fulltext 'random'
end
search.results
I'm trying index and search by email using Tire and elasticsearch.
The problem is that if I search for: "something#example.com". I get strange results because of # and . symbols. I "solved" by hacking the query string and adding "email:" before a string I suspect is a string. If I don't do that, when searching "something#example.com", I would get results as "something#gmail.com" or "asd#example.com".
include Tire::Model::Search
include Tire::Model::Callbacks
settings :analysis =>{
:analyzer => {
:whole_email => {
'tokenizer' => 'uax_url_email'
}
}
} do
mapping do
indexes :id
indexes :email, :analyzer => 'whole_email', :boost => 10
end
end
def self.search(params)
params[:query] = params[:query].split(" ").map { |x| x =~ EMAIL_REGEXP ? "email:#{x}" : x }.join(" ")
tire.search(load: {:include => {'event' => 'organizer'}}, page: params[:page], per_page: params[:per_page] || 10) do
query do
boolean do
must { string params[:query] } if params[:query].present?
must { term :event_id, params[:event_id] } if params[:event_id].present?
end
end
sort do
by :id, 'desc'
end
end
end
def to_indexed_json
self.to_json
end
When searching with "email:" the analyzer works perfectly but without it, it search that string in email without the specified analyzer, getting lots of undesired results.
I think your issue is to do with the _all field. By default, all fields get indexed twice, once under their field name, and again, using a different analyzer, in the _all field.
If you send a query without specifying which field you are searching in, then it will be executed against the _all field. When you index your doc, the email fields content is indexed again under the _all field (to stop this set include_in_all: false in your mapping) where they are tokenized the standard way (split on # and .). This means that unguided queries will give strange results.
The way I would fix this is to use a term query for the emails and make sure to specify the field to search on. A term query is faster as it doesn't have a query parsing step the query_string query has (which is why when you prefix the string with "email:" it goes to the right field, that's the query parser working). Also you don't need to specify a custom analyzer unless you are indexing a field that contains both free text and urls and emails. If the field only contains emails then just set index: not_analyzed and it will remain a single token. (You might want to have a custom analyzer that lowercases the email though.)
Make your search query like this:
"term": {
"email": "example#domain.com"
}
Good luck!
Add the field to _all and try search with adding escape character(\) to special characters of emailid.
example:something\#example\.com
I am using rails and for search I am using Tire and elasticsearch. I have a string type field which in some records have value and in some records is nil.
I'd like to sort and show last, all the records that have null value in this field. As I see in this issue https://github.com/elasticsearch/elasticsearch/issues/896 in the current version this can't be possible through sort and elasticsearch.
Is there a workaround with rails? I am trying to do it using two searches and using filters like the following example:
filter :not, :missing => { :field => :video_url } if params[:video].present?
filter :missing, { :field => :video_url } if params[:video].blank?
But it didn't work (I can't understand why until now, I'll continue debugging).
Another idea is to create two methods with the specific fields. Any other solution/idea?
Update 2/2/2013
I finally did it like the following:
if video == "yes"
filter :not, :missing => { :field => :video_url }
elsif video == "no"
filter :missing, { :field => :video_url }
end
And I am passing the video parameter by my own. I am sorting and boosting the search but additionally I want all the objects that hasn't got video_url field, to appear at the bottom no matter how relevant they are. Indeed I don't need to sort by this field, just to show last the nil value fields.
So to solve this I am calling two times the search and with the addition of the code above, it works like a charm.
Just for completeness, my search method is the following:
def self.search(params, video = nil)
tire.search do
query do
boolean do
must { string params[:query], default_operator: "AND" } if params[:query].present?
must { term :active, true }
end
end
sort { by :update_ad => "desc" } unless params[:query].present?
facet "categories" do
terms :category_id
end
if video == "yes"
filter :not, :missing => { :field => :video_url }
elsif video == "no"
filter :missing, { :field => :video_url }
end
end
end
If you don't pass the video param, it won't apply any filter. In my mapping, I have set the boost, analyzers etc.
Thank you
First, the Elasticsearch issue you're linking to is still open and is only a feature suggestion.
Second, just as a note, are you really sure you want to sort as opposed to boost the score of certain records?
Third, if you indeed do want to sort on this field, the easiest way is to just index the field with some value which comes last ("ZZZ", weird Unicode chars, you get the picture). You probably don't want to do this by default, so it's a good idea to use the multi_field feature. Of course, you have to reindex your corpus to pick up the new settings.
Lastly, it is possible to sort by a script (see documentation), but it has the usual and obvious performance impact.
My trouble with this snippet is that it's returning an integer -1 which means a universal Message that is shared in other accounts as well.
def build_results
search = Sunspot.new_search(Message) do
any_of do
member.lists.map { |list| with :enterprise_list_id, list.search_id }
end
How can I add on to this statement to query all Message's with -1 as a search_id but scope it so that it belongs exclusively to member.account ?
I'm trying to scope it as so :
searchable :include => :repliable do
integer :account_id do
repliable.try(:account_id)
end
and..
def build_results
search = Sunspot.new_search(Message) do
with :account_id, member.account_id
But this returns nothing even though I know for a fact that there are search results that should be returned because they share a commont account_id.
If I understood your question correctly then you just need to add this statements in your Message searchable block.
searchable do
integer :some_search_ids, :multiple => true do
member.lists.map { |list| list.search_id} if member.present?
end
integer :member_account_id do
member.account_id if member.present?
end
end
then,
def build_results
search = Sunspot.new_search(Message) do
with(:some_search_ids, some_list_search_id)
with(:member_account_id, some_member_account_id)
end
end
I am still doing something wrong.
Could somebody pls help me?
I want to create a custom analyzer with ascii filter in Rails + Mongoid.
I have a simple model product which has field name.
class Product
include Mongoid::Document
field :name
settings analysis: {
analyser: {
ascii: {
type: 'custom',
tokenizer: 'whitespace',
filter: ['lowercase','asciifolding']
}
}
}
mapping do
indexes :name, analyzer: 'ascii'
end
end
Product.create(name:"svíčka")
Product.search(q:"svíčka").count #1
Product.search(q:"svicka").count #0 can't find - expected 1
Product.create(name:"svicka")
Product.search(q:"svíčka").count #0 can't find - expected 1
Product.search(q:"svicka").count #1
And when I check the indexes with elasticsearch-head I expected that the index is stored without accents like this "svicka", but the index looks like this "Svíčka".
What am I doing wrong?
When I check it with API it looks OK:
curl -XGET 'localhost:9200/_analyze?tokenizer=whitespace&filters=asciifolding' -d 'svíčka'
{"tokens":[{"token":"svicka","start_offset":0,"end_offset":6,"type":"word","position":1}]}
http://localhost:9200/development_imango_products/_mapping
{"development_imango_products":{"product":{"properties":{"name":{"type":"string","analyzer":"ascii"}}}}}
curl -XGET 'localhost:9200/development_imango_products/_analyze?field=name' -d 'svíčka'
{"tokens":[{"token":"svicka","start_offset":0,"end_offset":6,"type":"word","position":1}]}
You can check how you are actually indexing your document using the analyze api.
You need also to take into account that there's a difference between what you index and what you store. What you store is returned when you query, and it is exactly what you send to elasticsearch, while what you index determines what documents you get back while querying.
Using the asciifolding is a good choice for you usecase, it should return results either query ing for svíčka or svicka. I guess there's just a typo in your settings: analyser should be analyzer. Probably that analyzer is not being used as you'd expect.
UPDATE
Given your comment you didn't solve the problem yet. Can you check what your mapping looks like (localhost:9200/index_name/_mapping)? The way you're using the analyze api is not that useful since you're manually providing the text analysis chain, but that doesn't mean that chain is applied as you'd expect to your field. Better if you provide the name of the field like this:
curl -XGET 'localhost:9200/index_name/_analyze?field=field_name' -d 'svíčka'
That way the analyze api will rely on the actual mapping for that field.
UPDATE 2
After you made sure that the mapping is correctly submitted and everything looks fine, I noticed you're not specifying the field that you want to to query. If you don't specify it you're querying the _all special field, which contains by default all the field that you're indexing, and uses by default the StandardAnalyzer. You should use the following query: name:svíčka.
elasticsearch needs settings and mapping in a single api call. I am not sure if its mentioned in tire docs, but I faced a similar problem, using both settings and mapping when setting up tire. Following should work:
class Product
include Mongoid::Document
# include tire stuff
field :name
settings(self.tire_settings) do
mapping do
indexes :name, analyzer: 'ascii'
end
end
# this method is just created for readablity,
# settings hash can also be passed directly
def self.tire_settings
{
analysis: {
analyzer: {
ascii: {
type: 'custom',
tokenizer: 'whitespace',
filter: ['lowercase','asciifolding']
}
}
}
}
end
end
Your notation for settings/mappings is incorrect, as #rubish suggests, check documentation in https://github.com/karmi/tire/blob/master/lib/tire/model/indexing.rb (no question the docs should be better)
Always, always, always check the mapping of the index to see if your desired mapping has been applied.
Use the Explain API, as #javanna suggests, to check how your analyzing chain works quickly, without having to store documents, check results, etc.
Please note that It is very important to add two lines in a model to make it searchable through Tire. Your model should look like
class Model
include Mongoid::Document
include Mongoid::Timestamps
include Tire::Model::Search
include Tire::Model::Callbacks
field :filed_name, type: String
index_name "#{Tire::Model::Search.index_prefix}model_name"
settings :analysis => {
:analyzer => {
"project_lowercase_analyzer" => {
"tokenizer" => "keyword",
"filter" => ["lowercase"],
"type" => "custom"
}
},
} do
mapping do
indexes :field_name, :boost => 10, :type => 'String', :analyzer => 'standard', :filter => ['standard', 'lowercase','keyword']
end
end
def self.search( params = {} )
query = params[:search-text_field_name_from_form]
Model.tire.search(load: true, page: params[:page], per_page: 5) do
query { string query, default_operator: "AND" } if query.present?
end
end
You can change the index_name(should be unique) and the analyzer
And Your controller would be like
def method_name
#results = Model.search( params ).results
end
You can use #results in your view. Hope this may help you.