Sunspot solr doesn't indexing all models - ruby-on-rails

I have a model Post. Also I have STI model Review. Here is search config for Post:
searchable do
text :title, :content
text :username do
user.try(:username)
end
text :user_full_name do
user.try(:full_name_with_username)
end
text :user_full_name_with_username do
user.try(:full_name_with_username)
end
end
The problem is, that not all models were indexed even if I add certain model through Sunspot.index(Review.find(id)) and Sunspot.commit
After indexing, I trying to find some reviews by username :
reviews_ids = Review.search do
fulltext params[:titles_search] do
fields(:username)
end
end.results.map(&:id)
and there are not all reviews in results.
What could it be? How to debug it?

The search results from Solr are always paginated so I suspect that you're not seeing all the results from your search. By default, I believe the results are paginated to 20 results per page.
To see all the search results at once you could do something like:
search = Review.search do
fulltext params[:titles_search] do
fields(:username)
end
paginate page: 1, per_page: Review.count
end
This sets the "number of elements per page" equal to the number of total Reviews. So this should allow you to see all the search results without paging. Obviously if you have a large number of Review objects, this is going to be a huge memory hog.
A better way to do it is to work with the pagination behavior, so if you rewrite your search like this:
def search_reviews(opts = {})
options = {
page: nil,
per_page: 20,
}.merge(opts)
Review.search do
fulltext params[:titles_search] do
fields(:username)
end
paginate page: options[:page] if options[:page].present?
paginate per_page: options[:per_page] if options[:per_page].present?
end.results
end
You can call it successively with different page numbers to get all your results.

Related

Cannot get will paginate total_entries to work, rails

The problem is I would like to limit the number of results to say 30, so I put in
controller
#users = User.user_search(params[:name], params[:gender]).paginate(page: params[:page], per_page: 20, total_entries: 30)
render 'user_results'
Say I only get 1 or 2 results back, I will still get pagination for two pages. ie will paginate seems to get the result of how many times the total_entries will divide in to the per_page value. So if I set per_page: 20, total_entries: 110 it will give me six links adding the extra 10 as a page also.
My User.user_search method in the User model is
def self.user_search(a, b)
users = User.all
users = User.where('LOWER (name) LIKE LOWER(?)', "%#{a}%") if a.present?
users = users.where('LOWER (gender) LIKE LOWER(?)', "%#{b}%") if b.present?
return users
end
I have seen people with problems about using join models and will paginate but I am just using simple single model paginate. Searched lots about this. From my knowledge will paginate is passing on total_pages method to view and rendering this results. Is there a way to set a limit to my results?
The total_entries option just is a shortcut to avoid will_paginate querying the count for the results. So if you really want to limit the number of results shown in will paginate, but not mess up with the pagination results when there are only a few results you can do something like this:
total_records = User.user_search(params[:name], params[:gender]).count
total_entries = total_records > 30 ? 30 : total_records
#users = User.user_search(params[:name], params[:gender]).paginate(page: params[:page], per_page: 20, total_entries: total_entries)
render 'user_results'
You might think that this adds an additional query, but will_paginate was still going to do the count query in order to do its logic, here we're just overriding the total_entries by setting a max number of records to show.
Note however that by doing this you'll still get more records in the last page (if the result of dividing the total_records between the pages is not exact), or it might even be possible to ask for a higher page number and still get the 'hidden' results.
If you really need to avoid showing results over a certain number, you'll be forced to use a subquery like this:
#users = User.where(id: User.user_search(params[:name], params[:gender]).limit(30).map(&:id)).paginate(page: params[:page], per_page: 20)
However, this might cause issues if instead of 30 you use a very large number.
Previously I was suggesting using ActiveRecord::QueryMethods.limit, on the search result, but will_paginate overwrites this logic when paginating.

Random record selection and pagination in rails 4

I need to select random 5 records in sqlite3 and paginate them.
def test
add_breadcrumb "Age Test", ages_test_url
#posts = Age.limit(5).order('RANDOM()')
#posts = #posts.page(params[:page]).per_page(2)
end
The above code displays all the record but I need only 5.
Try alternative syntax:
Age.paginate(:page => params[:page], :per_page => 2).order('RANDOM()').limit(5)
However, what is the point of paginating a random set of elements. Every time a user visits the first "page" s/he will see elements other than s/he saw the first. The idea of paginating a randomized elements doesn't seem logical.

Using sunspot to search multiple fields of different types on same model

I am adding search to my rails app with the sunspot gem and I would like to be able to search for transactions by id, amount, or description. Searching by a single attribute works fine, but when I add multiple with or fulltext calls in the search block I get no results returned. I found I can wrap the with calls in a any_of block, but including a fulltext causes a undefined method 'fulltext' for #<Sunspot::DSL::Scope:0x007fb6519c13a0> error.
Search returns the correct results when I search only on 1 attribute, meaning I only have 1 with or 1 fulltext in the any_of block. So I am to search by id, amount, and description invidually. Meaning if there is a transaction with id 213, searching for 213 returns the transaction with id 213. If I search for $4.25, then the results returns every transaction with amount $4.25. If I search for 'Starbucks', then I get every transaction with 'Starbucks' in the description. If I have multiple with or fulltext inside the anyblock I do not get any results returned.
What am I missing?
I have a transaction model like so:
class Transaction < ActiveRecord::Base
...
searchable do
text :description
integer :id
float :amount
end
...
end
And an action in the controller like so:
def search
#search = Transaction.search do
any_of do
with(:amount, params[:search])
with(:id, params[:search])
fulltext(params[:search])
end
end
#transactions = #search.results
end
Sunspot is not intended to search non text fields. Other fields types date/integer/etc.. can be used to scope the search prior to the fulltext search.
As you have posed the question it is not possible with sunspot.
In this example you can see how the float field amount is used to scope prior to the fulltext search.
def search
#search = Transaction.search do
with(:amount).greater_than(params[:amount])
fulltext params[:search] do
fields :description
end
end.results
end
If you wanted to search non text value, you would need to change them to text values first; I can see in some cases where this would be valuable for searching, such as if you had a unique numeric userid.
So bigtunacan is right. It is not possible to search on non text fields. The documentation says:
text fields will be full-text searchable. Other fields (e.g., integer
and string) can be used to scope queries.
But to make this work you can pass a block to the text method and sunspot will index the result of the block. So I pass in all the fields I want to search on in a string.
So in my transaction model I have:
# we have to create an alias because sunspot uses 'id' for something
alias_attribute :transaction_id, :id
searchable do
text :transaction do
"#{description} #{amount} #{transaction_id}"
end
end
And in my transaction controller I have:
def search
search = Transaction.search do
fulltext(params[:search])
end
#transactions = search.results
end
So now I can search by description, id, or amount.

Sunspot Solr index time boost

I try to use document boost on index time, but it seems, that it hasn't any effect. I've set up my model for Sunspot like
Spree::Product.class_eval do
searchable :auto_index => true, :auto_remove => true do
text :name, :boost => 2.0, stored: true
text :description, :boost => 1.2, stored: false
boost { boost_value }
end
end
The boost_value field is a field in the database, where a user can change the boost in the frontend. It gets stored at index time (either the first time I build the index, or when a product is updated). I have about 3600 products in my database, with a default boost_valueof 1.0. Two of the products got different boost_values, one with 5.0 and the other with 2.0.
However, If I just want to retrieve all products from Solr, the document boost seems to have no effect on the order or the score:
solr = ::Sunspot.new_search(Spree::Product) do |query|
query.order_by("score", "desc")
query.paginate(page: 1, per_page: Spree::Product.count)
end
solr.execute
solr.results.first
The Solr query itself looks like this:
http://localhost:8982/solr/collection1/select?sort=score+desc&start=0&q=*:*&wt=xml&fq=type:Spree\:\:Product&rows=3600&debugQuery=true
I've appended a debugQuery=true at the end, to see what the scores are. But there are no scores shown.
The same things happens, when I search for a term. For examle, I have 2 products that have a unique string testtest inside the name field. When I search for this term, the document boost has no effect on the order.
So my questions are:
Can per document index time boosting be used based on a database field?
Does the document boost has any effect for q=*:*?
How can I debug this?
Or do I have to specify, that solr should involve the document boost?
In solr, the boosts only apply to text searches, so it applies only if you do a fulltext search.
Something like this:
solr = ::Sunspot.new_search(Spree::Product) do |query|
fulltext 'somesearch'
query.order_by("score", "desc") # I think this isn't necesary
query.paginate(page: 1, per_page: Spree::Product.count)
end
If you want to boost certain products more than others:
solr = ::Sunspot.new_search(Spree::Product) do |query|
fulltext 'somesearch' do
boost(2.0) { with(:featured, true) }
end
query.paginate(page: 1, per_page: Spree::Product.count)
end
As you see, this is much powerfull than boosting at index time, and you could put different boostings for different conditions, all at query time with no need of reindexing if you want to change the boost or the conditions.

rails tire elasticsearch weird error

I have indexed a Car model with one car record mercedes benz in the database. If I search for the word benz I get an error:
ActiveRecord::RecordNotFound in CarsController#index
Couldn't find all Cars with IDs (1, 3) (found 1 results, but was looking for 2)
If I search for hello I get:
Couldn't find Car with id=2
Other random search terms work returning accurate results.
So it's basically random errors generated by random search terms. What could be the cause of this?
Controller:
def index
if params[:query].present?
#cars = Car.search(params)
else
#cars = Car.paginate(:page => params[:page], :per_page => 10)
end
end
Model:
def self.search(params)
tire.search(load: true, page: params[:page], per_page: 10) do |s|
s.query { string params[:query]} if params[:query].present?
end
end
This happens because, you are using the load => true option to load the search results from database. The activerecord seems to be missing in DB, but the elasticsearch index contains the same document.
Reindex is not always the solution IMHO.
The best solution is to delete the document when it is deleted in db. You can use the after_destroy callback for this.
Tire remove api is used to remove a single document from index.
Tire.index('my-index-name').remove('my-document-type', 'my-document-id')
Reference: https://github.com/karmi/tire/issues/43

Resources