UPDATE:
Ok so I am no longer getting an error, but I am not getting any results back (even when there is only one search option).
I added the has clause to my list of indexes:
define_index do
has bar_profiles(:day), :as => :days
indexes bar_profiles.budget, :as => :budget_tags
.
.
end
So my search is:
bars = Bar.search(search_options)
with
search_options = {:conditions=>{:budget_tags=>"LOW BUDGET"}, :with=>{:days=>"thursday"}, :page=>1, :per_page=>20}
bar_profiles has rows for budget, experience, tags, day, etc.
Bar has many bar_profiles (potentially 1 for each day)
What I am trying to do is use the thinking sphinx search (in the bar model) to match the users selected criteria for budget, experience, tags against the bar_profile that has the day that matches with "today" (the current day).
This is the last thing I have to do to finish this app and I'm pulling my hair out cause I can't find any examples of how to set this up right...
If you have any insight please post it, anything helps. Thanks.
At first I thought my question was similar to this with an extra layer of abstraction, but I think my problem is with the search options not the indexing.
First off let me state that I have been having the worse time trying to fix a previous' groups implementation using thinking sphinx. I have finally got the project 90% working and the last 10% deals with being able to get the right filters to check against.
Here is a brief overview. The application has bars and bar_profiles (amongst many other tables that connect to these 2, and users, but they are not necessary to understand this issue.) There can be a bar_profile for each day of the week, for each bar.
So in the bar_profile model there is:
belongs_to :bar
and in bar there is:
has_many :bar_profiles
followed by the indexes in bar (written by the previous developer):
define_index do
# name is a reserved keyword so we need to specify it as a symbol to work
indexes :name
indexes tags
indexes bar_profiles.day, :as => :day
indexes bar_profiles.tags, :as => :daily_tags
indexes bar_profiles.budget, :as => :budget_tags
indexes bar_profiles.experience, :as => :experience_tags
set_property :delta => true
end
The issue I am having is this current implementation does not constrain the search properly to the current day. Instead of checking the current days profile for the bar, it seems to be checking against ALL the bars profiles.
So I set the current day at the start of the method:
today = (Time.now + Time.zone_offset('EST')).strftime("%A")
Then I think it needs to be something like below. I referenced this post by pat about using 'with', but I am not sure if I am messing up the syntax (because I am getting an error):
search_options = {:conditions => {}, :with => {:day=>today}, :page => 1, :per_page => algorithm.results_per_page}
Then I use these search options:
search_options[:conditions][:experience_tags] = options[:experience] unless options[:experience].blank?
budget = combine_budgets(options[:budget])
search_options[:conditions][:budget_tags] = budget unless budget.blank?
But when I try to run the search I get this in my development log:
^^^^ ERROR! Reason: index bar_core,bar_delta: no such filter attribute 'day'
Now I am pretty confused by this since the index for :day was set up as shown above... I'm not sure if 'filter attribute' is different then an index attribute. If someone could please offer some insight into this it would be greatly appreciated (looking at you #pat).
This is the final issue in this app, so if anyone can help me I would be very grateful.
Thanks,
Alan
Related
What should I do to speed up Sphinx indexing (using MySQL)?
Should I use other database, noSQL database?
Note also that delta indexing is fast, only a full reindeinx process is slow.
Please explain in details. Thx!
UPDATE:
I'm reindexing over 100.000 items and my thinking-sphinx index definition looks like this
define_index do
indexes [text, user(:nickname), user(:full_name)]
has rewrites(:id), :as => :rewrite_id
has rewrites(:user_id), :as => :rewrite_user_id
has [rewrites(:user_id), user_id], :as => :user_id_or_rewrites_user_id
has comments(:user_id), :as => :comments_user_id
has simbols(:id), :as => :simbol_ids
has followings(:follower_id), :as => :follower_id
has follows(:followable_id), :as => :followable_id
has created_at, :sortable => true
has rewrites_count, :sortable => true
has relevance, :sortable => true
has user_id
set_property :delta => :datetime
end
Building a full index is slow. How slow?
Building a delta index is fast.
This sounds normal in my experience.
noSQL databases (last I heard #Rails 2.3.5) were kind of difficult to integrate with Rails. No SQL speeds depend on your data sets and relations.
Without more information this sounds normal.
== Edit ==
Make sure you have SQL indexes on
created_at
rewrites_count
relevance
in addition to your primary keys naturally.
When working with thinking_sphinx always look at the SQL it generates in the real sphinx configuration file. Run a query analyzer against all the queries it will run. I have found you can also manipulate the queries quite a bit.
Also for one to many relationships you may need to add this:
:source => :ranged_query
It will cause the sphinx to use a separate query to gather the children rather than an outer join. It is much faster in many cases.
how about using Real Time indexes (with adjusting appropriate memory limit)
I have this scenario where I thought it would be pretty basic, but found out that I can't really achieve what I need. This is why I have this question for a thinking_sphinx's expert.
The scenario is this: I need do a search within a list of companies and only return those who has an address (there can be many address by company) which belongs to a particular city or none at all (this I can do).
I have the following models :
class Company < ActiveRecord::Base
has_many :company_addresses
define_index
indexes :name
indexes :description
indexes :keywords
end
end
and
class CompanyAddress < ActiveRecord::Base
end
The CompanyAddress has a city_id property. Without looping through all returned records from a sphinx search, is there a way to achieve the same thing more easily?
I'm using Rails 3.0.3 and thinking_sphinx.
You'll want to add an attribute pointing to the city_id values for the company:
has company_addresses.city_id, :as => :city_ids
And then you can filter on Companies belonging to a specific city:
Company.search 'foo', :with => {:city_ids => #city.id}
If you want both matching to a specific city or has no cities, that's a little trickier, as OR logic for attribute filters is more than a little tricky at best. Ideally what you want is a single attribute that contains either 0, or all city ids. Doing this depends on your database, as MySQL and Postgres functions vary.
As a rough idea, though - this might work in MySQL:
has "IF(COUNT(city_id) = 0, '0', GROUP_CONCAT(city_id SEPARATOR ',')",
:as => :city_ids, :type => :multi
Postgres is reasonably similar, though you may need to use a CASE statement instead of IF, and you'll definitely want to use a couple of functions for the group concatenation:
array_to_string(array_accum(city_id, '0')), ',')
(array_accum is provided by Thinking Sphinx, as there was no direct equivalent of GROUP_CONCAT in PostgreSQL).
Anyway, if you need this approach, and get the SQL all figured out, then your query looks something like:
Company.search 'foo', :with => {:city_ids => [0, #city.id]}
This will match on either 0 (representing no cities), or the specific city.
Finally: if you don't reference the company_addresses association anywhere in your normal fields and attributes, you'll need to force to join in your define_index:
join company_addresses
Hopefully that provides enough clues - feel free to continue the discussion here or on the Google Group.
I have model Products with columns:
name, number, description
Index defined for this model looks like this:
define_index do
indexes :name, :sortable => true
indexes :number
indexes :description
where "amount > 0"
has :price
end
Since in description can be lots of random words I want to exclude it from searching sometimes (when user clicks chceckbox 'don't search in descriptions').
I went to the sphinx page and found following:
#(name, number) *pencil* *123*
And it seems like I don't understand how sphinx works. When I execute search
*pencil* *123*
word 'pencil' is found in name and '123' is found in number and I get 1 result. But when I execute
#(name, number) *pencil* *123*
no results are found.
Is searching by columns somehow different?
You can only search on fields when using the :extended match mode - Thinking Sphinx sets this automatically if you use :conditions - but you're constructing a multi-field query yourself, hence why this isn't happening. Try this:
Product.search "#(name, number) *pencil* *123*", :match_mode => :extended
Hope this helps.
It was all about spaces :/
This works:
#(name,number) *pencil* *123*
I have implemented the full text search using Sphinx and Thinking Sphinx.
I want to add column wise search. Some thing like:-(taking an example of Stack Overflow)
Suppose you want to see actvities related to you, just type:
user:me
Then it will return a result with all the questions and answers related to piemesons.
If you type
votes:15
then it will return a result with all the questions tagged with having more than 15 votes.
And if you type
user:me votes:15
then it will return all the questions and answers belonging to you with more than 15 votes.
How can I implement this thing?
Right now my search results are based upon full text search. How can these kinds of features be included?
Any options avaliable in Sphinx or Solr or any other search engines?
:with option in thinking sphinx.
First of all, you have to define those attributes in your index definition (check out attributes section here).
has views_count, :as => :views, :type => :integer
has user.id, :as => :user, :type => :integer
Then you could search for posts like this:
Post.search '', :with => {:views => 12..maxint, :user => User.first.id}
(I am not sure if there is any more elegant possibility of giving open ranges, but 12..max_int should be enough)
Two important things:
if you want to count related objects (e.g. responses), you should use counter cache
if your "user" is polymorphic association, I recommend "CRC32(CONCAT(user_type, user_id))" instead of user.id
i got this error:
SQLite3::SQLException: no such column: apis.name: SELECT * FROM examples WHERE ("apis"."name" = 'deep')
my code
Api.find :all, :from => params[:table_name], :conditions => {:name => 'deep' }
I need to make a back end rails application which will be used by a silverlight application. one of the requirements is to fetch simple data from the database. i need to be able to query different tables with the same code.(my app has 2000 tables!)
i think it does not make sense for rails to put in "apis" in the WHERE clause. is there any speciic reason for this?
It does that so when joins are performed, the where clauses will line up with the right tables' columns. This is handy most of the time, but in your particular case causes issues.
What you could do is use the other conditions syntax, which will not add rails table names to the attributes, but still sanitize the inputs properly.
Api.find :all, :from => params[:table_name], :conditions => ['name = ?','deep']