I have delta indexing setup for thinking sphinx on one of my models. When ever a record gets updated, the delta is being set to True, but I don't see the index getting updated with the changes made to the record. I have my sphinx configuration files updated with the delta property changes. Any idea why the delta indexing is not getting triggered?
According to the documentation after you update the database and the model, you should do this:
rake thinking_sphinx:rebuild
Maybe you've omit that step..
Related
As per my search and understanding in delta indexing when we add new records or do changes in records we need to re index sphinx to show that data otherwise it will not show.
But I check that data is updating without re indexing. So what the purpose of re indexing delta
With Thinking Sphinx, there's the distinction between a full re-index where all the indices are reprocessed (via rake ts:index and rake Ts:rebuild), and processing a single index.
When you have delta indexing enabled, it means that the delta index for a given model is automatically processed straight after the change to a record, or adding a new record. This is either done as part of the standard callback process (when using :delta => true) or via a background worker (Sidekiq, DelayedJob, etc) if you're using the appropriate delta gem for those.
All of this means that you don't need to run a full reprocessing of all indices for the change to be present - the delta index is reprocessed automatically, and the record's changes are reflected in Sphinx.
One catch worth noting is that the more changes that happen, the larger the delta index gets, and thus the slower it is to process. So, a full re-index is still required on a regular basis (hourly? daily? depends on your app) to keep delta processing times fast.
Even when no fields specified in update attributes actually changes , thinking sphinx sets delta = 1 which results in large number of unwanted queries being fired . can we do something to let sphinx know that there was no actual update.
When I run a reindexing task(rake ts:reindex), it automatically sets delta value to false.But I definitely want delta indexing working after reindexing. So I want to set the delta value back to 'true'. How can I do that??
You don't need delta indexing after your reindex as the main index will be up to date and complete. Your model should only set the delta flag to true after your next update, which is when your main index will be incomplete.
Thinking Sphinx will automatically set delta to true when you make changes to a model instance.
The only cases where this is not the case is when you're actually changing an association instance, instead of the indexed model, or you're changing the indexed model in some way which doesn't fire the callbacks. #update_attribute (note: singular) does not fire callbacks. #save and #update_attributes do.
So: how are you changing your model instances? Is delta indexing not occurring when you make those changes?
We are running thinking sphinx on a utility instance in our server cluster. It is rerunning the index every minute. But, if you make a change to a record, it disappears from search results until the index is updated (up to 1 minute).
Is Thinking Sphinx only returning rows that have updated_at times that less than their last index?
If so, how can I get db changes to update the TS on the utility instance?
Instead of re-indexing every minute try using the Delayed Deltas approach. It is designed to tide over your search results until you fully re-index.
See:
http://freelancing-god.github.com/ts/en/deltas.html
Update:
Looks like the team at sphinx is trying to solve these problems with real-time indexes:
http://sphinxsearch.com/docs/current.html#rt-indexes
Environment:
Memcached, Rails 2.2.2 + cache_money, Sphinx + thinking sphinx
The following yields stale results:
- add a record; mysql contains the correct data
- the record is probably cached in memory at this point
- re-index sphinx
- sphinx returns the proper result with the correct data
- edit the record
- the cache is invalidated properly, mysql contains the correct, updated data
- re-index sphinx again
- sphinx is now stale
Re-indexing sphinx, clearing memcached, and/or editing the questionable records all have no effect. Disabling the cache layer all together (cache_money plus memcached) also has no effect.
Does your delta query just get new, unindexed rows from your table(s), or is it grabbing every row with a modified time greater than a specified value?