ThinkingSphinx: OR-condition on the SQL-backed indices? - ruby-on-rails

Question

I am trying to use ThinkingSphinx in my Rails 5 project. I read an instruction at http://freelancing-gods.com/thinking-sphinx/
I need to implement the OR logic on SQL-backed indices.
Here is my class:
class Message < ApplicationRecord
belongs_to :sender, class_name: 'User', :inverse_of => :messages
belongs_to :recipient, class_name: 'User', :inverse_of => :messages
end
and its indexer:
ThinkingSphinx::Index.define :message, :with => :active_record, :delta => true do
indexes text
indexes sender.email, :as => :sender_email, :sortable => true
indexes recipient.email, :as => :recipient_email, :sortable => true
has sender_id, created_at, updated_at
has recipient_id
end
schema.rb:
create_table "messages", force: :cascade do |t|
t.integer "sender_id"
t.integer "recipient_id"
t.text "text"
t.datetime "created_at", null: false
t.datetime "updated_at", null: false
t.boolean "read", default: false
t.boolean "spam", default: false
t.boolean "delta", default: true, null: false
t.index ["recipient_id"], name: "index_messages_on_recipient_id", using: :btree
t.index ["sender_id"], name: "index_messages_on_sender_id", using: :btree
end
So I need to search only within 2 indices at once - :sender_email and :recipient_email - but ignoring indexes text.
In pseudocode I need something like this:
Message.search 'manager1#example.com' :conditions => {:sender_email => 'client1#example.com' OR :receiver_email => 'client1#example.com'}
Which means: find all the messages between 'manager1#example.com' and 'client1#example.com' (each of them could be either a sender or a receiver) - ignoring the messages containing the text with words 'manager1#example.com' or 'client1#example.com'.
Unfortunately, the docs say:
The :conditions option must be a hash, with each key a field and each value a string.
In other words, I need a conditional index set (at run-time) - but simultaneously over 2 indices (not 1 as documented).
I mean that it is a bad idea to allow only hashes as a condition - and no strings (like ActiveRecord queries do allow http://guides.rubyonrails.org/active_record_querying.html#pure-string-conditions ).
PS I would say that the ThinkingSphinx documentation http://freelancing-gods.com/thinking-sphinx/ is pretty bad and needs to be totally rewritten from scratch. I read it all and did not understand anything. It has no examples (complete examples - only partial - thus totally unclear). I even don't understand what are fields and attributes and how do they differ. Associations, conditions, etc - all is unclear. Very bad. The gem itself looks pretty good - but its documentation is awful.

Answer 1

I'm sorry to hear you've not been able to find a solution that works for you in the documentation. The challenging thing with Sphinx is that it uses the SphinxQL syntax, which is very similar to SQL, but also quite different at times - and so people often expect SQL-like behaviour.
It's also part of the challenge of maintaining this gem - I'm not sure it's wise to mimic the ActiveRecord syntax too closely, otherwise that could make things more confusing.
The key thing to note here is that you can make use of Sphinx's extended query syntax for matches to get the behaviour you're after:
Message.search :conditions => {
:sender_email => "(client1#example.com | manager1#example.com)",
:receiver_email => "(client1#example.com | manager1#example.com)"
}
This will return anything where the sender is either of the two values, and the receiver is either of the two values. Of course, this will include any messages sent from client1 to client1, or manager1 to manager1, but I'd expect that's rare and maybe not that big a problem.
One caveat to note is that # and . aren't usually treated as searchable word characters, so you may need to add them to your charset_table.
Also, given you're actually performing exact matches on the entire values of database columns, this does feel like a query that's actually better served by some database indices on the columns and using SQL instead. Sphinx (and I'd say most/all other full-text search libraries) are best suited to matching words and phrases within larger text fields.
As for the documentation… I've put a lot of effort into trying to make them useful, though I realise there's still a lot of improvement that could take place. I do have a page that outlines how fields and attributes differ - if that's not clear, feedback is definitely welcome.
Keeping documentation up-to-date requires a lot of effort in small and new projects - and Thinking Sphinx is neither of these, being 10 years old in a few months. I'm proud at how it still works well, it still supports the latest versions of Rails, and it's still actively maintained and supported. But it's open source - it's done in my (and others') spare time. It's not perfect. If you find ways to improve things, then please do contribute! The code and docs are on GitHub, and pull requests are very much welcome.