Data starting with same words in sunspot rails - ruby-on-rails

My controller is
#search = Sunspot.search(User) do
fulltext params[:search]
end
#search_products = #search.results
I want it also match those name which start with search params

create a new type in your solr schema file (solr/conf/schame.xml). Add this to your schema.xml under types tag :
<fieldType class="solr.TextField" name="text_pre" positionIncrementGap="100">
<analyzer type="index">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.EdgeNGramFilterFactory" minGramSize="1" maxGramSize="10" />
</analyzer>
<analyzer type="query">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>
Here minGramSize is the minimum number of characters and maxGramSize is the maximum number of characters to start with that can be searched.
For example, minGramSize=8 would mean that a word like Kapil Kumar Yadav can be searched with all these substring => "K", "Ka", "Kap", "Kapi", "Kapil", "Kapil ", "Kapil K", "Kapil Ku", but will not be searched with search params having length geater then 8 characters, so its suggested to keep a value for maxGramSize that fits all your fields that have been indexed.
Now create a new dynamic field definition in same solr schama file, Add this to your schema.xml under fields tag:
<dynamicField name="*_textp" stored="false" type="text_pre" multiValued="true" indexed="true"/>
Finally onfigure all text fields in User class to index into your newly created dynamic field using as option like:
class User < ActiveRecord::Base
searchable do
text :name, as: :name_textp
text :address, as: :address_textp
# etc.
end
end
I found this here Matching substrings in fulltext search

Related

How to return search results in Sunspot Solr (rails) where not all tokens are present?

I'm running Sunspot Solr in my rails app. I am using it to enable a user to search for different "articles" by using fulltext search on the :name attribute. At this point in time, I have Sunspot Solr configured and it's working nicely.
However, when I search for dog mouse cat (as an example), it only returns articles that contain all of the keywords. How can I configure Solr to show articles like 'The dog and the cat' - which contains only 2 of the 3 search keywords in the query example above?
My searchable block in the model:
searchable do
text :name
end
My current schema.xml for fulltext search looks like this:
<fieldType name="text" class="solr.TextField" omitNorms="false">
<analyzer type="query">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StandardFilterFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.PorterStemFilterFactory"/>
</analyzer>
<analyzer type="index">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StandardFilterFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true" tokenizerFactory="solr.StandardTokenizerFactory"/>
<filter class="solr.EdgeNGramFilterFactory" minGramSize="2" maxGramSize="15" side="front"/>
<filter class="solr.PorterStemFilterFactory"/>
</analyzer>
</fieldType>
Model.search do
fulltext text do
fields :name
minimum_match 2
end
end

Solr - NGramFilterFactory dont works

I am trying configure my Solr (with sunspot, rails), to works like this:
Given a name: Willie Price
I want be able to search for: llie (for exemple)
On my schema.xml I added:
<fieldType name="text" class="solr.TextField" omitNorms="false">
<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StandardFilterFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.NGramFilterFactory" minGramSize="2" maxGramSize="15"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StandardFilterFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>
My Rails model:
searchable do
text :name, :as => :ngram
end
I also try just:
searchable do
text :name
end
On my controller:
#search = Client.search do
keywords params[:search]
end
And also try this:
#search = Client.search do
full_text params[:search]
end
The problem: I just be able to search using complete word, so given "Willie Price" only works with Willie or Price.
Thanks a lot

Partial search with sunspot

Given I have a model
class Firm < ActiveRecord::Base
searchable do
text :name
end
end
And solr's schema.xml contains
<fieldType name="text" class="solr.TextField" omitNorms="false">
<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StandardFilterFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.NGramFilterFactory" minGramSize="2" maxGramSize="30"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StandardFilterFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>
And I have a Firm with name == 'Ойл-М (Oil-M)'
When I try to search
Sunspot.search(Firm) do
fulltext 'Ойл-М'
end
Then I get nothing
When I try to search
Sunspot.search(Firm) do
fulltext 'Ойл'
end
Then I get needed Firm
How should I set up Solr and/or search to be able to find this Firm by both queries?
Your NGramFilter is cutting off the final 'M', because you have minGramSize=2. Setting minGramSize=1 will work, but this greatly increases the size of data Solr will have to store, and also drives up noise.
When you index and query a field in Solr, two things happen:
The field is split up into smaller pieces (tokenized),
Each token is then filtered.
This happens separately for indexing and querying.
In this case, you are indexing the field with StandardTokenizerFactory, StandardFilter, LowercaseFilter, and an NGramFilter, and querying the field with everything except for the NGramFilter.
Here's what's happening when you index "Ойл-М (Oil-M)" into Solr.
StandardTokenizerFactory: ['Ойл', 'М', 'Oil', 'M']
StandardFilter: ['Ойл', 'М', 'Oil', 'M']
LowerCaseFilter: ['ойл', 'м', 'oil', 'm']
NGramFilter: ['ой', 'йл', 'ойл', 'oi', 'il', 'oil']
The 'm' drops away completely. Searching for "Ойл-М" returns nothing, because there is no M to search.
Cut out the NGramFilter unless you have a very good reason to use it, and stick with the standard Russian fieldType.
<fieldType name="text_ru" class="solr.TextField" positionIncrementGap="100">
<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true" words="lang/stopwords_ru.txt" format="snowball" enablePositionIncrements="~
<filter class="solr.SnowballPorterFilterFactory" language="Russian"/>
</analyzer>
</fieldType>
NOTE: Notice that there is no distinction here between the index analyzer and query analyzer. Each query is transformed in the exact same manner as when indexed.

How can I set Sunspot to search for sequences of characters instead of words?

I wanted to understand whether Sunspot, in standard mode, searches for words or sequences of characters in full-text search and how to make it search for sequences.
For example, I have the following setup:
class User < ActiveRecord::Base
searchable do
text :email
end
end
with one User with e-mail "panayotis#matsinopoulos.gr"
the following query :
search = User.search do
fulltext 'matsinopoulos'
end
does not bring any result, whereas:
search = User.search do
fulltext 'panayotis#matsinopoulos.gr'
end
brings.
Is there any configuration setting for sunspot to match sequences of characters instead of words?
Or, am I doing something wrong?
One needs to configure file:
solr/conf/schema.xml
The standard entry:
<fieldType name="text" class="solr.TextField" omitNorms="false">
<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StandardFilterFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>
has to be turned to:
<fieldType name="text" class="solr.TextField" omitNorms="false">
<analyzer type="index">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StandardFilterFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.NGramFilterFactory"
minGramSize="3"
maxGramSize="30"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StandardFilterFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>`
</fieldType>
A very nice reference on Solr configuration can be found here:
http://techbot.me/2011/01/full-text-search-in-in-rails-with-sunspot-and-solr/
but, watch out that when it comes to partial words matching this reference talks about the EdgeNGramFilterFactory which indexes the beginnings of the words only. For making Solr match any part of the word, the NGramFilterFactory needs to be used.
Note also that we have set minGramSize to 3 and maxGramSize to 30. So, patterns with length less than 3 or greater than 30 will not be returned in queries.

Solr Sunspot multivalued field get matched element

I'm working on autocomplete functionality with solr and EdgeNGram
And create come multivalued field
here is details from schema.xml
.....
<fieldType name="autocomplete" class="solr.TextField" positionIncrementGap="100">
<analyzer type="index">
<!--tokenizer class="solr.KeywordTokenizerFactory"/-->
<!--tokenizer class="solr.NGramTokenizerFactory" minGramSize="2" maxGramSize="50"/-->
<tokenizer class="solr.WhitespaceTokenizerFactory" minGramSize="2" maxGramSize="50"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.EdgeNGramFilterFactory" minGramSize="1" maxGramSize="25" />
</analyzer>
<analyzer type="query">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>
.......
<dynamicField name="*_ac" type="autocomplete" indexed="true" stored="true" multiValued="true" />
That's how I declare user model
class User < ActiveRecord::Base
attr_accessible :email, :name
searchable do
text :email
text :name
autocomplete :text_ac, :as => :text_ac, :multiple => true do
[self.email,self.name]
end
end
end
And what I'm looking for is the way to get "hited" value
When I'm querying solr with
Sunspot.search(User, Post) do with(:user_id, 1);
adjust_solr_params do |p| p[:q] = 'text_ac:"some s"'; p[:fq] = '' end
end.results
I can get instances (Users, Posts)
, but how can I get hitted values - exact row in multivalued field - element of array

Resources