World.
I'm new in rais. I have the problem:
After I use class for steamming
<filter class="solr.HunspellStemFilterFactory" dictionary="ru_RU.dic" affix="ru_RU.aff" ignoreCase="true" />
in "schema.xml"
<fieldType name="text" class="solr.TextField" indexed="true" stored="true" multiValued="true" positionIncrementGap="100">
<analyzer type="index">
<tokenizer class="solr.LowerCaseFilterFactory"/>
<filter class="solr.HunspellStemFilterFactory" dictionary="ru_RU.dic" affix="ru_RU.aff" ignoreCase="true" />
<filter class="solr.EdgeNGramFilterFactory" minGramSize="2" maxGramSize="25" side="front" />
<filter class="solr.EdgeNGramFilterFactory" minGramSize="2" maxGramSize="25" side="back" />
</analyzer>
<analyzer type="query">
<tokenizer class="solr.LowerCaseFilterFactory"/>
<filter class="solr.HunspellStemFilterFactory" dictionary="ru_RU.dic" affix="ru_RU.aff" ignoreCase="true" />
</analyzer>
</fieldType>
I restart solr-server and then can't reindexing (rake sunspot:reindex). Get this error:
Errno::ECONNREFUSED: Connection refused - {:data=>"<?xml version=\"1.0\" encoding=\"UTF-8\"?><delete><query>type:Ad</query></delete>", :headers=>{"Content-Type"=>"text/xml"}, :method=>:post, :params=>{:wt=>:ruby}, :query=>"wt=ruby", :path=>"update", :uri=>#<URI::HTTP:0xa2a3280 URL:http://localhost:8982/solr/development/update?wt=ruby>, :open_timeout=>nil, :read_timeout=>nil, :retry_503=>nil, :retry_after_limit=>nil}
If I delete hunspell from schema.xml reindexing are completed.
ru_RU.dic and ru_RU.aff in utf-8.
Thank you in advance)
Related
I'm running Sunspot Solr in my rails app. I am using it to enable a user to search for different "articles" by using fulltext search on the :name attribute. At this point in time, I have Sunspot Solr configured and it's working nicely.
However, when I search for dog mouse cat (as an example), it only returns articles that contain all of the keywords. How can I configure Solr to show articles like 'The dog and the cat' - which contains only 2 of the 3 search keywords in the query example above?
My searchable block in the model:
searchable do
text :name
end
My current schema.xml for fulltext search looks like this:
<fieldType name="text" class="solr.TextField" omitNorms="false">
<analyzer type="query">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StandardFilterFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.PorterStemFilterFactory"/>
</analyzer>
<analyzer type="index">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StandardFilterFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true" tokenizerFactory="solr.StandardTokenizerFactory"/>
<filter class="solr.EdgeNGramFilterFactory" minGramSize="2" maxGramSize="15" side="front"/>
<filter class="solr.PorterStemFilterFactory"/>
</analyzer>
</fieldType>
Model.search do
fulltext text do
fields :name
minimum_match 2
end
end
I am trying configure my Solr (with sunspot, rails), to works like this:
Given a name: Willie Price
I want be able to search for: llie (for exemple)
On my schema.xml I added:
<fieldType name="text" class="solr.TextField" omitNorms="false">
<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StandardFilterFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.NGramFilterFactory" minGramSize="2" maxGramSize="15"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StandardFilterFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>
My Rails model:
searchable do
text :name, :as => :ngram
end
I also try just:
searchable do
text :name
end
On my controller:
#search = Client.search do
keywords params[:search]
end
And also try this:
#search = Client.search do
full_text params[:search]
end
The problem: I just be able to search using complete word, so given "Willie Price" only works with Willie or Price.
Thanks a lot
Given I have a model
class Firm < ActiveRecord::Base
searchable do
text :name
end
end
And solr's schema.xml contains
<fieldType name="text" class="solr.TextField" omitNorms="false">
<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StandardFilterFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.NGramFilterFactory" minGramSize="2" maxGramSize="30"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StandardFilterFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>
And I have a Firm with name == 'Ойл-М (Oil-M)'
When I try to search
Sunspot.search(Firm) do
fulltext 'Ойл-М'
end
Then I get nothing
When I try to search
Sunspot.search(Firm) do
fulltext 'Ойл'
end
Then I get needed Firm
How should I set up Solr and/or search to be able to find this Firm by both queries?
Your NGramFilter is cutting off the final 'M', because you have minGramSize=2. Setting minGramSize=1 will work, but this greatly increases the size of data Solr will have to store, and also drives up noise.
When you index and query a field in Solr, two things happen:
The field is split up into smaller pieces (tokenized),
Each token is then filtered.
This happens separately for indexing and querying.
In this case, you are indexing the field with StandardTokenizerFactory, StandardFilter, LowercaseFilter, and an NGramFilter, and querying the field with everything except for the NGramFilter.
Here's what's happening when you index "Ойл-М (Oil-M)" into Solr.
StandardTokenizerFactory: ['Ойл', 'М', 'Oil', 'M']
StandardFilter: ['Ойл', 'М', 'Oil', 'M']
LowerCaseFilter: ['ойл', 'м', 'oil', 'm']
NGramFilter: ['ой', 'йл', 'ойл', 'oi', 'il', 'oil']
The 'm' drops away completely. Searching for "Ойл-М" returns nothing, because there is no M to search.
Cut out the NGramFilter unless you have a very good reason to use it, and stick with the standard Russian fieldType.
<fieldType name="text_ru" class="solr.TextField" positionIncrementGap="100">
<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.StopFilterFactory" ignoreCase="true" words="lang/stopwords_ru.txt" format="snowball" enablePositionIncrements="~
<filter class="solr.SnowballPorterFilterFactory" language="Russian"/>
</analyzer>
</fieldType>
NOTE: Notice that there is no distinction here between the index analyzer and query analyzer. Each query is transformed in the exact same manner as when indexed.
I'm working on autocomplete functionality with solr and EdgeNGram
And create come multivalued field
here is details from schema.xml
.....
<fieldType name="autocomplete" class="solr.TextField" positionIncrementGap="100">
<analyzer type="index">
<!--tokenizer class="solr.KeywordTokenizerFactory"/-->
<!--tokenizer class="solr.NGramTokenizerFactory" minGramSize="2" maxGramSize="50"/-->
<tokenizer class="solr.WhitespaceTokenizerFactory" minGramSize="2" maxGramSize="50"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.EdgeNGramFilterFactory" minGramSize="1" maxGramSize="25" />
</analyzer>
<analyzer type="query">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>
.......
<dynamicField name="*_ac" type="autocomplete" indexed="true" stored="true" multiValued="true" />
That's how I declare user model
class User < ActiveRecord::Base
attr_accessible :email, :name
searchable do
text :email
text :name
autocomplete :text_ac, :as => :text_ac, :multiple => true do
[self.email,self.name]
end
end
end
And what I'm looking for is the way to get "hited" value
When I'm querying solr with
Sunspot.search(User, Post) do with(:user_id, 1);
adjust_solr_params do |p| p[:q] = 'text_ac:"some s"'; p[:fq] = '' end
end.results
I can get instances (Users, Posts)
, but how can I get hitted values - exact row in multivalued field - element of array
I just have done setting up sunspot_rails and it seems working well except one thing.
After I made 3 records like below
name=John
name=John2
name=John3
when I search with the keyword "John", only 1st record shows up. it looks like complete matching.
I'd like to have all of them to be appeared as search result.
Is this supposed to be happened as default?
or did I setup something wrong??
If you want return substrings in fulltext search, you can take a look in
https://github.com/sunspot/sunspot/wiki/Matching-substrings-in-fulltext-search
Also you can add a file sunspot_solr.rb for pagination of results in myapp/config/initializers/ with:
Sunspot.config.pagination.default_per_page = 100
return 100 results for this case.
Added:
Your schema.xml file is founded in yourappfolder/solr/conf
Also you can add <filter class="solr.NGramFilterFactory"/> to match arbitrary substrings.
This is my particular config for schema.xml:
<fieldType name="text" class="solr.TextField" omitNorms="false">
<analyzer>
<tokenizer class="solr.StandardTokenizerFactory"/>
<filter class="solr.StandardFilterFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>
<fieldtype class="solr.TextField" name="text_pre" positionIncrementGap="100">
<analyzer type="index">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.NGramFilterFactory" minGramSize="2" maxGramSize="10"/>
<filter class="solr.ISOLatin1AccentFilterFactory"/>
<filter class="solr.TrimFilterFactory" />
<filter class="solr.EdgeNGramFilterFactory" minGramSize="2" maxGramSize="10"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.ISOLatin1AccentFilterFactory"/>
<filter class="solr.TrimFilterFactory" />
</analyzer>
</fieldtype>
For me it does works fine with full strings and substrings for all keywords. Please do not forget to restart the server and reindex your models for the changes to take effect.
Regards!
Thanks!!!
block from girls controller(girls_controller.rb)
def index
#search = Girl.search do
fulltext params[:search]
end
#girls = #search.results
# #girls = Girl.all
#
# respond_to do |format|
# format.html # index.html.erb
# format.json { render json: #girls }
# end
end
block from Girl model(girl.rb)
searchable do
text :name_en, :name_es, name_ja
end