tire and elasticsearch_autocomplete accents and facets - ruby-on-rails

I'm using elasticsearch_autocomplete gem for autocomplete feature.
I have a problem with special characters ñ and accents áéíóú.
Model:
class Car
ac_field :name, :description, :city, :skip_settings => true
def self.ac_search(params, options={})
tire.search load: true, page: params[:page], per_page: 9 do
query do
boolean do
must { string params[:query], default_operator: "AND" } if params[:query].present?
must { term :city, params[:city] } if params[:city].present?
end
end
filter :term, city: params[:city] if params[:city].present?
facet "city" do
terms :city
end
end
end
end
This version works fine with special characters.
e.g.: Query with Martin I get all results with Martín, martín, martin, Martin
With this approach this is the problem:
Now what results is individual words. e.g. A city tagged ["San Francisco", "Madrid"] will end up having three separate tags. Similarly, if I do a query to search on "san francisco" (must { term 'city', params[:city] }), that will fail, while a query on "San" or "Francisco" will succeed. The desired behaviour here is that the tag should be atomic, so only a "San Francisco" (or "Madrid") tag search should succeed.
To fix this problem I create my custom mapping:
model = self
settings ElasticsearchAutocomplete::Analyzers::AC_BASE do
mapping _source: {enabled: true, includes: %w(name description city)} do
indexes :name, model.ac_index_config(:name)
indexes :description, model.ac_index_config(:description)
indexes :city, :type => 'string', :index => :not_analyzed
end
end
With this mapping the problem with multi-words is fixed, and now facets with city field works fine:
Instead of getting the type facets San and Francisco Now I get San Francisco
Now, the problem is that with this mapping inside of the model the search doesn't find results with special characters
e.g.: Query with Martin I get only results with Martin martin
I'm using mongoid instead active record.
How can I fix this problem? I think that the problem is with asciifolding tokenfilter.

I fixed the problem with:
class User
include Mongoid::Document
field :city, :type => String
has_one: car
end
class Car
ac_field :name, :description, :user_city, :skip_settings => true
def self.ac_search(params, options={})
tire.search load: true, page: params[:page], per_page: 9 do
query do
boolean do
must { term :user_city, params[:user_city] } if params[:user_city].present?
end
end
facet "cities" do
terms :user_city
end
end
end
model = self
settings ElasticsearchAutocomplete::Analyzers::AC_BASE do
mapping _source: {enabled: true, includes: %w(car_city name description)} do
indexes :car_city, :type => 'string', :index => :not_analyzed
end
end
def to_indexed_json
to_json(methods: [:user_city])
end
def user_city
user.city
end
end

Related

How to boost query depends of attribute's value in ElasticSearch?

I have a Profile class. In ES index I have company_type attribute.
class Profile
...
include Tire::Model::Search
include Tire::Model::Callbacks
def to_indexed_json
{ name: self.name,
company_type: self.company.company_type
}.to_json
end
end
Tire.search('profiles') do
query do
custom_filters_score do
query { all }
filter do
filter :range, last_contact_at: { gte: 7.days.ago }
boost 1
end
score_mode :total
end
end
end.results
I would like to boost by 10 queries if company_type == 'intern'.
did you try adding
filter do
filter :term, company_type: "intern"
boost 10.0
end
to your custom_filters_score filter?

Having trouble with mixed complex boolean with Tire (and ElasticSearch)

I've been trying to figure out how to do mixed boolean searches that use nested objects using Tire. All the simple examples I've found don't include a more complex query (when searching on other attributes).
My search involves finding a Team that 'needs' a specific type of person. When trying to build a football team, the team needs to fill the roster with certain types of players of a given weight class, with the option of excluding one term or the other.
Other parameters such as 'region' or 'kind' have to do with where the team plays, and what kind of team (casual, competitive, etc) it is.
My current setup:
mapping do
indexes :region, index: :not_analyzed
indexes :kind, index: :not_analyzed
indexes :role_requirements do
indexes :need, type: 'boolean'
indexes :weight_class_id, type: 'integer'
indexes :role_id, type: 'integer'
end
.
.
.
end
def self.search(params)
team_params = params[:team_search]
tire.search(page: params[:page], per_page: 10) do
query do
boolean do
must { string team_params[:query], default_operator: "AND" } if team_params[:query].present?
must { term :kind, team_params[:kind] } if team_params[:kind].present?
must { term :region, team_params[:region] } if team_params[:region].present?
if team_params[:weight_class_id].present? || team_params[:role_id].present?
must { term 'role_requirements.need', true }
end
must { term 'role_requirements.weight_class_id', team_params[:job_id].to_i } if team_params[:weight_class_id].present?
must { term 'role_requirements.role_id', team_params[:role_id].to_i } if team_params[:role_id].present?
.
.
.
end
end
end
end
I've tried a number of ways, but there usually seems to be a problem either with ElasticSearch not able to parse things or Tire not having the method within the scope:
With this implementation, here is the generated to_json: https://gist.github.com/8a615e701eb31ff2e250
Which are currently not giving me any results.
All the different ways I've tried: https://gist.github.com/907c9571caa0e87bad27
None are really able to give me full results.
You seem to be missing the nested type in the mapping:
mapping do
indexes :region, index: :not_analyzed
indexes :kind, index: :not_analyzed
indexes :role_requirements, type: 'nested' do
indexes :need, type: 'boolean'
indexes :weight_class_id, type: 'integer'
indexes :role_id, type: 'integer'
end
# ..more mappings..
end
Then you can build your query like this:
tire.search(page: params[:page], per_page: 10) do
query do
boolean do
must { string team_params[:query], default_operator: "AND" } if team_params[:query].present?
must { term :kind, team_params[:kind] } if team_params[:kind].present?
must { term :region, team_params[:region] } if team_params[:region].present?
must do
nested path: 'role_requirements' do
query do
boolean do
if team_params[:weight_class_id].present? || team_params[:role_id].present?
must { term 'role_requirements.need', true }
end
must { term 'role_requirements.weight_class_id', team_params[:job_id].to_i } if team_params[:weight_class_id].present?
must { term 'role_requirements.role_id', team_params[:role_id].to_i } if team_params[:role_id].present?
end
end
end
end
end
end
end
Here are some examples in Tire's integration tests.
Hope this helps :)

Elastic Search/Tire: How do I filter a boolean attribute?

I want to filter the private boolean of my class so it only shows resources that aren't private but it's not working for me. (I dumbed down the code tremendously)
mapping do
indexes :private, type: "boolean"
indexes :name, type: "string"
end
end
def self.search(params)
tire.search(load: true, page: params[:page], per_page: 20) do
query { string params[:query] } if params[:query].present?
# So far I've tried...
# filter :bool, :private => ["false"]
# filter :bool, private: false
end
end
How do I do this correctly?
filter :term, :private => false
Should do the trick. Depending on whether you want to do stuff with facets it may be more efficient to do your filtering a filtered query rather than at the top level, ie
tire.search(...) do
query do
filtered do
query { string, params[:query] }
filter :term, :private => false
end
end
end
It shouldn't change the results though.
You can also do this with a bool filter, but not quite how you tried - within a bool filter you need to build up a structure that says what's optional and what's not
For example
tire.search(load: true, page: params[:page], per_page: 20) do
query { string params[:query] } if params[:query].present
filter :bool, :must => {:term => {:private => true}}
end
A bool filter is slower than using an and filter (which is what tire does behind the scenes if you specify multiple filters) but obviously gives you more flexibility.
You can try:
tire.search(load: true, page: params[:page], per_page: 20) do
query do
boolean do
must { string params[:query] } if params[:query].present?
must { term :private, true }
end
end
end
According to the elasticsearch - guide, booleans are stored as T or F, so I would try filtering by T or F.
For example
filter :terms, :private => ['T']
I haven't actually used tires, this is just based on some research on the guide and in the examples.

RoR: Elasticsearch sort by mapped field

I have Player and Player have fullname from Profil. I use elasticsearch on Player. I need default sort by fullname. How can I set it? My code from Player class file:
......
mapping do
indexes :id, type: 'integer'
indexes :fullname, boost: 10
indexes :name
indexes :surname
indexes :position_name
indexes :info
indexes :club_id, type: 'integer'
indexes :club_name
end
def self.search(params)
tire.search(load: true, page: params[:page], per_page: 20) do
query do
boolean do
if params[:query].present?
must { string params[:query], default_operator: "AND" }
else
must {string '*'}
end
must { term :position_id, params[:position_id]} if params[:position_id].present?
must { term :club_id, params[:club_id]} if params[:club_id].present?
end
end
# filter :term, user_id: params[:user_id] if params[:user_id].present?
# sort { by Item.column_names.include?(params[:sort]) ? params[:sort] : "created_at", %w[asc desc].include?(params[:direction]) ? params[:direction] : "desc" } if params[:query].blank?
sort { by Player.column_names.include?(params[:sort]) ? params[:sort] : :id ,%w[asc desc].include?(params[:direction]) ? params[:direction] : "desc"}
facet "positions" do
terms :position_id
end
facet "clubs" do
terms :club_id
end
# raise to_curl
end
end
def to_indexed_json
to_json(methods: [:fullname, :name, :surname, :position_name, :club_name])
end
def fullname
self.profil.fullname
end
......
If I changed :id to :fullname in sort I have this error:
500 : {"error":"SearchPhaseExecutionException[Failed to execute phase [query], total failure; shardFailures {[DBgvi6JiQAi0FTwhonq8Ag][players][0]: QueryPhaseExecutionException[[players][0]: query[ConstantScore(NotDeleted(cache(_type:player)))],from[0],size[20],sort[<custom:\"fullname\": org.elasticsearch.index.field.data.strings.StringFieldDataType$1#33a626ac>!]: Query Failed [Failed to execute main query]]; nested: IOException[Can't sort on string types with more than one value per doc, or more than one token per field]; }{[DBgvi6JiQAi0FTwhonq8Ag][players][4]: QueryPhaseExecutionException[[players][4]: query[ConstantScore(NotDeleted(cache(_type:player)))],from[0],size[20],sort[<custom:\"fullname\": org.elasticsearch.index.field.data.strings.StringFieldDataType$1#3274eb8a>!]: Query Failed [Failed to execute main query]]; nested: IOException[Can't sort on string types with more than one value per doc, or more than one token per field]; }]","status":500}
I got it! After dumb amounts of research I found this article here. The first answer mentions:
The field user by default gets analyzed and broken down into one or more tokens. If it gets broken down into more than one token, then you can't really sort on it. If you want to sort on it, you either need to set it i(in the mappings) as not analyzed
In your situation, all you would need to do is:
mapping do
indexes :id, type: 'integer'
indexes :fullname, :index => 'not_analyzed'
indexes :name
indexes :surname
indexes :position_name
indexes :info
indexes :club_id, type: 'integer'
indexes :club_name
end
And it should work.
sort { by Player.column_names.include?(params[:sort]) ? params[:sort] : :fullname ,%w[asc desc].include?(params[:direction]) ? params[:direction] : "desc"}
Changing 'id' to 'fullname' should work.

Sunspot Solr, Rails and Ordering

I have a project model in my rails 3.1 application and I want to use Solr to run a search on it.
I defined the search like this:
searchable do
text :nr, :boost => 5 # nr is integer
text :name, :boost => 5
text :description, :boost => 2
text :client do
client.name
end
text :tasks do
tasks.map(&:name)
end
end
The project-nr, in my model just called nr, type integer, is the most used reference for finding a project.
Now besides having a search form I still want my projects ordered by the nr when no search was performed, but this does not work - my project seem to be in totally random order.
The code of my ProjectsController index action looks like this:
def index
#search = Project.search do
fulltext params[:search]
paginate :page => params[:page]
order_by :nr, :desc
end
#projects = #search.results
##projects = Project.active.visible.design.order("nr desc")
respond_to do |format|
format.html # index.html.erb
format.json { render json: #projects }
end
But when I visit then myapp/projects I get a
Sunspot::UnrecognizedFieldError in ProjectsController#index
No field configured for Project with name 'nr'
error...
any ideas what I need to do to order by nr. ?
thanks
Okay, I solved it by turning the nr field to an integer in my searchable:
searchable do
integer :nr
text :name, :boost => 5
text :description, :boost => 2
text :client do
client.name
end
text :tasks do
tasks.map(&:name)
end
end
Now I was able to order it nicely but I couldn't perform a text search on the project_nr anymore.
So I added a virtual attribute name_number to my Project model and instead searched on this field.
def name_number
"#{self.nr} - #{self.name[0..24]}"
end
Now I have ordering and searching in place... If there are other / better ideas, keep em coming!

Resources