I have the following code and i'm trying to use ElasticSearch to query it.
It is working when i do Book.search(:q=>'Foo') but it doesn't work when i do Book.search(:author=>'Doctor'). In my database I have a entry with a name like "Barz, Foo, Doctor"
I'm not sure if I should use terms or term, in my query, because i'm breaking the name using snowball. I tried with terms and then I get an error. With term I get no results.
class Author < ActiveRecord::Base
has_many :books
end
class Book < ActiveRecord::Base
belongs_to :author
include Tire::Model::Search
include Tire::Model::Callbacks
mapping do
indexes :title,
indexes :description
indexes :author,type: 'object', properties: {
name: { type: 'multi_field',
fields: { name: { type: 'string', analyzer: 'snowball' },
exact: { type: 'string', index: 'not_analyzed' }
}
} }
end
def to_indexed_json
to_json(:include=>{:author=>{:only=>[:name]}} )
end
def self.search(params = {})
tire.search(load:true) do
query do
boolean do
should { string params[:q] } if params[:q].present?
should { term params[:author] } if params[:author].present?
end
end
filter :term, :active=>true
end
end
end
You can do like this
should { terms :author, [params[:author]]} if params[:author].present?
OR
should { term :author, params[:author]} if params[:author].present?
OR
should { string "author:#{params[:author]}"} if params[:author].present?
As #Karmi stated enter link description here
Hi, yeah, your approach seems one. Couple of things:
* unless you want to use Lucene query syntax (boosting, ranges, etc), it's maybe best to use the text query,
* yes, filters are more performant then queries, an the active=true in your example is a good fit for filters. Beware of the interplay between queries, filters and facets, though.
Your definition of the term query is incorrect, though -- it should be:
term :author, params[:author]
Related
I have an app where I'm loading a bunch of ebooks along with metadata into Neo4j with the goal of producing an IPFS directory tree to input into another app.
It is a rails application, so I have a Book class as such:
class Book
include Neo4j::ActiveNode
property :title, type: String
property :author, type: String
property :created_at, type: DateTime
property :updated_at, type: DateTime
has_many :in, :contexts, type: :FOR
has_one :out, :cover, type: :CVR, model_class: :Content
has_one :out, :content, type: :DAT
end
In the rake task to export to IPFS, I am using the following to perform the query:
q = Neo4j::ActiveBase.current_session.query(
"MATCH path = (n:Context)-[s:SUB*]->(m:Context)-[f:FOR]->(o:Book) WHERE n.name = '∅' RETURN DISTINCT path LIMIT 500"
)
This is returning the paths that I want, but q.first.path.nodes.first (and all the other nodes) are Neo4j::Core::Nodes. I want to access the cover and content relationships of my Book class, but can't. For example:
q.each do |ret|
nodes = ret.path.nodes
nodes.shift # remove ∅
book = nodes.pop
path = nodes.map(&:name)
if book.content
system('ipfs', 'files', 'cp', "/ipfs/#{book.content.ipfs_id}", "/#{path.join('/')}/index.epub")
end
⋮
One option is to use the uuid to look up the book:
book = Book.find(nodes.pop.properties[:uuid])
I am using elasticsearch-rails and elasticsearch-model gem for searching words in my rails app.
I want to make my search case insensitive and must be independent of pluralization. I researched a lot on google but got a hunch on how to do it using analyzer but not success. So I had to post a new question.
Here is my model where I want to search to take place
require 'elasticsearch/model'
class Article < ActiveRecord::Base
include Elasticsearch::Model
include Elasticsearch::Model::Callbacks
settings index: { number_of_shards: 1 } do
mappings dynamic: 'false' do
indexes :title, analyzer: 'english', index_options: 'offsets'
indexes :text, analyzer: 'english'
end
end
def self.search(query)
__elasticsearch__.search(
{
query: {
multi_match: {
query: query,
fields: ['title^10', 'text']
}
},
highlight: {
pre_tags: ['<em>'],
post_tags: ['</em>'],
fields: {
title: {},
text: {}
}
}
}
)
end
end
# Delete the previous articles index in Elasticsearch
Article.__elasticsearch__.client.indices.delete index: Article.index_name rescue nil
# Create the new index with the new mapping
Article.__elasticsearch__.client.indices.create \
index: Article.index_name,
body: { settings: Article.settings.to_hash, mappings: Article.mappings.to_hash }
# Index all article records from the DB to Elasticsearch
Article.import
#Article.import force: true
My questions are how do I do search word? tshirt, T-shirt, Tshirts should all match.
Any links for further research is also helpful.
Thank you
One of the method for searching in a table is:
User.where("firstname ILIKE ? OR lastname ILIKE ?", "%#{params[q]}%", "%#{params[q]}%" );
where
User is Model
params[q] is search query parameter.
This seems like a really easy issue, but everything I've tried from other solutions and websites is not working. I have three fields I do not want indexed or queried--:p_s, :gender, and :part_of_speech--but elasticsearch is still returning values from those fields even though I don't specify that they should be indexed or queried. About halfway down, this article says to say no to indexing, but they don't indicate where this would occur.
Term Controller:
def search
#terms = Term.search(params[:query]).page(params[:page])
end
Model:
require 'elasticsearch/model'
class Term < ActiveRecord::Base
include Elasticsearch::Model
include Elasticsearch::Model::Callbacks
settings index: { number_of_shards: 1, number_of_replicas: 0 },
do
mappings dynamic: 'false' do
indexes :id, index: :not_analyzed
indexes :name, analyzer: :spanish_analyzer
indexes :definition, analyzer: :combined_analyzer
indexes :etymology1, analyzer: :combined_analyzer
indexes :etymology2, analyzer: :combined_analyzer
indexes :uses, analyzer: :combined_analyzer
indexes :notes1, analyzer: :combined_analyzer
indexes :notes2, analyzer: :combined_analyzer
end
end
def self.search(query)
__elasticsearch__.search(
{
query: {
multi_match: {
query: query,
fields: ['name^7', 'definition^6', 'etymology1^5', 'etymology2^4', 'uses^3', 'notes1^2', 'notes2^1'],
operator: 'and'
}
}
}
)
end
end
# Delete the previous term index in Elasticsearch
Term.__elasticsearch__.client.indices.delete index: Term.index_name rescue nil
# Create the new index with the new mapping
Term.__elasticsearch__.client.indices.create \
index: Term.index_name,
body: { settings: Term.settings.to_hash, mappings: Term.mappings.to_hash }
# Index all term records from the DB to Elasticsearch
Term.import(force: true)
To mark a field as non-indexed use this:
mappings dynamic: 'false' do
...
indexes :p_s, index: :no
indexes :gender, index: :no
indexes :part_of_speech, index: :no
...
end
By default elasticsearch returns all document fields under "_source" key. To only get specific fields you can either specify fields array on the top query level like this
def self.search(query)
__elasticsearch__.search(
{
query: {
multi_match: {
query: query,
fields: ['name^7', 'definition^6', 'etymology1^5', 'etymology2^4', 'uses^3', 'notes1^2', 'notes2^1'],
operator: 'and'
}
},
fields: ['name', 'definition', 'etymology1', 'etymology2', 'uses', 'notes1', 'notes2']
}
)
end
or filter "_source"
def self.search(query)
__elasticsearch__.search(
{
query: {
multi_match: {
query: query,
fields: ['name^7', 'definition^6', 'etymology1^5', 'etymology2^4', 'uses^3', 'notes1^2', 'notes2^1'],
operator: 'and'
}
},
'_source': ['name', 'definition', 'etymology1', 'etymology2', 'uses', 'notes1', 'notes2']
}
)
end
See Elasticsearch source filtering docs for more.
When using multi_match clause, the inner fields element specifies the fields to run the search on and, optionally, the boost like in your example. The outer fields or '_source' clause in turn determines which fields to return and this is the one you're after.
To have a better visibility into what's going on while debugging elasticsearch queries, use a tool like Sense. When you get the result you want it may be much easier to transfer the query to ruby code than vice versa.
I think using the included elasticsearch methods makes a lot of sense. However, in my own case, in my model I did something like this, modified for your own case:
def as_indexed_json
as_json(only: [:id, :name, :definition, :etymology1, :etymology2, :uses, :notes1, :notes2])
end
This should work because by default Elasticsearch would call the as_indexed_json method in your model to get the data it needs to index.
The data structure is a Post which has_many Post_text. Following a great example at https://github.com/elasticsearch/elasticsearch-rails/blob/master/elasticsearch-model/examples/activerecord_associations.rb. I have defined the mapping as the following:
include SearchableModule
mapping do
indexes :country
indexes :post_texts do
indexes :subject, type: 'string', boost: 10, analyzer: 'snowball'
indexes :description, type: 'string', analyzer: 'snowball'
end
end
And of course, in the searchable_module.rb I just copy what's in the example with some changes in as_index_json():
def as_indexed_json(options={})
self.as_json(
include: { post_texts: { only: [:subject, :description]}
})
end
And things seems ok. I have re-import the data:
Post.import
Post.__elasticsearch__.
Then I try to check the result of SQL's LIKE and Elasticsearch by:
SQL LIKE:
PostText.where("subject LIKE '%Testing%' OR description LIKE '%Testing%'").each do |r|
puts r.post_id
end
There are 12 unique post_id with this approach.
Elasticsearch:
Post.search("Testing").results.count
=> 10
Is there anything I have missed? Thank you!!!!
you could try Post.search("Testing").total which should return summary number of results, in case with results.count you just count number of returned records suppose limited per_page
I am using the tire gem for ElasticSearch in Rails.
Ok so I have been battling with this the whole day and this is how far I have got. I would like to make fields on my model not searchable but they should still be available in the _source so I can use them for sorting on the search result.
My mappings:
mapping do
indexes :created_at, :type => 'date', :index => :not_analyzed
indexes :vote_score, :type => 'integer', :index => :not_analyzed
indexes :title
indexes :description
indexes :tags
indexes :answers do
indexes :description
end
end
My to_indexed_json method:
def to_indexed_json
{
vote_score: vote_score,
created_at: created_at,
title: title,
description: description,
tags: tags,
answers: answers.map{|answer| answer.description}
}.to_json
end
My Search query:
def self.search(term='', order_by, page: 1)
tire.search(page: page, per_page: PAGE_SIZE, load: true) do
query { term.present? ? string(term) : all }
sort {
by case order_by
when LAST_POSTED then {created_at: 'desc'}
else {vote_score: 'desc', created_at: 'desc'}
end
}
end
end
The only issue I am battling with now is how do I make vote_score and created_at field not searchable but still manage to use them for sorting when I'm searching.
I tried indexes :created_at, :type => 'date', :index => :no but that did not work.
If I understand you, you are not specifying a field when you send your search query to elasticsearch. This means it will be executed agains the _all field. This is a "special" field that makes elasticsearch a little easier to get using quickly. By default all fields are indexed twice, once in their own field, and once in the _all field. (You can even have different mappings/analyzers applied to these two indexings.)
I think setting the field's mappings to "include_in_all": "false" should work for you (remove the "index": "no" part). Now the field will be tokenized (and you can search with it) under it's fieldname, but when directing a search at the _all field it won't affect results (as none of it's tokens are stored in the _all field).
Have a read of the es docs on mappings, scroll down to the parameters for each type
Good luck!
I ended up going with the approach of only matching on the fields I want and that worked. This matches on multiple fields.
tire.search(page: page, per_page: PAGE_SIZE, load: true) do
query { term.present? ? (match [:title, :description, :tags, :answers], term) : all }
sort {
by case order_by
when LAST_POSTED then {created_at: 'desc'}
else {vote_score: 'desc', created_at: 'desc'}
end
}
end