I am trying to implement sorting into my search. I am searching with use of the gem mongoid-elasticsearch. This is my model configuration:
class ActivityLog
include Mongoid::Document
include Mongoid::Timestamps
include Mongoid::Elasticsearch
elasticsearch!(
{
wrapper: :load,
sort: [
{ created_at: "desc" }
]
}
)
field :type, type: String
end
This configuration does not raise any errors, but it does not either seem like it has any effect, because search results are listed randomly.
I think I am implementing the configuration in accordance to the documentation:
Check mongoid-elasticsearch documentation here
Check elasticsearch documentation here
My search query btw is:
ActivityLog.es.search(params[:query], page: params[:page]).results.paginate(per_page: 5, page: params[:page])
You should remove Array wrapper:
elasticsearch!(
{
wrapper: :load,
sort: { created_at: "desc" }
}
)
The same way you would do it in Elasticsearch itself
ActivityLog.es.search({
body: {
query: {
query_string: {
query: params[:search]
}
},
sort: [
{created_at: {order: "asc"}},
]
})
Related
I have a search functionality that I would like to search by various fields. I am already using multi_search to look for different records from different models.
#entries_result = Post.search(params[:q].presence || '*',
fields: ['title^10', 'content', 'rating', 'parent_id', 'tags'],
...
where: { parent_id: #parent.id },
limit: 3)
#categories_result = Category.search(params[:q].presence || '*', ...)
#tags_result = Post.search(params[:q].presence || '*',
fields: ['tags'],
match: :word_start,
highlight: true,
misspellings: { below: 3 },
where: { parent_id: #parent.id, tags: params[:q] })
Searchkick.multi_search([#categories_result, #entries_result, #tags_result])
#tags_result.map(:&tags)
As you can see I am searching through Post model twice, as I need
Posts with specific name(shown on top of the search results)
Tags which are an Array in psql, that I will be showing on the side of the search results
# tags :text default([]), is an Array, indexed
I can't figure out how to find these tags that are a result of searching by params[:q]. I know that Elasticsearch can search these arrays in documents, but I couldn't find any info on how to do this with searchkick gem.
For example when params[:q] is dog I'd like to search for Posts that start with the keyword dog and also tags that start with keyword dog.
Rails 7, gems opensearch (2.0.0), searchkick (5.0.3)
I ended up building an advanced query as it's also possible to just use elasticsearch/opensearch queries in searchkick, and below code works fine for me.
#tags_result = Post.search(body:
{ query: {
bool: {
should: [
prefix: {
tags: params[:q].presence || '*',
},
],
filter: {
term: {
parent_id: #parent.id,
},
},
},
} })
I'm attempting to build a custom analyzer using nGram and apparently it's working ok, I just can't query it for some reason.
I'm using `elasticsearch-model in Ruby
Here is how the index is defined:
include Elasticsearch::Model
index_name "stemmed_videos"
settings index: { number_of_shards: 5 },
analysis: {
analyzer: {
video_analyzer: {
tokenizer: :stemmer,
filter: [
"lowercase"
]
},
standard_lowercase: {
tokenizer: :standard,
filter: [
"lowercase"
]
}
},
tokenizer: {
stemmer: {
type: "nGram",
min_gram: 2,
max_gram: 10,
token_chars: [
"letter",
"digit",
"symbol"
]
}
}
} do
mappings do
indexes :title, type: 'string', analyzer: 'video_analyzer'
indexes :description, type: 'string', analyzer: 'standard_lowercase'
end
end
def as_indexed_json(options = {})
as_json(only: [:title, :description])
end
I've attempted to take one of the strings I'm trying to index and run it through "http://localhost:9200/stemmed_videos/_analyze?pretty=1&analyzer=video_analyzer&text=indiana_jones_4-tlr3_h640w.mov" and it's apparently doing the right thing.
But then, the only way I have to make a generic query is by adding wildcards, which is not what I'm expecting.
[8] pry(main)> Video.__elasticsearch__.search('*ind*').results.total
=> 4
[9] pry(main)> Video.__elasticsearch__.search('ind').results.total
=> 0
(4 is the right number of results in my test data).
What I'd love to accomplish is to get the right results without the wildcards because with what I have now I'd need to take the query string and add the wildcards in the code, which honestly is rather bad.
How can I accomplish this?
Thanks in advance.
My elasticsearch is currently giving different results on different environments even though I'm doing the same search.
It works fine in development on my localhost, however it doesn't work on my test server (doesn't give expected records, yes I do have the database seeded).
Far as I understand what this should do is check whether it finds a hit on one of the three matches, and if it does return all the hits.
I'm running Windows 10, just using rails s.
The server is running Ubuntu 16, using nginx and unicorn.
Here's my mapping: (note: I'm not completely sure whether the analyzer does anything but it shouldn't matter)
settings index: { number_of_shards: 1 } do
mappings dynamic: 'true' do
indexes :reportdate, type: 'date'
indexes :client do
indexes :id
indexes :name, analyzer: 'dutch'
end
indexes :animal do
indexes :id
indexes :species, analyzer: 'dutch'
indexes :other_species, analyzer: 'dutch'
indexes :chip_code
end
indexes :locations do
indexes :id
indexes :street, analyzer: 'dutch'
indexes :city, analyzer: 'dutch'
indexes :postalcode
end
end
end
Here's my search:
__elasticsearch__.search({
sort: [
{ reportdate: { order: "desc" }},
"_score"
],
query: {
bool: {
should: [
{ multi_match: {
query: query,
type: "phrase_prefix",
fields: [ "other_species", "name"]
}},
{ prefix: {
chip_code: query
}},
{ match_phrase: {
"_all": {
query: query,
fuzziness: "AUTO"
}
}}
]
}
}
})
EDIT #1: Note: I'm fairly new to ruby on rails, started about 2 weeks ago, doing maintenance work on an old project and they also requested a search function.
Turns out that the problem was that I was using foreign tables (well, kinda) and nested mapping (probably this).
Here's the updated code that works on both production and locally:
__elasticsearch__.search({
sort: [
{ reportdate: { order: "desc" }},
"_score"
],
query: {
bool: {
should: [
{ multi_match: {
query: query,
type: "phrase_prefix",
fields: [ "animal.other_species", "client.name"]
}},
{ prefix: {
"animal.chip_code": query
}},
{ match_phrase: {
"_all": {
query: query,
fuzziness: "AUTO"
}
}}
]
}
}
})
Not sure why it doesn't need the animal and client parents preappended to work locally whilst it does need them on my testing server. However this works on both this way.
There is the following Post model:
class Post < ActiveRecord::Base
include Elasticsearch::Model
include Elasticsearch::Model::Callbacks
def self.search query
__elasticsearch__.search(
{
query: {
multi_match: {
query: query,
fields: ['title']
}
},
filter: {
and: [
{ term: { deleted: false } },
{ term: { enabled: true } }
]
}
}
)
end
settings index: { number_of_shards: 1 } do
mappings dynamic: 'false' do
indexes :title, analyzer: 'english'
end
end
end
Post.import
I have one Post with 'Amsterdam' title. When I execute Post.search('Amsterdam') I will get one record, all is good. But if I execute Post.search('Amster') I will get no records. What do I wrong? How can I fix it? Thanks!
OS - OS X, ElasticSearch I installed using Homebrew
You will have to use nGram tokenizer, in order to create a partial text search. A very good example of how to do this can be found here. That said, I would be very careful with nGram, as it can often turn up unrelated results.
This is because the substring "mon" is contained within all of the strings: "monkey", "money", and "monday". All of which are unrelated.
Alternatively (What I would do.)
You could try making it a fuzzy search. However, the max distance with fuzzy search is only two, which still doesn't return anything in your example. However, it tends to return relevant results.
The example I found: How to use Fuzzy Search
# Perform a fuzzy search!
POST /fuzzy_products/product/_search
{
"query": {
"match": {
"name": {
"query": "Vacuummm",
"fuzziness": 2,
"prefix_length": 1
}
}
}
}
I'm trying to use Elasticsearch and Tire to index some data. I want to be able to search it on partial matches, not just full words. When running a query on the example model below, it will only match words in the "notes" field that are full word matches. I can't figure out why.
class Thingy
include Tire::Model::Search
include Tire::Model::Callbacks
# has some attributes
tire do
settings analysis: {
filter: {
ngram_filter: {
type: 'nGram',
min_gram: 2,
max_gram: 12
}
},
analyzer: {
index_ngram_analyzer: {
type: 'custom',
tokenizer: 'standard',
filter: ['lowercase']
},
search_ngram_analyzer: {
type: 'custom',
tokenizer: 'standard',
filter: ['lowercase', 'ngram_filter']
}
}
} do
mapping do
indexes :notes, :type => "string", boost: 10, index_analyzer: "index_ngram_analyzer", search_analyzer: "search_ngram_analyzer"
end
end
end
def to_indexed_json
{
id: self.id,
account_id: self.account_id,
created_at: self.created_at,
test: self.test,
notes: some_method_that_returns_string
}.to_json
end
end
The query looks like this:
#things = Thing.search page: params[:page], per_page: 50 do
query {
boolean {
must { string "account_id:#{account_id}" }
must_not { string "test:true" }
must { string "#{query}" }
}
}
sort {
by :id, 'desc'
}
size 50
highlight notes: {number_of_fragments: 0}, options: {tag: '<span class="match">'}
end
I've also tried this but it never returns results (and ideally I'd like the search to apply to all fields, not just notes):
must { match :notes, "#{query}" } # tried with `type: :phrase` as well
What am I doing wrong?
You almost got there! :) The problem is that you've swapped the role of index_analyzer and search_analyzer, in fact.
Let me explain briefly how it works:
You want to break document words into these ngram "chunks" during indexing, so when you are indexing a word like Martian, it get's broken into: ['ma', 'mar', 'mart', ..., 'ar', 'art', 'arti', ...]. You can try it with the Analyze API: http://localhost:9200/thingies/_analyze?text=Martian&analyzer=index_ngram_analyzer.
When people are searching, they are already using these partial ngrams, so to speak, since they search for "mar" or "mart" etc. So you don't break their phrases further with the ngram tokenizer.
That's why you (correctly) separate index_analyzer and search_analyzer in your mapping, so Elasticsearch knows how to analyze the notes attribute during indexing, and how to analyse any search phrase against this attribute.
In other words, do this:
analyzer: {
index_ngram_analyzer: {
type: 'custom',
tokenizer: 'standard',
filter: ['lowercase', 'ngram_filter']
},
search_ngram_analyzer: {
type: 'custom',
tokenizer: 'standard',
filter: ['lowercase']
}
}
Full, working Ruby code is below. Also, I highly recommend you to migrate to the new elasticsearch-model Rubygem, which contains all important features of Tire and is actively developed.
require 'tire'
Tire.index('thingies').delete
class Thingy
include Tire::Model::Persistence
tire do
settings analysis: {
filter: {
ngram_filter: {
type: 'nGram',
min_gram: 2,
max_gram: 12
}
},
analyzer: {
index_ngram_analyzer: {
type: 'custom',
tokenizer: 'standard',
filter: ['lowercase', 'ngram_filter']
},
search_ngram_analyzer: {
type: 'custom',
tokenizer: 'standard',
filter: ['lowercase']
}
}
} do
mapping do
indexes :notes, type: "string", index_analyzer: "index_ngram_analyzer", search_analyzer: "search_ngram_analyzer"
end
end
end
property :notes
end
Thingy.create id: 1, notes: 'Martial Partial Martian'
Thingy.create id: 2, notes: 'Venetian Completion Heresion'
Thingy.index.refresh
# Find 'art' in 'martial'
#
# Equivalent to: http://localhost:9200/thingies/_search?q=notes:art
#
results = Thingy.search do
query do
match :notes, 'art'
end
end
p results.map(&:notes)
# Find 'net' in 'venetian'
#
# Equivalent to: http://localhost:9200/thingies/_search?q=notes:net
#
results = Thingy.search do
query do
match :notes, 'net'
end
end
p results.map(&:notes)
The problem for me was that I was using the string query instead of the match query. The search should have been written like this:
#things = Thing.search page: params[:page], per_page: 50 do
query {
match [:prop_1, prop_2, :notes], query
}
sort {
by :id, 'desc'
}
filter :term, account_id: account_id
filter :term, test: false
size 50
highlight notes: {number_of_fragments: 0}, options: {tag: '<span class="match">'}
end