Ok so i have a date field that i need to search on, but i need to search on it by day like in a mysql query
search_conditions << ["DAY(open_date) != ?", event.thursday.day] if options[:thur].blank?
and i need to do this condition with Thinking Sphinx so i tried this
attr_accessor :event_day
def event_day
self.start_date.day
end
#thinking sphinx configurations for the event search
define_index do
indexes event_day
...
...
and in the search i tried this
search_string = "#event_day -#{event.thursday.day}" unless options[:thur].blank?
but i keep getting this error
index event_core: query error: no field 'event_day' found in schema
Any way to make this work
You can't use a ruby attribute in an SQL query. Rails isn't that clever.
You need to write SQL that replicates that function, or filter the results of a query through it, e.g.
#my_query.where(:a => "b").select { |rec| rec.some_method == "some value" }
As Michael's pointed out, Ruby attributes aren't accessible by Sphinx - it talks directly to your database.
So, either you can create a column that holds the event day value, and reference that via Sphinx, or you can create a field that uses a SQL function (which could vary, depending on MySQL or PostgreSQL) that extracts the day from the start_date column - not particularly complex. It'd probably end up looking like this:
indexes "GET_DAY_FROM_DATE(start_date)", :as => :event_day
Related
I want to implement a scope overdue in a model Invoice to return all invoices that exceeded the date, until they had to be paid. I have the fields invoice_date, :type => Date and days_for_payment, :type => Integer.
In my previous version, which was built on ActiveRecord, I could use the query
Invoice.where("invoice_date + days_for_payment < ?", Date.today)
This query made the calculation on the DB side.
Is there a way to get the same thing done with Mongoid? Or does anyone know a good workaround (proc, lambda, etc.)?
I use mongoid '2.4.12'
Found the answer myself. With the prefix this.* I can reference to the fields. And I can use JavaScript functions. MongoDB gets cooler and cooler!
So here is my solution:
class Invoice
include Mongoid::Document
field :invoice_date, :type => Date
field :days_for_payment, :type => Integer
...
scope :overdue, where("(Math.round(this.invoice_date.getTime() / 1000) + (this.days_for_payment * 24 * 3600)) < #{Time.now.to_i}")
...
end
Timestamp creation in js works different. So I had to get rid of the last three numbers and round them.
If anybody knows a more elegant way, please let me know.
My only problem left is, that I can't store a Date object to MongoDB. It always tells me I have to use Time. I think I better upgrade mongoid to 3.0.1.
I am not sure about mongoid, if you are querying mongodb directly you can use the $where operator. It is not recommended as it doesn't use indexes. If you have another condition that filters the records to a small set then you can use the $where to further filter it.
I'm trying to implement search over tags as part of a Texticle search. Since texticle doesn't search over multiple tables from the same model, I ended up creating a new model called PostSearch, following Texticle's suggestion about System-Wide Searching
class PostSearch < ActiveRecord::Base
# We want to reference various models
belongs_to :searchable, :polymorphic => true
# Wish we could eliminate n + 1 query problems,
# but we can't include polymorphic models when
# using scopes to search in Rails 3
# default_scope :include => :searchable
# Search.new('query') to search for 'query'
# across searchable models
def self.new(query)
debugger
query = query.to_s
return [] if query.empty?
self.search(query).map!(&:searchable)
#self.search(query) <-- this works, not sure why I shouldn't use it.
end
# Search records are never modified
def readonly?; true; end
# Our view doesn't have primary keys, so we need
# to be explicit about how to tell different search
# results apart; without this, we can't use :include
# to avoid n + 1 query problems
def hash
id.hash
end
def eql?(result)
id == result.id
end
end
In my Postgres DB I created a view like this:
CREATE VIEW post_searches AS
SELECT posts.id, posts.name, string_agg(tags.name, ', ') AS tags
FROM posts
LEFT JOIN taggings ON taggings.taggable_id = posts.id
LEFT JOIN tags ON taggings.tag_id = tags.id
GROUP BY posts.id;
This allows me to get posts like this:
SELECT * FROM post_searches
id | name | tags
1 Intro introduction, funny, nice
So it seems like that should all be fine. Unfortunately calling
PostSearch.new("funny") returns [nil] (NOT []). Looking through the Texticle source code, it seems like this line in the PostSearch.new
self.search(query).map!(&:searchable)
maps the fields using some sort of searchable_columns method and does it ?incorrectly? and results in a nil.
On a different note, the tags field doesn't get searched in the texticle SQL query unless I cast it from a text type to a varchar type.
So, in summary:
Why does the object get mapped to nil when it is found?
AND
Why does texticle ignore my tags field unless it is varchar?
Texticle maps objects to nil instead of nothing so that you can check for nil? - it's a safeguard against erroring out checking against non-existent items. It might be worth asking tenderlove himself as to exactly why he did it that way.
I'm not completely positive as to why Texticle ignores non-varchars, but it looks like it's a performance safeguard so that Postgres does not do full table scans (under the section Creating Indexes for Super Speed):
You will need to add an index for every text/string column you query against, or else Postgresql will revert to a full table scan instead of using the indexes.
I'm trying to do a simple query of a serialized column, how do you do this?
serialize :mycode, Array
1.9.3p125 :026 > MyModel.find(104).mycode
MyModel Load (0.6ms) SELECT `mymodels`.* FROM `mymodels` WHERE `mymodels`.`id` = 104 LIMIT 1
=> [43565, 43402]
1.9.3p125 :027 > MyModel.find_all_by_mycode("[43402]")
MyModel Load (0.7ms) SELECT `mymodels`.* FROM `mymodels` WHERE `mymodels`.`mycode` = '[43402]'
=> []
1.9.3p125 :028 > MyModel.find_all_by_mycode(43402)
MyModel Load (1.2ms) SELECT `mymodels`.* FROM `mymodels` WHERE `mymodels`.`mycode` = 43402
=> []
1.9.3p125 :029 > MyModel.find_all_by_mycode([43565, 43402])
MyModel Load (1.1ms) SELECT `mymodels`.* FROM `mymodels` WHERE `mymodels`.`mycode` IN (43565, 43402)
=> []
It's just a trick to not slow your application. You have to use .to_yaml.
exact result:
MyModel.where("mycode = ?", [43565, 43402].to_yaml)
#=> [#<MyModel id:...]
Tested only for MySQL.
Basically, you can't. The downside of #serialize is that you're bypassing your database's native abstractions. You're pretty much limited to loading and saving the data.
That said, one very good way to slow your application to a crawl could be:
MyModel.all.select { |m| m.mycode.include? 43402 }
Moral of the story: don't use #serialize for any data you need to query on.
Serialized array is stored in database in particular fashion eg:
[1, 2, 3, 4]
in
1\n 2\n 3\n etc
hence the query would be
MyModel.where("mycode like ?", "% 2\n%")
put space between % and 2.
Noodl's answer is right, but not entirely correct.
It really depends on the database/ORM adapter you are using: for instance PostgreSQL can now store and search hashes/json - check out hstore. I remember reading that ActiveRecord adapter for PostgreSQl now handles it properly. And if you are using mongoid or something like that - then you are using unstructured data (i.e. json) on a database level everywhere.
However if you are using a db that can't really handle hashes - like MySQL / ActiveRecord combination - then the only reason you would use serialized field is for somet data that you can create / write in some background process and display / output on demand - the only two uses that I found in my experience are some reports ( like a stat field on a Product model - where I need to store some averages and medians for a product), and user options ( like their preferred template color -I really don't need to query on that) - however user information - like their subscription for a mailing list - needs to be searchable for email blasts.
PostgreSQL hstore ActiveRecord Example:
MyModel.where("mycode #> 'KEY=>\"#{VALUE}\"'")
UPDATE
As of 2017 both MariaDB and MySQL support JSON field types.
You can query the serialized column with a sql LIKE statement.
MyModel.where("mycode LIKE '%?%'", 43402)
This is quicker than using include?, however, you cannot use an array as the parameter.
Good news! If you're using PostgreSQL with hstore (which is super easy with Rails 4), you can now totally search serialized data. This is a handy guide, and here's the syntax documentation from PG.
In my case I have a dictionary stored as a hash in an hstore column called amenities. I want to check for a couple queried amenities that have a value of 1 in the hash, I just do
House.where("amenities #> 'wifi => 1' AND amenities #> 'pool => 1'")
Hooray for improvements!
There's a blog post from 2009 from FriendFeed that describes how to use serialized data within MySQL.
What you can do is create tables that function as indexes for any data that you want to search.
Create a model that contains the searchable values/fields
In your example, the models would look something like this:
class MyModel < ApplicationRecord
# id, name, other fields...
serialize :mycode, Array
end
class Item < ApplicationRecord
# id, value...
belongs_to :my_model
end
Creating an "index" table for searchable fields
When you save MyModel, you can do something like this to create the index:
Item.where(my_model: self).destroy
self.mycode.each do |mycode_item|
Item.create(my_model: self, value: mycode_item)
end
Querying and Searching
Then when you want to query and search just do:
Item.where(value: [43565, 43402]).all.map(&:my_model)
Item.where(value: 43402).all.map(&:my_model)
You can add a method to MyModel to make that simpler:
def find_by_mycode(value_or_values)
Item.where(value: value_or_values).all.map(&my_model)
end
MyModel.find_by_mycode([43565, 43402])
MyModel.find_by_mycode(43402)
To speed things up, you will want to create a SQL index for that table.
Using the following comments in this post
https://stackoverflow.com/a/14555151/936494
https://stackoverflow.com/a/15287674/936494
I was successfully able to query a serialized Hash in my model
class Model < ApplicationRecord
serialize :column_name, Hash
end
When column_name holds a Hash like
{ my_data: [ { data_type: 'MyType', data_id: 113 } ] }
we can query it in following manner
Model.where("column_name = ?", hash.to_yaml)
That generates a SQL query like
Model Load (0.3ms) SELECT "models".* FROM "models" WHERE (column_name = '---
:my_data:
- :data_type: MyType
:data_id: 113
')
In case anybody is interested in executing the generated query in SQL terminal it should work, however care should be taken that value is in exact format stored in DB. However there is another easy way I found at PostgreSQL newline character to use a raw string containing newline characters
select * from table_name where column_name = E'---\n:my_data:\n- :data_type: MyType\n :data_id: 113\n'
The most important part in above query is E.
Note: The database on which I executed above is PostgreSQL.
To search serialized list you need to prefix and postfix the data with unique characters.
Example:
Rather than something like:
2345,12345,1234567 which would cause issues you tried to search for 2345 instead, you do something like <2345>,<12345>,<1234567> and search for <2345> (the search query get's transformed) instead. Of course choice of prefix/postfix characters depends on the valid data that will be stored. You might instead use something like ||| if you expect < to be used and potentially| to be used. Of course that increases the data the field uses and could cause performance issues.
Using a trigrams index or something would avoid potential performance issues.
You can serialize it like data.map { |d| "<#{d}>" }.join(',') and deserialize it via data.gsub('<').gsub('>','').split(','). A serializer class would do the job quite well to load/extract tha data.
The way you do this is by setting the database field to text and using rail's serialize model method with a custom lib class. The lib class needs to implement two methods:
def self.dump(obj) # (returns string to be saved to database)
def self.load(text) # (returns object)
Example with duration. Extracted from the article so link rot wouldn't get it, please visit the article for more information. The example uses a single value, but it's fairly straightforward to serialize a list of values and deserialize the list using the methods mentioned above.
class Duration
# Used for `serialize` method in ActiveRecord
class << self
def load(duration)
self.new(duration || 0)
end
def dump(obj)
unless obj.is_a?(self)
raise ::ActiveRecord::SerializationTypeMismatch,
"Attribute was supposed to be a #{self}, but was a #{obj.class}. -- #{obj.inspect}"
end
obj.length
end
end
attr_accessor :minutes, :seconds
def initialize(duration)
#minutes = duration / 60
#seconds = duration % 60
end
def length
(minutes.to_i * 60) + seconds.to_i
end
end
If you have serialized json column and you want to apply like query on that. do it like that
YourModel.where("hashcolumn like ?", "%#{search}%")
I have a Ruby on Rails site that uses thinking sphinx for searching a postgres database.
One of the fields in a table I am searching on is a boolean.
I'd like to be able to match on that boolean when a certain keyword is used in the search query.
To fully explain with an example:
My site is for people who develop their own black and white film.
I have a recipe table where people describe how they develop a film. That table has a boolean column called "stand_developed" (a method of developing film).
I'd like to return results where that field is true when the user searches for the word "stand".
I've been through the sphinx docs and not really found if it's possible.
I guess I could hack something inside my controller method by parsing the query terms and adding a condition but is there a cleaner way of doing it?
This is what I've done as far as searching on boolean fields using ThinkingSphinx. Pass stand_developed as a URL parameter along with your query_string in the following ways:
URL for a general query without search on stand_developed will be http://yoursite.com/search?q=your_query_string
URL for query with stand_developed == TRUE will be http://yoursite.com/search?q=your_query_string&stand_developed=1
URL for query with stand_developed == FALSE will be http://yoursite.com/search?q=your_query_string&stand_developed=0
Then, in your controller, you would do this:
if params[:stand_developed] && params[:stand_developed].to_i == 1
# perform query search with stand_developed == true
#search_results = YourModel.search(params[:q], :with => {:stand_developed => true})
elsif params[:stand_developed] && params[:stand_developed].to_i == 0
# perform query search with stand_developed == false
#search_results = YourModel.search(params[:q], :with => {:stand_developed => false})
else
# perform general query search
#search_results = YourModel.search(params[:q])
end
You could have just tested if params[:search] included the text 'stand' then done the searching from there. You don't need an extra column in your table, that's just overhead that's not needed.
if params[:search].downcase.include?('stand')
Model.search params[:search], :with => {:stand_developed => true}
else
Model.search params[:search]
end
I've come up with a solution to this now.
I tried the "hack" I mentioned in my question - parsing the word "stand" and searching for it explicitly (a variation on the answer from Paul Davis) but it didn't work very well.
I didn't explain this in my question (didn't realise the full implications at the time I asked) but I need it to also match if the user used the word "stand" in their recipe description too.
So, I tried to get sphinx to add a condition along the lines of "if stand_developed is true or notes contains 'stand'" but I couldn't seem to find the right syntax for that.
It also had to deal with any other search text too.
In the end I added an extra column to my recipe table called "search_tags" and I add the word "stand" into it if the user selects "stand_developed" when adding a recipe.
I then get Sphinx to index that field as well as my other fields and it all works perfectly.
I want to grab the most recent entry from a table. If I was just using sql, you could do
Select top 1 * from table ORDER BY EntryDate DESC
I'd like to know if there is a good active record way of doing this.
I could do something like:
table.find(:order => 'EntryDate DESC').first
But it seems like that would grab the entire result set, and then use ruby to select the first result. I'd like ActiveRecord to create sql that only brings across one result.
You need something like:
Model.first(:order => 'EntryDate DESC')
which is shorthand for
Model.find(:first, :order => 'EntryDate DESC')
Take a look at the documentation for first and find for details.
The Rails documentation seems to be pretty subjective in this instance. Note that .first is the same as find(:first, blah...)
From:http://api.rubyonrails.org/classes/ActiveRecord/Base.html#M002263
"Find first - This will return the first record matched by the options used. These options can either be specific conditions or merely an order. If no record can be matched, nil is returned. Use Model.find(:first, *args) or its shortcut Model.first(*args)."
Digging into the ActiveRecord code, at line 1533 of base.rb (as of 9/5/2009), we find:
def find_initial(options)
options.update(:limit => 1)
find_every(options).first
end
This calls find_every which has the following definition:
def find_every(options)
include_associations = merge_includes(scope(:find, :include), options[:include])
if include_associations.any? && references_eager_loaded_tables?(options)
records = find_with_associations(options)
else
records = find_by_sql(construct_finder_sql(options))
if include_associations.any?
preload_associations(records, include_associations)
end
end
records.each { |record| record.readonly! } if options[:readonly]
records
end
Since it's doing a records.each, I'm not sure if the :limit is just limiting how many records it's returning after the query is run, but it sure looks that way (without digging any further on my own). Seems you should probably just use raw SQL if you're worried about the performance hit on this.
Could just use find_by_sql http://api.rubyonrails.org/classes/ActiveRecord/Base.html#M002267
table.find_by_sql "Select top 1 * from table ORDER BY EntryDate DESC"