My development database is SQLite but I deploy my app to Heroku and they are using PostgreSQL.
Now sometimes I have two different results coming out if I perform searches because PostgreSQL is case-sensitive but SQLite is not.
Shouldn't Rails standardize these things? What methods should I use to solve that?
Here is how to fix it with raw SQL
Case insensitive searching in Postgres:
use ilike instead of like (case-insensitive like)
if you want to use =, make both sides either UPPER or LOWER
Another DDL way of handling this is the citext data type, or case-insensitive TEXT.
There are many relatively common things that ActiveRecord doesn't handle, and LIKE matching is one of them. Here is how to achieve this using straight Arel instead.
Model.where(
Model.arel_table[:title].matches("%#{search_term}%")
)
You can install Arel-Helpers to make this a little easier
Model.where(Model[:title].matches("%#{search_term}%"))
I previously recommended Squeel for this, but the original creator has stopped supporting it and there doesn't seem to be a full-time developer maintaining it. And since it plays around with private ActiveRecord APIs, it needs constant tending.
Related
I am having a database in Ruby on Rails application using SQLite3. Knowing my table, is there an open source project which can provide me an easy way of creating sqlite queries?
For example, based on the table names and order, I have a the available joins, and then I can add my conditions based on my variables.
Probably not.
Active record will provide you a lot of the functionalities you need to query, create and update records. It's not all that good at generating joins though.
The thing is, only you know the query you need to generate, hence there are very few "one-size-fits-all" query generator. It's a bit like asking "is there a Ruby code generator for generating my application". Well no, because only you knows what needs to be coded.
I tempted to say, keep your infrastructure simple, code the few queries you need rather than "rocket-science" a fancy generator that'll be hard to maintain. If in then end you need to generate a query from various arguments, concatenate strings and you'll get there faster than the time you'd take to research a generator.
I'm working on an RoR project where I have a model with a string attribute that needs to be entirely case insensitive. I'm just wondering what the best way to do this is.
I've seen some cases where people add setters to their models which uppercase the string before it's put in the database, and I've seen some people that make their queries case insensitive when search the database.
Is there a way to do it on the DB level? I'm using postgres. If not, what's the best way to approach this problem? Does the database take a big hit when searching case insensitive? What about from a security perspective? Is it more secure to check case insensitive every time to prevent problems should someone somehow create a lowercase row? Basically, what is best practice on this?
There are a number of options you can do:
You can set the local for the database that is case insensitive. Look at pg under Ubunto for an example. This is very OS dependent and tricky. E.g. Mac and RHEL pg is always case sensitive while Ubuntu pg is case insensitive by default.
While it has promise, I didn't have much success. The locales installed are OS dependent and I wasn't able to track down a case insensitive local nor information on creating one easily. Installing and distributing the local wasn't trivial. Mysql and SqlServer ship with their own locales while pg uses the OS provided locales. Using their own locales is less flexible and potentially introduces surprises for some languages, but it more consistent across operating systems since the locale is not tied to the operating system.
You can also install the citext extension. Create fields that are citext instead of varchar. Pretty sure extensions make their way into schema.rb now a days.
Model.where(:field => "AnY cAsE").
Using ILIKE will perform a case insensitive search as well even if you don't have a wildcard (%).
Model.where(Model.arel_table[:field].match("AnY cAsE"))
Create an index on lower(field) and use that to search the table. LIKE vs ILIKE have very different performance characteristics and LIKE tends to be much better.
Model.where(Model.arel_table[:field].lower.match("AnY cAsE".downcase, nil, true))
As mentioned above, Store all values as lowercase. Note: This can be in a separate field that is populated in the before_validation callback.
Model.where(:field_lower => "AnY cAsE".downcase)
Best of luck.
You probably solved this already, but thought I'd share what we've considered at my company for you and others.
A db column is never case insensitive. Your interaction may be case insensitive. As you saw, there are several possible cases and solutions.
If the attribute has sense only in a case, but you want to allow searching regardless the case, then always normalize it into a case when writing to the database and filter the queries that access to it. A typical example is an email address. You want to store it always lowercase and make sure to query it in such way, lowering the case of user inputs.
Vice-versa, if in the field you want to save the case is important (such as Title or brand) but you want to allow case insensitive interaction, you have basically two options
Add a separate field that contains the attribute always case insensitive and proceed as in the previous example
Use the database engine functions when you query the database in order to convert the value at runtime. Keep in mind that this will probably prevent the database from being able to use indexes, thus it may result in degraded performances.
I am using thinking_sphinx in Rails. As far as I know, Sphinx is used for full text search. Let's say I have these queries:
keyword
country
sort order
I use Sphinx for all the search above. However, when I am querying without keyword, but just country and sort order only, is it a better to use just normal query in MySQL instead of using Sphinx?
In other words, should Sphinx be used only when keyword is searched?
Looking at overall performance and speed.
Not to sound snarky, but does performance really matter?
If you're building an application which will only be used by a handful of users within an organization, then you can probably dismiss the performance benefits of using one method over the other and focus instead on simplicity in your code.
On the other hand, if your application is accessed by a large number of users on the interwebz and you really need to focus on being performant, then you should follow #barryhunter's advice above and benchmark to determine the best approach in a given circumstance.
P.S. Don't optimize before you need to. Fight with all your heart to keep code out of your code.
Benchmark! Benchmark! Benchmark!
Ie test it yourself. The exact performance will vary depending on the exact data, and perhaps even the relative perofrmance of your sphinx and mysql servers.
Sphinx will offer killer-speeds over MySQL when searching by a text string and MySQL will probably be faster when searching by a numerical key.
So, assuming that both "country" and "sort order" can be indexed using a numerical index in MySQL, it will be better to use Sphinx only with "keyword" and for the other two - MySQL normal query.
However, benchmarks won't hurt, as barryhunter suggested ;)
I am making an application with very huge data and multiple joins. Is it a bad practice to right away use the full sql string in rails? What are the downsides of writing the full sql query in rails?
It's only bad practice if you do it without understanding the alternatives.
That said there is rarely a reason to do this. The framework encapsulates it for you and the benefit is that you have to write less code. The other benefit is database independence. The more direct queries you write, the more likely you'll write something that will break when you switch database engines.
It is easy to test. If you are using the framework properly (i.e. optimizing ActiveRecord as you will find discussed in numerous articles) and still feel like your queries are too slow...you can always benchmark direct queries.
But not knowing how to do something using ActiveRecord associations is not a good reason to resort to direct SQL.
http://guides.rubyonrails.org/association_basics.html
SQL is not a 'bad practice' per se. Database systems have plenty of native SQL ways of doing things that would be much slower to execute and more complex to write and maintain if written in Ruby. Like Oracle's Analytic Functions.
That said, ActiveRecord is pretty easy to write and you probably aren't going to get a performance boost just by using a SQL query. At least not if the query you write resembles the query ActiveRecord would have written anyway! ;)
Perhaps you should try to work with ActiveRecord and only resort to SQL if you hit problems you can't solve another way. That way you keep your code simple until you need to do it another way (i.e. don't 'optimise early').
I generally try to make things work in ActiveRecord (or DataMapper or Sequel or whatever), but I have definitely resorted finder_sql when the job needed doing quickly and I couldn't get where I wanted to go using the ORM's 'sugar'. Other times I have based a rails object on a single massive view in the database.
Hope this helps.
:D
If you need more powerfull syntax than provides standard ActiveRecord module, see meta_where gem.
When we use ActiveRecord, we can use:
User.find(:first, :conditions=>["name=?", name])
It looks like ActiveRecord are using 'prepared statement', but after looking into the code, I found ActiveRecord just use String.dup and connection.quote() to adjust the content to build a sql, not like Java.
So, there is no real prepared statment in raiils? And why rails doesn't provide it?
If by prepared statement you mean a piece of SQL that is held in a compiled form for more efficient execution, then you're correct: it's not something implemented in Rails. There are several reasons for this:
Rails is the web framework - it doesn't know or care about databases. By default, Rails uses
ActiveRecord for object-relational mapping. You can change this if you want to - it doesn't have to be a RDBMS even.
ActiveRecord doesn't know about specific database platforms. For that it relies on "adapters", which translate AR's requirements into platform-specific SQL.
Some RDBMSs that have AR adapters support prepared statements (or equivalent) but others don't.
Rails 3.1 (or greater) supports prepared statements. Every new query you make will be added to pool of prepared statement. If you are are using MySql DB it still won't support prepared statement because using prepared statements in MySql reduces the performance compare to without prepared statement. Here is the nice article by Pat Shaughnessy on this.
In many (most?) ways, a DB View is a prepared statement. And it is easy to use views with ActiveRecord. Just declare the view in your DB (I use rake tasks) and then access it via ActiveRecord.
Works great for read-only access to complicated SQL. Saves the DB from many of the steps required to parse/compute SQL.