I am currently trying to do a complicated WHERE search on a table using Rails, the trouble is I get the error:
PG::Error: ERROR: column "email" does not exist
LINE 1: SELECT "bans".* FROM "bans" WHERE (Email='' AND IP='' AND (...
^
: SELECT "bans".* FROM "bans" WHERE (Email='' AND IP='' AND (Username='NULL' ))
And I know that column actually exists, and doing a rails dbconsole gives me the following:
Jungle=> select * from bans;
id | Username | IP | Email | Reason | Length | created_at | updated_at
----+----------+----+-------+--------+--------+------------+------------
(0 rows)
So this is definatly in the database, has anyone had any experience with this?
SQL column names are case insensitive unless quoted, the standard says that identifiers should be normalized to upper case but PostgreSQL normalizes to lower case:
Quoting an identifier also makes it case-sensitive, whereas unquoted names are always folded to lower case. For example, the identifiers FOO, foo, and "foo" are considered the same by PostgreSQL, but "Foo" and "FOO" are different from these three and each other. (The folding of unquoted names to lower case in PostgreSQL is incompatible with the SQL standard, which says that unquoted names should be folded to upper case. Thus, foo should be equivalent to "FOO" not "foo" according to the standard. If you want to write portable applications you are advised to always quote a particular name or never quote it.)
You're referencing Email in your SQL:
SELECT "bans".* FROM "bans" WHERE (Email='' ...
but PostgreSQL is complaining about email:
column "email" does not exist
Your unquoted Email is being treated as email because PostgreSQL normalizes identifiers to lower case. Sounds like you created the columns with capitalized names by double quoting them:
create table "bans" (
"Email" varchar(...)
...
)
or by using :Email to identify the column in a migration. If you quote a column name when it is created, then it is not normalized to lower case (or upper case in the SQL standard case) and you'll have to double quote it and match the case forever:
SELECT "bans".* FROM "bans" WHERE ("Email"='' ...
Once you fix Email, you'll have the same problem with IP, Username, Reason, and Length: you'll have to double quote them all in any SQL that references them.
The best practise is to use lower case column and table names so that you don't have to worry about quoting things all the time. I'd recommend that you fix your table to have lower case column names.
As an aside, your 'NULL' string literal:
SELECT "bans".* FROM "bans" WHERE (Email='' AND IP='' AND (Username='NULL' ))
-- -------------------->------------------>---------->---------------^^^^^^
looks odd, are you sure that you don't mean "Username" is null? The 'NULL' string literal and the NULL value are entirely different things and you can't use = or != to compare things against NULL, you have to use is null, is not null, is distinct from, or is not distinct from (depending on your intent) when NULLs might be in play.
It doesn't look like you're error is from a test database but if so try rake db:test:prepare.
In general, be aware that you have 3 databases - Test, Development, Production. So it's easy to get them mixed up and check the wrong one.
I had the same problem here,
but as mu-is-too-short said, PostgreSql can be told to
search with case sensitivity on columns names.
So by implemented this code I managed to bypass the same error that you're facing:
Transaction.find(:all,:conditions => ["(date between ? and ?) AND \"Membership_id\" = ?", Time.now.at_beginning_of_month, Time.now.end_of_month,membership.id])
Noticed the \" sign srrounding the column name? Well.. as annoying as it is, that's the fix to this problem.
Related
I have the following as part of an AR query:
.having('COUNT(foo.id) > bar.maxUsers')
This generates an error:
ActiveRecord::StatementInvalid:
PG::UndefinedColumn: ERROR: column bar.maxusers does not exist
^
HINT: Perhaps you meant to reference the column "bar.maxUsers".
I am referencing the column bar.maxUsers.
So, apparently AR downcases the query. How to suppress this behavior?
Rails 4.2.10
PostgreSQL
EDIT: SQL:
SELECT ... HAVING COUNT(foo.id) > bar.maxUsers
So it is happening after the to_sql. Maybe from the execute?
This isn't an ActiveRecord or AREL issue, this is just how case sensitivity works in SQL and PostgreSQL.
Identifiers in SQL (such as table and column names) are case-insensitive unless they're quoted. Standard SQL says that unquoted identifiers are folded to upper case, PostgreSQL folds them to lower case, hence the bar.maxusers in the error message.
The solution is to quote the offending column name:
.having('COUNT(foo.id) > bar."maxUsers"')
Note that you must use double quotes for quoting the identifier as single quotes are only for string literals. Also note that identifier quoting is database-specific: standard SQL and PostgreSQL use double quotes, MySQL uses backticks, SQL Server uses brackets, ...
I am getting this error in the pg production mode, but its working fine in sqlite3 development mode.
ActiveRecord::StatementInvalid in ManagementController#index
PG::Error: ERROR: column "estates.id" must appear in the GROUP BY clause or be used in an aggregate function
LINE 1: SELECT "estates".* FROM "estates" WHERE "estates"."Mgmt" = ...
^
: SELECT "estates".* FROM "estates" WHERE "estates"."Mgmt" = 'Mazzey' GROUP BY user_id
#myestate = Estate.where(:Mgmt => current_user.Company).group(:user_id).all
If user_id is the PRIMARY KEY then you need to upgrade PostgreSQL; newer versions will correctly handle grouping by the primary key.
If user_id is neither unique nor the primary key for the 'estates' relation in question, then this query doesn't make much sense, since PostgreSQL has no way to know which value to return for each column of estates where multiple rows share the same user_id. You must use an aggregate function that expresses what you want, like min, max, avg, string_agg, array_agg, etc or add the column(s) of interest to the GROUP BY.
Alternately you can rephrase the query to use DISTINCT ON and an ORDER BY if you really do want to pick a somewhat arbitrary row, though I really doubt it's possible to express that via ActiveRecord.
Some databases - including SQLite and MySQL - will just pick an arbitrary row. This is considered incorrect and unsafe by the PostgreSQL team, so PostgreSQL follows the SQL standard and considers such queries to be errors.
If you have:
col1 col2
fred 42
bob 9
fred 44
fred 99
and you do:
SELECT col1, col2 FROM mytable GROUP BY col1;
then it's obvious that you should get the row:
bob 9
but what about the result for fred? There is no single correct answer to pick, so the database will refuse to execute such unsafe queries. If you wanted the greatest col2 for any col1 you'd use the max aggregate:
SELECT col1, max(col2) AS max_col2 FROM mytable GROUP BY col1;
I recently moved from MySQL to PostgreSQL and encountered the same issue. Just for reference, the best approach I've found is to use DISTINCT ON as suggested in this SO answer:
Elegant PostgreSQL Group by for Ruby on Rails / ActiveRecord
This will let you get one record for each unique value in your chosen column that matches the other query conditions:
MyModel.where(:some_col => value).select("DISTINCT ON (unique_col) *")
I prefer DISTINCT ON because I can still get all the other column values in the row. DISTINCT alone will only return the value of that specific column.
After often receiving the error myself I realised that Rails (I am using rails 4) automatically adds an 'order by id' at the end of your grouping query. This often results in the error above. So make sure you append your own .order(:group_by_column) at the end of your Rails query. Hence you will have something like this:
#problems = Problem.select('problems.username, sum(problems.weight) as weight_sum').group('problems.username').order('problems.username')
#myestate1 = Estate.where(:Mgmt => current_user.Company)
#myestate = #myestate1.select("DISTINCT(user_id)")
this is what I did.
How can I order query results by an hstore attribute?
#items = Item.includes(:product).order('products.properties #> hstore("platform")')
Causes
PG::Error: ERROR: column "platform" does not exist
LINE 1: ...oduct_id" ORDER BY products.properties #> hstore("platform"...
platform is a hstore key, stored in the properties column, which is an hstore type.
Double quotes are used to quote identifiers (such as table and column names) in PostgreSQL (and other databases that follow the standard). So when you say:
hstore("platform")
PostgreSQL sees "platform" as a quoted column name and since there is no platform column, you get an error.
Strings in standard SQL are quoted with single quotes, you want to say:
.order("products.properties #> hstore('platform')")
This will probably still fail though, hstore('platform') doesn't make much sense and neither does using #> here; a #> b means
does the hstore a contain the hstore b
If you're trying to sort on the value of the 'platform' key in the properties hstore then you'd want to use -> to lookup the 'platform' key like this:
.order("products.properties -> 'platform'")
In rails 3.0.0, the following query works fine:
Author.where("name LIKE :input",{:input => "#{params[:q]}%"}).includes(:books).order('created_at')
However, when I input as search string (so containing a double colon followed by a dot):
aa:.bb
I get the following exception:
ActiveRecord::StatementInvalid: SQLite3::SQLException: ambiguous column name: created_at
In the logs the these are the sql queries:
with aa as input:
Author Load (0.4ms) SELECT "authors".* FROM "authors" WHERE (name LIKE 'aa%') ORDER BY created_at
Book Load (2.5ms) SELECT "books".* FROM "books" WHERE ("books".author_id IN (1,2,3)) ORDER BY id
with aa:.bb as input:
SELECT DISTINCT "authors".id FROM "authors" LEFT OUTER JOIN "books" ON "books"."author_id" = "authors"."id" WHERE (name LIKE 'aa:.bb%') ORDER BY created_at DESC LIMIT 12 OFFSET 0
SQLite3::SQLException: ambiguous column name: created_at
It seems that with the aa:.bb input, an extra query is made to fetch the distinct author id_s.
I thought Rails would escape all the characters. Is this expected behaviour or a bug?
Best Regards,
Pieter
The "ambiguous column" error usually happens when you use includes or joins and don't specify which table you're referring to:
"name LIKE :input"
Should be:
"authors.name LIKE :input"
Just "name" is ambiguous if your books table has a name column too.
Also: have a look at your development.log to see what the generated query looks like. This will show you if it's being escaped properly.
Replace
.includes(:books)
with
.preload(:books)
This should force activerecord to use 2 queries instead of the join.
Rails has 2 versions of includes: One which constructs a big query with joins (the 2nd of your 2 queries and thus more likely to result in ambiguous column references and one that avoids the joins in favour of a separate query per association.
Rails decides which strategy to used based on whether it thinks that your conditions, order etc refer to the included tables (since in that case the joins version is required). Where a condition is a string fragment that heuristic isn't very sophisticated - i seem to recall that it just scans the conditions for anything that might look like a column from another table (ie foo.bar) so having a literal of that form could fool it.
You can either qualify your column names so that it doesn't matter which includes strategy is used or you can use preload/eager_load instead of includes. These behave similarly to includes but force a specific include strategy rather than trying to guess which is most appropriate.
I have a column with the type of Varchar in my Postgres database which I meant to be integers... and now I want to change them, unfortunately this doesn't seem to work using my rails migration.
change_column :table1, :columnB, :integer
Which seems to output this SQL:
ALTER TABLE table1 ALTER COLUMN columnB TYPE integer
So I tried doing this:
execute 'ALTER TABLE table1 ALTER COLUMN columnB TYPE integer USING CAST(columnB AS INTEGER)'
but cast doesn't work in this instance because some of the column are null...
any ideas?
Error:
PGError: ERROR: invalid input syntax for integer: ""
: ALTER TABLE table1 ALTER COLUMN columnB TYPE integer USING CAST(columnB AS INTEGER)
Postgres v8.3
It sounds like the problem is that you have empty strings in your table. You'll need to handle those, probably with a case statement, such as:
execute %{ALTER TABLE "table1" ALTER COLUMN columnB TYPE integer USING CAST(CASE columnB WHEN '' THEN NULL ELSE columnB END AS INTEGER)}
Update: completely rewritten based on updated question.
NULLs shouldnt be a problem here.
Tell us your postgresql version and your error message.
Besides, why are you quoting identifiers ? Be aware that unquoted identifiers are converted to lowercase (default behaviour), so there might be a problem with your "columnB" in your query - it appears quoted first, unquoted in the cast.
Update: Before converting a column to integer, you must be sure that all you values are convertible. In this case, it means that columnB should contains only digits (or null).
You can check this by something like
select columnB from table where not columnB ~ E'^[0-9]+$';
If you want your empty strings to be converted to NULL integers, then run first
UPDATE table set columnB = NULL WHERE columnB = '';