Inserting values in postgres but they are interpreted as column - ruby-on-rails

I am trying to insert some data in my postgres databae like this:
def insert_row(conn, row)
attendee = map_row_to_struct(row)
conn.execute(
<<-SQL
INSERT INTO tmp_attendee_import (email, first_name, last_name)
VALUES("#{attendee.email}", "#{attendee.first_name}", "#{attendee.last_name}");
SQL
)
end
The SQL is evaluated wonderfully as:
INSERT INTO tmp_attendee_import (email, first_name, last_name)
VALUES("myemail#yahoo.com", "Gigel", "Ion");
Yet I get this error:
Failure/Error:
conn.execute(
<<-SQL
INSERT INTO tmp_attendee_import (email, first_name, last_name)
VALUES("#{attendee.email}", "#{attendee.first_name}", "#{attendee.last_name}");
SQL
)
ActiveRecord::StatementInvalid:
PG::UndefinedColumn: ERROR: column "myemail#yahoo.com" does not exist
LINE 2: VALUES("myemail#yahoo.com", "Gigel", "Ion");
^
: INSERT INTO tmp_attendee_import (email, first_name, last_name)
VALUES("myemail#yahoo.com", "Gigel", "Ion");
Anyone has any clue ?

Postgres manual
A string constant in SQL is an arbitrary sequence of characters
bounded by single quotes ('), for example 'This is a string'. To
include a single-quote character within a string constant, write two
adjacent single quotes, e.g., 'Dianne''s horse'. Note that this is not
the same as a double-quote character (").
There is a second kind of identifier: the delimited identifier or
quoted identifier. It is formed by enclosing an arbitrary sequence of
characters in double-quotes ("). A delimited identifier is always an
identifier, never a key word. So "select" could be used to refer to a
column or table named "select", whereas an unquoted select would be
taken as a key word and would therefore provoke a parse error when
used where a table or column name is expected.
TL;DR: Single quotes for string constants, double quotes for table/column names.
BTW, the way you're choosing for inserting records is vulnerable to sql-injection.

Related

Postgres weird PG::UndefinedColumn: ERROR: on value

While executing a simple select - where operation using activerecord execute,
ActiveRecord::Base.connection.execute('select * from spree_variants where sku = "1SB-E4196-00";')
I got this error:
from /Users/abc/.rvm/gems/ruby-2.7.2#cboparts/gems/activerecord-6.0.3.5/lib/active_record/connection_adapters/postgresql/database_statements.rb:92:in `exec'
Caused by PG::UndefinedColumn: ERROR: column "1SB-E4196-00" does not exist
LINE 1: select * from spree_variants where sku = "1SB-E4196-00";
Why it is considering "1SB-E4196-00" as a column but not SKU? The error seems misleading.
Because PostgreSQL expects strings to be bounded in single quotes. While double quotes have a different meaning:
There is a second kind of identifier: the delimited identifier or quoted identifier. It is formed by enclosing an arbitrary sequence of characters in double-quotes ("). A delimited identifier is always an identifier, never a key word.
That means if the following query should work:
ActiveRecord::Base.connection.execute(
"select * from spree_variants where sku = '1SB-E4196-00';"
)
Btw you if you are using Rails and have a SpreeVariant model then you can see in the console how Rails formats and escapes the query like this:
puts SpreeVariant.where(sku: '1SB-E4196-00').to_sql

Split database string column twice in postgresql

I am using ruby on rails with postgresql. I have a table called "teams" with column "team_name". This "team_name" contains values like "1(Team1)", "1(Team2)", "2(Team1)", etc. Now want to take Team1, Team2, Team3 from those team names using sql query. If I use split_part as follows
sql = "SELECT SPLIT_PART(team_name,'(',2) FROM teams where id=20"
result = ActiveRecord::Base.connection.execute(sql).to_a
In the result I am getting
[{"split_part"=>"Team1)"}]
but I want the following result
[{"split_part"=>"Team1"}]
You can use SUBSTRING, but special characters must be escaped (backslash in this case):
Team
.select("SUBSTRING(team_name FROM '(Team\\d+)') AS team_name")
.where(id: id)

How to use Rails with uppercase column name?

I have the following as part of an AR query:
.having('COUNT(foo.id) > bar.maxUsers')
This generates an error:
ActiveRecord::StatementInvalid:
PG::UndefinedColumn: ERROR: column bar.maxusers does not exist
^
HINT: Perhaps you meant to reference the column "bar.maxUsers".
I am referencing the column bar.maxUsers.
So, apparently AR downcases the query. How to suppress this behavior?
Rails 4.2.10
PostgreSQL
EDIT: SQL:
SELECT ... HAVING COUNT(foo.id) > bar.maxUsers
So it is happening after the to_sql. Maybe from the execute?
This isn't an ActiveRecord or AREL issue, this is just how case sensitivity works in SQL and PostgreSQL.
Identifiers in SQL (such as table and column names) are case-insensitive unless they're quoted. Standard SQL says that unquoted identifiers are folded to upper case, PostgreSQL folds them to lower case, hence the bar.maxusers in the error message.
The solution is to quote the offending column name:
.having('COUNT(foo.id) > bar."maxUsers"')
Note that you must use double quotes for quoting the identifier as single quotes are only for string literals. Also note that identifier quoting is database-specific: standard SQL and PostgreSQL use double quotes, MySQL uses backticks, SQL Server uses brackets, ...

Rails reports can't find a column that is there

I am currently trying to do a complicated WHERE search on a table using Rails, the trouble is I get the error:
PG::Error: ERROR: column "email" does not exist
LINE 1: SELECT "bans".* FROM "bans" WHERE (Email='' AND IP='' AND (...
^
: SELECT "bans".* FROM "bans" WHERE (Email='' AND IP='' AND (Username='NULL' ))
And I know that column actually exists, and doing a rails dbconsole gives me the following:
Jungle=> select * from bans;
id | Username | IP | Email | Reason | Length | created_at | updated_at
----+----------+----+-------+--------+--------+------------+------------
(0 rows)
So this is definatly in the database, has anyone had any experience with this?
SQL column names are case insensitive unless quoted, the standard says that identifiers should be normalized to upper case but PostgreSQL normalizes to lower case:
Quoting an identifier also makes it case-sensitive, whereas unquoted names are always folded to lower case. For example, the identifiers FOO, foo, and "foo" are considered the same by PostgreSQL, but "Foo" and "FOO" are different from these three and each other. (The folding of unquoted names to lower case in PostgreSQL is incompatible with the SQL standard, which says that unquoted names should be folded to upper case. Thus, foo should be equivalent to "FOO" not "foo" according to the standard. If you want to write portable applications you are advised to always quote a particular name or never quote it.)
You're referencing Email in your SQL:
SELECT "bans".* FROM "bans" WHERE (Email='' ...
but PostgreSQL is complaining about email:
column "email" does not exist
Your unquoted Email is being treated as email because PostgreSQL normalizes identifiers to lower case. Sounds like you created the columns with capitalized names by double quoting them:
create table "bans" (
"Email" varchar(...)
...
)
or by using :Email to identify the column in a migration. If you quote a column name when it is created, then it is not normalized to lower case (or upper case in the SQL standard case) and you'll have to double quote it and match the case forever:
SELECT "bans".* FROM "bans" WHERE ("Email"='' ...
Once you fix Email, you'll have the same problem with IP, Username, Reason, and Length: you'll have to double quote them all in any SQL that references them.
The best practise is to use lower case column and table names so that you don't have to worry about quoting things all the time. I'd recommend that you fix your table to have lower case column names.
As an aside, your 'NULL' string literal:
SELECT "bans".* FROM "bans" WHERE (Email='' AND IP='' AND (Username='NULL' ))
-- -------------------->------------------>---------->---------------^^^^^^
looks odd, are you sure that you don't mean "Username" is null? The 'NULL' string literal and the NULL value are entirely different things and you can't use = or != to compare things against NULL, you have to use is null, is not null, is distinct from, or is not distinct from (depending on your intent) when NULLs might be in play.
It doesn't look like you're error is from a test database but if so try rake db:test:prepare.
In general, be aware that you have 3 databases - Test, Development, Production. So it's easy to get them mixed up and check the wrong one.
I had the same problem here,
but as mu-is-too-short said, PostgreSql can be told to
search with case sensitivity on columns names.
So by implemented this code I managed to bypass the same error that you're facing:
Transaction.find(:all,:conditions => ["(date between ? and ?) AND \"Membership_id\" = ?", Time.now.at_beginning_of_month, Time.now.end_of_month,membership.id])
Noticed the \" sign srrounding the column name? Well.. as annoying as it is, that's the fix to this problem.

How to convert a table column to another data type

I have a column with the type of Varchar in my Postgres database which I meant to be integers... and now I want to change them, unfortunately this doesn't seem to work using my rails migration.
change_column :table1, :columnB, :integer
Which seems to output this SQL:
ALTER TABLE table1 ALTER COLUMN columnB TYPE integer
So I tried doing this:
execute 'ALTER TABLE table1 ALTER COLUMN columnB TYPE integer USING CAST(columnB AS INTEGER)'
but cast doesn't work in this instance because some of the column are null...
any ideas?
Error:
PGError: ERROR: invalid input syntax for integer: ""
: ALTER TABLE table1 ALTER COLUMN columnB TYPE integer USING CAST(columnB AS INTEGER)
Postgres v8.3
It sounds like the problem is that you have empty strings in your table. You'll need to handle those, probably with a case statement, such as:
execute %{ALTER TABLE "table1" ALTER COLUMN columnB TYPE integer USING CAST(CASE columnB WHEN '' THEN NULL ELSE columnB END AS INTEGER)}
Update: completely rewritten based on updated question.
NULLs shouldnt be a problem here.
Tell us your postgresql version and your error message.
Besides, why are you quoting identifiers ? Be aware that unquoted identifiers are converted to lowercase (default behaviour), so there might be a problem with your "columnB" in your query - it appears quoted first, unquoted in the cast.
Update: Before converting a column to integer, you must be sure that all you values are convertible. In this case, it means that columnB should contains only digits (or null).
You can check this by something like
select columnB from table where not columnB ~ E'^[0-9]+$';
If you want your empty strings to be converted to NULL integers, then run first
UPDATE table set columnB = NULL WHERE columnB = '';

Resources