I have the unfortunate task of making some legacy data work with a rails ap. Some of the id fields are strings
:client_id => "30430"
But in postgres (which we’d prefer to use) the association query chokes because of the data type mismatch. Is there a way around this?
PG::UndefinedFunction: ERROR: operator does not exist: bigint = character varying LINE 1: ...ients" INNER JOIN "reservation" ON "clients"."id" = "reserva...
No operator matches the given name and argument type(s). You might need to add explicit type casts.
To be explicit, I'm looking for solutions that don't involve altering the underlying data (though that would be my first choice)
At first I thought you could just write a migration to convert the column type to integer, which works with mysql but apparently not postgres:
https://makandracards.com/makandra/18691-postgresql-vs-rails-migration-how-to-change-columns-from-string-to-integer
Assuming your table is called products, the following should work:
change_column :products, :client_id, 'integer USING CAST(column_name AS integer)'
As the link states, any strings with a non numeric value will obviously give weird results, if not throw an error. And it's obviously worth trying out locally first.
Related
I'm trying to convert an existing column of type varchar to jsonb. The column contains strings like "black white orange" and want to convert it into jsonb format such that it will be converted to ["black", "white", "orange"].
class AlterColorsDatatype < ActiveRecord::Migration[5.0]
def change
change_column :quotes, :colors, :jsonb, default: '[]', using: 'colors::jsonb'
end
end
I expected to this to convert the column type to jsonb and the using: part would convert existing data to jsonb as well.
Instead, I get this error:
ActiveRecord::StatementInvalid: PG::InvalidTextRepresentation: ERROR: invalid input syntax for type json
DETAIL: Token "Indigo" is invalid.
CONTEXT: JSON data, line 1: Indigo
: ALTER TABLE "quotes" ALTER COLUMN "colors" TYPE jsonb USING colors::jsonb
I have tried other syntax but still ends up with the same error. I am thinking I would have to convert the whole column attribute by attribute with something like to_json but am not sure how to approach solving this error. After many google searches, other people with the same error did not seem to find a solution.
You can't get from a space-delimited string to a JSON with a simple cast. An easy way is to first break the string up to get a PostgreSQL array (text[]):
regexp_split_to_array(colors, E'\\s+')
and then convert that array to JSON:
to_json(regexp_split_to_array(colors, E'\\s+'))
You have to be careful with the quoting and backslashes to get that bit of SQL through Ruby and into the database so you'd say:
using: %q{to_json(regexp_split_to_array(colors, E'\\\\s+'))}
The %q{...} is like a single quoted string but lets you avoid having to escape the single quotes in the SQL string literal, then double up your backslashes to keep them from being interpreted by %q{...}.
I have three rails objects: User, DemoUser and Stats. Both the User and the DemoUser have many stats associated with them. The User and Stats tables are stored on Postgresql (using ActiveRecord). The DemoUser is stored in redis. The id for the DemoUser is a (random) string. The id for the User is a (standard-rails) incrementing integer.
The stats table has a user_id column that can contain either the User id or the DemoUser id. For that reason, the user_id column is a string, rather than an integer.
There isn't an easy way to translate from the random string to an integer, but there's a very easy way to translate the integer id to a string (42 -> "42"). The ids are guaranteed not to overlap (there won't be a User instance with the same id as a DemoUser, ever).
I have some code that manages those stats. I'd like to be able to pass over a some_user instance (which can either be a DemoUser or a User) and then be able to use the id to fetch Stats, update them etc. Also would be nice to be able to define a has_many for the User model, so I can do things like user.stats
However, operations like user.stats would create a query like
SELECT "stats".* FROM "stats" WHERE "stats"."user_id" = 42
which then breaks with PG::UndefinedFunction: ERROR: operator does not exist: character varying = integer
Is there a way to either let the database (Postgresql), or Rails do auto-translation of the ids on JOIN? (the translation from integer to string should be simple, e.g. 42 -> "42")
EDIT: updated the question to try to make things as clear as possible. Happy to accept edits or answer questions to clarify anything.
You can't define a foreign key between two types that don't have built-in equality operators.
The correct solution is to change the string column to be an integer.
In your case you could create a user-defined = operator for varchar = string, but that would have messy side effects elsewhere in the database; for example, it would allow bogus code like:
SELECT 2014-01-02 = '2014-01-02'
to run without an error. So I'm not going to give you the code to do that. If you truly feel it's the only solution (which I don't think is likely to be correct) then see CREATE OPERATOR and CREATE FUNCTION.
One option would be to have separate user_id and demo_user_id columns in your stats table. The user_id would be an integer that you could use as a foreign key to the users table in PostgreSQL and the demo_user_id would be a string that would link to your Redis database. If you wanted to treat the database properly, you'd use a real FK to link stats.user_id to users.id to ensure referential integrity and you'd include a CHECK constraint to ensure that exactly one of stats.user_id and stats.demo_user_id was NULL:
check (user_id is null <> demo_user_id is null)
You'll have to fight ActiveRecord a bit to properly constrain your database of course, AR doesn't believe in fancy things like FKs and CHECKs even though they are necessary for data integrity. You'd have to keep demo_user_id under control by hand though, some sort of periodic scan to make sure they link up with values in Redis would be a good idea.
Now your User can look up stats using a standard association to the stats.user_id column and your DemoUser can use stats.demo_user_id.
For the time being, my 'solution' is not to use a has_many in Rails, but I can define some helper functions in the models if necessary. e.g.
class User < ActiveRecord::Base
# ...
def stats
Stats.where(user_id: self.id.to_s)
end
# ...
end
also, I would define some helper scopes to help enforce the to_s translation
class Stats < ActiveRecord::Base
scope :for_user_id, -> (id) { where(user_id: id.to_s) }
# ...
end
This should allow calls like
user.stats and Stats.for_user_id(user.id)
I think I misunderstood a detail of your issue before because it was buried in the comments.
(I strongly suggest editing your question to clarify points when comments show that there's something confusing/incomplete in the question).
You seem to want a foreign key from an integer column to a string column because the string column might be an integer, or might be some unrelated string. That's why you can't make it an integer column - it's not necessarily a valid number value, it might be a textual key from a different system.
The typical solution in this case would be to have a synthetic primary key and two UNIQUE constraints instead, one for keys from each system, plus a CHECK constraint preventing both from being set. E.g.
CREATE TABLE my_referenced_table (
id serial,
system1_key integer,
system2_key varchar,
CONSTRAINT exactly_one_key_must_be_set
CHECK (system1_key IS NULL != system2_key IS NULL),
UNIQUE(system1_key),
UNIQUE(system2_key),
PRIMARY KEY (id),
... other values ...
);
You can then have a foreign key referencing system1_key from your integer-keyed table.
It's not perfect, as it doesn't prevent the same value appearing in two different rows, one for system1_key and one for system2_key.
So an alternative might be:
CREATE TABLE my_referenced_table (
the_key varchar primary key,
the_key_ifinteger integer,
CONSTRAINT integerkey_must_equal_key_if_set
CHECK (the_key_ifinteger IS NULL OR (the_key_ifinteger::varchar = the_key)),
UNIQUE(the_key_ifinteger),
... other values ...
);
CREATE OR REPLACE FUNCTION my_referenced_table_copy_int_key()
RETURNS trigger LANGUAGE plpgsql STRICT
AS $$
BEGIN
IF NEW.the_key ~ '^[\d]+$' THEN
NEW.the_key_ifinteger := CAST(NEW.the_key AS integer);
END IF;
RETURN NEW;
END;
$$;
CREATE TRIGGER copy_int_key
BEFORE INSERT OR UPDATE ON my_referenced_table
FOR EACH ROW EXECUTE PROCEDURE my_referenced_table_copy_int_key();
which copies the integer value if it's an integer, so you can reference it.
All in all though I think the whole idea is a bit iffy.
I think I may have a solution for your problem, but maybe not a massively better one:
class User < ActiveRecord::Base
has_many :stats, primary_key: "id_s"
def id_s
read_attribute(:id).to_s
end
end
Still uses a second virtual column, but maybe more handy to use with Rails associations and is database agnostic.
I'm trying to change my polygon column type to a multipolygon column type.
My code is a simple line.
change_column :messages, :area_shape, :multipolygon, srid: 3785
But Postgres doesn't know this type. I thought that I missed something on my PostGIS configuration but I can't see it.
This my exact error:
rake aborted!
PG::UndefinedObject: ERROR: type "multipolygon" does not exist
: ALTER TABLE "messages" ALTER COLUMN "area_shape" TYPE multipolygon
This is how I've created my area_shape as a polygon type:
add_column :messages, :area_shape, :polygon, srid: 3785
Thank you for your help.
I have no idea how RGeo attempts to implement change_column (is there documentation for it?), but it isn't correct since there is no such multipolygon type.
If you have direct access to PostgreSQL, following from this answer, use this DDL:
ALTER TABLE my_table
ALTER COLUMN area_shape TYPE geometry(MultiPolygon,3785)
USING ST_Multi(area_shape);
Finally I had to remove then recreate my column:
remove_column :messages, :area_shape
add_column :messages, :area_shape, :multi_polygon, srid: 3785
I think I can now understand why it's not possible. Indeed, it seems difficult to change a polygon type to a multi_polygon type without losing data logique.
If you really need to change the type, you can use what was said by #Mike (manually) and create a small method to convert polygon to multi_polygon but it's not really safe in my mind.
Tip: a multi_polygon type is an Enumerable that means multi_polygon accepts Array type.
can anyone please tell me how to write query in select_value.
I have tried,
ActiveRecord::Base.connection.select_value("select count(*) from leave_details where status= 'Pending' and 'employeedetails_id'=25")
but it showing error
invalid input syntax for integer: "employeedetails_id".
I am using PostgreSQL.
Single quotes are used to quote strings in PostgreSQL (and every other SQL database that even pretends to respect the SQL standard) so you're saying something like this:
some_string = some_integer
when you do this:
'employeedetails_id'=25
and that doesn't make any sense: you can't compare strings and integers without an explicit type cast. You don't need to quote that identifier at all:
ActiveRecord::Base.connection.select_value(%q{
select count(*)
from leave_details
where status = 'Pending'
and employeedetails_id = 25
})
If you even do need to quote an identifier (perhaps it is case sensitive or contains spaces), then you'd use double quotes with PostgreSQL.
Apparently you created your column as "EmployeeDetails_id" so that it is case sensitive. That means that you always have to use that case and you always have to double quote it:
ActiveRecord::Base.connection.select_value(%q{
select count(*)
from leave_details
where status = 'Pending'
and "EmployeeDetails_id" = 25
})
I'd recommend reworking your table to not use mixed case identifiers:
They go against standard Ruby/Rails naming.
They force you to double quote the mixed case column names everywhere you use them.
They go against standard PostgreSQL practice.
This is going to trip you up over and over again.
Executing SQL directly isn't really The Rails Way, and you lose any database portability by doing it that way.
You should create a model for leave_details. E.g.
rails g model LeaveDetails status:string employeedetails_id:integer
Then, the code would be:
LeaveDetails.where({ :status => 'Pending', :employeedetails_id => 25 }).count
I have a Rails app using a Postgres database with a table called geolite_blocks. If I call ActiveRecord like this:
GeoliteBlock.find_by_startIpNum 2776360991
The query works perfectly. However, if I do the query like this:
GeoliteBlock.where("startIpNum >= ?", 2776360991)
I get this error:
ActiveRecord::StatementInvalid: PGError: ERROR: column "startipnum" does not exist
LINE 1: ... "geolite_blocks".* FROM "geolite_blocks" WHERE (startIpNum...
^
: SELECT "geolite_blocks".* FROM "geolite_blocks" WHERE (startIpNum >= 2776360991)
But I know that the column exists because I just queried by it with the first code example. Any ideas as to why this might be happening, and how I can eliminate it? Thanks for any help!
Column names in SQL are case insensitive unless they were quoted when they were created. Someone created your startIpNum column with quotes around it so you have to quote it every time you use it:
GeoliteBlock.where('"startIpNum" >= ?', 2776360991)
The error you're getting from PostgreSQL mentions startipnum because PostgreSQL normalizes identifiers to lower case (the SQL standard says that they should be normalized to upper case though).
This:
GeoliteBlock.find_by_startIpNum 2776360991
works because AR will quote the startIpNuM part behind your back. Similarly, GeoliteBlock.where(:startIpNum => 2776360991) would also work.
I'd recommend that you change the schema to use lower case column names so that you won't have to worry about this anymore.