I'm trying to convert an existing column of type varchar to jsonb. The column contains strings like "black white orange" and want to convert it into jsonb format such that it will be converted to ["black", "white", "orange"].
class AlterColorsDatatype < ActiveRecord::Migration[5.0]
def change
change_column :quotes, :colors, :jsonb, default: '[]', using: 'colors::jsonb'
end
end
I expected to this to convert the column type to jsonb and the using: part would convert existing data to jsonb as well.
Instead, I get this error:
ActiveRecord::StatementInvalid: PG::InvalidTextRepresentation: ERROR: invalid input syntax for type json
DETAIL: Token "Indigo" is invalid.
CONTEXT: JSON data, line 1: Indigo
: ALTER TABLE "quotes" ALTER COLUMN "colors" TYPE jsonb USING colors::jsonb
I have tried other syntax but still ends up with the same error. I am thinking I would have to convert the whole column attribute by attribute with something like to_json but am not sure how to approach solving this error. After many google searches, other people with the same error did not seem to find a solution.
You can't get from a space-delimited string to a JSON with a simple cast. An easy way is to first break the string up to get a PostgreSQL array (text[]):
regexp_split_to_array(colors, E'\\s+')
and then convert that array to JSON:
to_json(regexp_split_to_array(colors, E'\\s+'))
You have to be careful with the quoting and backslashes to get that bit of SQL through Ruby and into the database so you'd say:
using: %q{to_json(regexp_split_to_array(colors, E'\\\\s+'))}
The %q{...} is like a single quoted string but lets you avoid having to escape the single quotes in the SQL string literal, then double up your backslashes to keep them from being interpreted by %q{...}.
Related
A user model has a goal field that is an array:
t.integer "goal", default: [], array: true
A valid entry for goal is [[],[],[]]. If I do
Match.create(goal: [[],[],[]])
I get an exception:
ActiveRecord::StatementInvalid: PG::InvalidTextRepresentation: ERROR: malformed array literal: "{{},{},{}}"
DETAIL: Unexpected "}" character.
I know that I can work around this by saving it as goal: [], but I lose information about the number of sub-arrays. Is there a way to fix this?
You’ve gone from trying to store an array of numbers to an array of fixed length which includes null values or even sub arrays. Therefore, a t.integer field is no good. You need to use a t.jsonb or t.json field.
See the Postgres docs on JSON datatypes
I have the unfortunate task of making some legacy data work with a rails ap. Some of the id fields are strings
:client_id => "30430"
But in postgres (which we’d prefer to use) the association query chokes because of the data type mismatch. Is there a way around this?
PG::UndefinedFunction: ERROR: operator does not exist: bigint = character varying LINE 1: ...ients" INNER JOIN "reservation" ON "clients"."id" = "reserva...
No operator matches the given name and argument type(s). You might need to add explicit type casts.
To be explicit, I'm looking for solutions that don't involve altering the underlying data (though that would be my first choice)
At first I thought you could just write a migration to convert the column type to integer, which works with mysql but apparently not postgres:
https://makandracards.com/makandra/18691-postgresql-vs-rails-migration-how-to-change-columns-from-string-to-integer
Assuming your table is called products, the following should work:
change_column :products, :client_id, 'integer USING CAST(column_name AS integer)'
As the link states, any strings with a non numeric value will obviously give weird results, if not throw an error. And it's obviously worth trying out locally first.
In my Rails 5 app, I read in a feed for products. In the JSON, when the price is over $1,000, it the JSON has a comma, like 1,000.
My code seems to be truncating it, so it's storing as 1 instead of 1,000.
All other fields are storing correctly. Can someone please tell me what I'm doing wrong?
In this example, the reg_price saves as 2, instead of 2590.
json sample (for reg_price field):
[
{
"reg_price": "2,590"
}
]
schema
create_table "products", force: :cascade do |t|
t.decimal "reg_price", precision: 10, scale: 2
end
model
response = open_url(url_string).to_s
products = JSON.parse(response)
products.each do |product|
product = Product.new(
reg_price: item['reg_price']
)
product.save
end
You are not doing anything wrong. Decimals don't work with comma separator. I'm not sure there is a nice way to fix the thing. But as an option you could define a virtual attribute:
def reg_price=(reg_price)
self[:reg_price] = reg_price.gsub(',', '')
end
The reason this is happening has nothing to do with Rails.
JSON is a pretty simple document structure and doesn't have any support for number separators. The values in your JSON document are strings.
When you receive a String as input and you want to store it as an Integer, you need to cast it to the appropriate type.
Ruby has built in support for this, and Rails is using it: "1".to_s #=> 1
The particular heuristic Ruby uses to convert a string to an integer is to take any number up to a non-numerical character and cast it as an integer. Commas are non-numeric, at least by default, in Ruby.
The solution is to convert the string value in your JSON to an integer using another method. You can do this any of these ways:
Cast the string to an integer before sending it to your ActiveRecord model.
Alter the string in such a way that the default Ruby casting will cast the string into the expected value.
Use a custom caster to handle the casting for this particular attribute (inside of ActiveRecord and ActiveModel).
The solution proposed by #Danil follows #2 above, and it has some shortcomings (as #tadman pointed out).
A more robust way of handling this without getting down in the mud is to use a library like Delocalize, which will automatically handle numeric string parsing and casting with consideration for separators used by the active locale. See this excellent answer by Benoit Garret for more information.
I am seeing Rails generate the wrong query value when I use an enum col value in a where clause, like this (self added for clarity). dominant_product_strategy is an enum.
def some_model_method_on_myModel
MyModel.where(dominant_product_strategy: self.dominant_product_strategy)
end
This produces the correct value (again, self just added for clarity):
MyModel.where(dominant_product_strategy: self.attributes["dominant_product_strategy"])
I'm guessing that Rails sees the enum as a string, and then converts to a integer value of zero. Ughhhhh!
Am I missing something?
This also works:
MyModelwhere(dominant_product_strategy: MyModel.dominant_product_strategies[dominant_product_strategy])
It seems to be that you have answered your question by yoursef. Enum variables is a hash:
{str1: int1, str2: int2, ...}
The value (integer) is storing in DB, and the string is just representation of the int value. When you call self.dominant_product_strategy, you get the representation (string) of dominant_product_strategy column storing as integer in DB.
I think that your first working solution (self.attributes["dominant_product_strategy"]) is fine.
I have a Rails model that has a field array_field, which is a serialized text array. I want the combination of this array value and the value of another_field to be unique.
Should be straightforward, no?
class Foo < ActiveRecord::Base
validates_uniqueness_of :array_field, scope: [:another_field]
serialize :filters, Array
end
This doesn't work. However, if I switch them around in the validations,
validates_uniqueness_of :another_field, scope: [:array_field] works as expected.
Can someone explain why this is the case? Is this expected behavior?
The Postgres error for the former setup when array_field's value is nil or [] is this:
PG::SyntaxError: ERROR: syntax error at or near ")"
LINE 1: ...other_field" = 103 AND "foo"."array_field" = ) LIMIT 1
When array_field is [[1, 2], [3, 4, 5]] (a sample multiarray I was using), it's:
PG::UndefinedFunction: ERROR: operator does not exist: text = integer
LINE 1: ...other_field" = 103 AND "foo"."array_field" = 1, 2, 3, 4, 5) LIMIT 1
It seems that Rails doesn't know how to translate the serialized object for this query. Am I missing something or is this a bug?
Edit: This is occurring in Rails 4.0.2.
Second Edit:
Clarification: I understand why this is happening (Rails has custom logic for list queries), and I'm using both a custom validator to manually perform the serialization before validating and a custom serializer to avoid problems with comparisons of Yaml strings (as detailed in my other question here).
At this point I'm mostly just wondering why validates_uniqueness_of treats the primary field differently from the scope fields, and am hoping someone can shed some light.
I can't explain why the validations work one way around, but not the other.
But I think basically your problems are due to the fact that serialize only defines that an attribute is to be serialized using Yaml on save and deserialized upon load.
In other words: the only thing you say by doing serialize :filters, Array is that
when saving a Foo, serialize it's filters attribute using Yaml first,
when loading a Foo from the DB, make sure that the value of the
filters attribute is an Array after deserialization, otherwise raise an exception
It does not affect how queries are constructed. Instead, Rails' usual rules for queries are used. So an array is converted into a comma separated list of numbers. This makes sense for example when constructing a LIKE query. This is the reason why the query fails. The DB field is a string but you're trying to compare it to a list.
I haven't used native PostgreSQL array columns with Rails 4, but my guess is that these issues would solved if you used those instead a serialization-type solution. You get the added benefit of being able to search within the contents of arrays on the DB level.