I'm writing a migration to change a column type from string to integer.
def change
change_column :emails, :object_id, :integer
end
This migration is failed because this column already contains string values. Before executing this migration I'm trying to remove all letters from this column so that i can get only integer values. Existing values are like
"AB12345"
"A12345X"
"789X26A"
What script i should execute before migration to remove all letters and achieve only integer values like this?
"12345"
"12345"
"78926"
Thanks
If you have more than say 10,000 records do the conversion in the database itself. For postgres, that'd be something like:
select regexp_replace('1231ASDF12', '[^0-9]', '', 'g')
You can use execute to run raw sql in a migration:
update table set col = regexp_replace(col, '[^0-9]', '', 'g')
Keep in mind if you're intending object_id to be a foreign key, you'll need to update whatever table is referenced and also ensure you haven't inadvertently broken anything (e.g., if there was AB123 and BC123 in the dataset).
I think you could use the trim function but the folloing line would do just fine as well.
result = Replace("Some sentence containing Avenue in it.", "Avenue", "Ave")
Example from Access VBA | How to replace parts of a string with another string
You could change "A" into ""
"B" into "" ect.
You would end up whit a code like this
Do While ActiveCell.Value <>""
ActiveCell.Value = Replace(ActiveCell.Value, "A", "")
ActiveCell.Value = Replace(ActiveCell.Value, "B", "")
ActiveCell.Value = Replace(ActiveCell.Value, "C", "")
ect...
ActiveCell.Offset (-1,0).Select
Loop
Related
I am attempting to order by a column of type "character varying []" and cast it to an integer[] while sorting (instead of using the default ASCII comparison sort). I am using a Postgresql database.
I've found that the following query works as expected:
select <col> from <table> order by <col>::integer[] desc
Unfortunately, when I attempt to programmatically do this in rails it is adding quotes around the column and casting suffix. This results in it thinking "::integer[]" is part of the column name - and of course there is no such column. Thus the query fails.
Here is the rails code:
scope.order([ '<col>::integer[]', 'desc', 'NULLS LAST' ].join(' '))
And this is the query it produces:
select <col> from <table> order by "<table>"."<col>::integer[]" desc
How can I implement this properly with rails?
Thanks to #engineersmnky comment, I found the solution that I need in my code.
In the code I'm actually processing an array of columns (and directions) to sort by. It turns out the solution was indeed to use the Arel.sql() function to process the order by parameters prior to calling scope.order(), with the end result looking something like this:
def sort(scope, sorts)
str = \
sorts.map |sort| do
col = get_sort_column_alias(sort[0])
dir = sort[1]
nullpos = (dir == 'asc') ? 'FIRST' : 'LAST'
"#{col} #{dir} NULL #{nullpos}"
end
scope.order(Arel.sql(str))
end
def get_sort_column_alias(col)
case col
when 'target' then 'target::integer[]'
...
else col
end
end
I'm trying to find every items that has been set to a specifique status last year.
I'm using Rail 5 with audited so I created a specific Audit Model and I try to write a scope to return my condition :
Audit = Audited.audit_class
class Audit
scope :followupRebus, -> { where(auditable_type: 'Followup')
.where(action: 'update')
.where("audited_changes LIKE '%step_id:[#{Step::REBUS}%'")
}
end
the content of the audited text field in postgres looks like this when I take it and show it with a .to_s
{"step_id"=>[9, 4], "active"=>[false, true]}
How can I get all audit with step_id = 9 ?
EDIT
Great thanks to DRSE, I finally found a working solution :
changing the default TEXT type of the Column with the migration sent by DRSE
Change the request like this :
class Audit
scope :followupRebus, -> { where(auditable_type: 'Followup')
.where(action: 'update')
.where("((audited_changes -> 'step_id')::json->>0)::int = :step_id", step_id: Step::REBUS)
}
end
You need to use Postgres JSON functions to query the JSON column audited_changes instead of the LIKE operator.
To find audits where the step_id was changed, you can use
.where("(audited_changes -> 'step_id')::jsonb ? :step_id", step_id: '9')
Note the use of the named bind variable :step_id, instead of using active record question mark (?) replacement because Postgres uses the question mark as a JSON query operator.
The clause above will find any audits where step_id = 9, whether that was the value set in the previous version or the updated version of your model.
To find audits where step_id = 9 in the the previous version:
# Check if the first value (indexed by 0) in 'step_id' array is '9'
.where("(audited_changes -> 'step_id')::jsonb->0 ? :step_id", step_id: '9')
To find audits with step_id = 9 set in the updated version:
# Check if the second value (indexed by 1) in 'step_id' array is 9
.where("(audited_changes -> 'step_id')::jsonb->1 ? :step_id", step_id: '9')
(Note: you should not directly string interpolate your conditions for your where clause because you are opening your app to a SQL injection vulnerability. Use rails style conditions with sanitized inputs instead.)
EDIT
Since you have indicated that your audited_changes column type is TEXT and not a JSON type, you will need to either run a migration to change the column type or else cast the column in the query.
To cast the column to JSON at query execution time, use audited_changes::json, so the example would be like this:
.where("(audited_changes::json -> 'step_id')::json ? :step_id", step_id: '9')
To change the column to JSON, start with rails g migration ChangeAuditedChangesColumnToJSONB. Then in your migration file (db/migrate/{timestamp}_change_audited_changes_column_to_jsonb.rb) write:
class ChangeAuditedChangesColumnToJSONB < ActiveRecord::Migration[5.1]
def up
change_column :audits, :audited_changes, :jsonb, using: 'audited_changes::JSONB'
end
def down
change_column :audits, :audited_changes, :text
end
end
Then run rails db:migrate and you should be good to go.
For a new project using Audited, you can add a param to the install step to specify the use of JSON or JSONB type for the audited_changes column.
rails generate audited:install --audited-changes-column-type jsonb # or json
I have a psql table with a column named params that contains a hash. Below is my scenario:
When I do the following query...
game = Game.select('games.params').where(id: 123).first
... it prints out...
p game.params.class # => !ruby/class 'ActiveSupport::HashWithIndifferentAccess
Now I can go ahead an access values inside the returned hash normally.
But when I give the column name an alias in my query (here is my problem)...
game = Game.select('games.params as parameters').where(id: 123).first
... it prints out...
p game.parameters.class # => !ruby/class 'String'
I need to be able to change the column name in my query and then access the values inside the hash, but when I try to do p game[:time] it's being treated as a string.
I'm new to Ruby and Ruby on Rails, so this might be very simple to figure out but I am at a dead end right now.
What about this query:
g = Game.find 123
g.params
My use case is as follows:
I am reading a csv file and I want to get the column headers there as a hash.
Then Im saving the row-values of the these column names into a database.
What I want to do is:
incoming CSV file hash-keys = [Col1, Col2, Col3]
I want to map the values of these to the following database columns: [colA, colB, colC]
I don't want my database columns to be named Col1, Col2, Col3 but I want these as model attributes.
How do I make a model where some attributes are not database columns?
Thanks
Write the database tables as Row ->* Field (row.has_many fields).
Then in your Row class override method_missing, and inside that look up the target field via the missing name. So if Field.name == 'foo', then row.foo will fetch this record and return its value. When you call row.foo = 'bar', that will pass :'foo=' to method_missing; you can detect the = and assign a value to the target field. Googling for 'ActiveRecord method_missing has_many' might hit some code that already does this.
Here's what you do with each column name.
col = 'Col20'
To convert it to lowercase + letters format that you want, first get the number:
number = col.match(/(\d+)/).try(:[],1).to_i
=> "20"
The above will attempt to find a number and then if there's a match, it will try to get that number.
Then convert (1=A, 20=T, ...):
new_col = col.downcase.sub( /\d+/, (number+64).chr )
=> "colT"
Then define such a property on the object you want it on like this:
my_object.class_eval { attr_accessor new_col }
And assign a value like this:
my_object.colT = "New Value"
Or dynamically:
my_object.send "#{new_col}=", "New Value"
And get its existing value by calling it by name:
my_object.colT
=> "New Value"
Or dynamically:
my_object.send new_col
=> "New Value"
I was debugging a Ruby on Rails engine which has problems when running on Sqlite, it has a problem in finding records that the app itself creates. When run on MySQL everything works but the same query on SQLite is failing.
I've tracked down the issue and I found that the problem is in a simple WHERE query which won't find the created record. Essentially the table structure has a column called key which stores some md5 hashes. The failing spec insert a record with a given hash then on the following instruction do a SELECT query for the same hash, but SQLite returns no record for the same key. I've extracted the generated database and the failing query from the app and this is a copy of the app database:
http://dl.dropbox.com/u/2289657/combustion_test.sqlite
Here is a transcript of the queries executed by the software (made with the command line utility):
# Here I'm selecting all the records from the table
# there is a single record in it, the key is the third field
$ sqlite3 combustion_test.sqlite 'SELECT * FROM tr8n_translation_keys'
1||b56c67d10759f8012aff28fc03f26cbf|Hello World|We must start with this sentence!||||en-US|0|2012-03-14 11:49:50.335322|2012-03-14 11:49:50.335322|
# Here I'm selecting the record with that key and it doesn't return anything
$ sqlite3 combustion_test.sqlite "SELECT * FROM tr8n_translation_keys WHERE key = 'b56c67d10759f8012aff28fc03f26cbf'"
# Here I'selecting the record with a LIKE clause and it finds the record
$ sqlite3 combustion_test.sqlite "SELECT * FROM tr8n_translation_keys WHERE key LIKE 'b56c67d10759f8012aff28fc03f26cbf'"
1||b56c67d10759f8012aff28fc03f26cbf|Hello World|We must start with this sentence!||||en-US|0|2012-03-14 11:49:50.335322|2012-03-14 11:49:50.335322|
Should I report this as a bug to SQLite site?
P.S. I've tried also on a different system with a different SQLite version, but the results are the same.
Update
Here is the table schema
sqlite> .schema tr8n_translation_keys
CREATE TABLE "tr8n_translation_keys" (
"id" INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
"type" varchar(255), "key" varchar(255) NOT NULL,
"label" text NOT NULL,
"description" text,
"verified_at" datetime,
"translation_count" integer,
"admin" boolean,
"locale" varchar(255),
"level" integer DEFAULT 0,
"created_at" datetime,
"updated_at" datetime,
"synced_at" datetime
);
CREATE UNIQUE INDEX "index_tr8n_translation_keys_on_key" ON "tr8n_translation_keys" ("key");
CREATE INDEX "index_tr8n_translation_keys_on_synced_at" ON "tr8n_translation_keys" ("synced_at");
Update 2
Here is the rails code which compute the key value inserted into the table (I've removed some code, full method is here)
def self.find_or_create(label, desc = "", options = {})
key = generate_key(label, desc).to_s
# IF I UNCOMMENT THIS LINE EVERYTHING WORKS
#key = 'b56c67d10759f8012aff28fc03f26cbf'
tkey = Tr8n::Cache.fetch("translation_key_#{key}") do
existing_key = where(:key => key).first ### THIS IS THE FAILING WHERE
existing_key ||= begin
new_tkey = create(:key => key.to_s,
:label => label,
:description => desc,
:locale => locale,
:level => level,
:admin => Tr8n::Config.block_options[:admin])
# rest of method...
And here is the generate_key method, the comment about sqlite is from author, not mine)
def self.generate_key(label, desc = "")
# TODO: there is something iffy going on with the strings from the hash
# without the extra ~ = the strings are not seen in the sqlite database - wtf?
"#{Digest::MD5.hexdigest("#{label};;;#{desc}")}"
end
This works:
SELECT * FROM tr8n_translation_keys WHERE LOWER(key)='b56c67d10759f8012aff28fc03f26cbf';
But this doesn't:
SELECT * FROM tr8n_translation_keys WHERE key='b56c67d10759f8012aff28fc03f26cbf' COLLATE NOCASE;
When I examine the database in SQLiteManager, it shows the key as this:
X'6235366336376431303735396638303132616666323866633033663236636266'
which implies it's treating the key as a BLOB (raw binary data) rather than TEXT. This is why the comparison fails. But LOWER(key) causes the field to be cast to text, hence the comparison succeeds.
So, we need to find out why the entry has been stored as a BLOB instead of TEXT. How were these values inserted into the database?
Following your update 2: I'm not a Ruby expert, but the value returned from generate_key is not being converted to a string in the way you expect. Try to_str instead of to_s when calling generate_key.
Based on the following Stack Overflow answer...
https://stackoverflow.com/a/6591427/18064
... you might want to update the generation of your key as follows:
def self.generate_key(label, desc = "")
# TODO: there is something iffy going on with the strings from the hash
# without the extra ~ = the strings are not seen in the sqlite database - wtf?
"#{Digest::MD5.hexdigest("#{label};;;#{desc}").encode('UTF-8')}"
end
Note the addition of .encode('UTF-8').
This worked for me when I had the same problem as yourself.