Rails - Soulmate, export to json one item per line - ruby-on-rails

I'm trying to use Soulmate to make the search and autocompletion. However it requires a json file with all data under a model and one item per line. When I export with to_json i get all object separated by comma.
This is what I want to print:
{"id":1,"term":"Dodger Stadium","score":85,"data":{"url":"\/dodger-stadium-tickets\/","subtitle":"Los Angeles, CA"}}
{"id":28,"term":"Angel Stadium","score":85,"data":{"url":"\/angel-stadium-tickets\/","subtitle":"Anaheim, CA"}}
{"id":30,"term":"Chase Field ","score":85,"data":{"url":"\/chase-field-tickets\/","subtitle":"Phoenix, AZ"}}
{"id":29,"term":"Sun Life Stadium","score":84,"data":{"url":"\/sun-life-stadium-tickets\/","subtitle":"Miami, FL"}}
{"id":2,"term":"Turner Field","score":83,"data":{"url":"\/turner-field-tickets\/","subtitle":"Atlanta, GA"}}
And this is what I print with to_json
[{"id":1,"first_name":"Philip","last_name":"nalle","location":"nallemia","email":"hejsan#hej.com","active":false,"created_at":"2013-06-26T15:00:38.990Z","updated_at":"2013-06-26T15:00:38.990Z"},{"id":2,"first_name":"Philip","last_name":"nalle","location":"hejsan123","email":"hejsan#asd.com","active":false,"created_at":"2013-06-26T15:01:45.905Z","updated_at":"2013-06-26T15:01:45.905Z"},{"id":3,"first_name":"hejsan","last_name":"hejsan","location":"asd","email":"asd#asda.com","active":false,"created_at":"2013-06-26T15:08:20.354Z","updated_at":"2013-06-26T15:08:20.354Z"},{"id":4,"first_name":"well well","last_name":"hello","location":"asd123","email":"asd#asd.com","active":false,"created_at":"2013-06-26T15:10:27.121Z","updated_at":"2013-06-26T15:12:29.991Z"}]
Don't mind the actual data inside the json, I just copied a sample from soulmates readme. This is the rake task.
desc "Save all participants to json"
task :save_to_json => :environment do
File.open("participants.json", "w") { |f| f.write(Participant.all.to_json)}
end

To achieve what you want, you need to output elements one by one.
It will also save memory if instead of .all you grab them by 1000 if there are many of them.
desc "Save all participants to json"
task :save_to_json => :environment do
File.open("participants.json", "w") do |f|
Participant.all.each do |participant|
f.write("#{participant.to_json}\n")
end
end
end

Related

Check if a value exists in a database and return the value if it does Rails

I am running Rails here with a User model that contains an :email_address column
I have an array - emails_to_check[email1,email2,email3] that i want to check if they exist in the User database.
I only want to return the values that DO exist in the database
Here's a simple one-liner for you. There may be more performant ways, but this is maybe the most straight-forward and idiomatic Rails.
emails_to_check = ['email1', 'email2', 'email3']
User.where(email_address: emails_to_check).pluck(:email_address)
Here is the resulting SQL query:
SELECT `users`.`email_address` FROM `users` WHERE `users`.`email_address` IN ('email1', 'email2', 'email3');
so i solved this using a rake task
task :find_users_in_array,[:emails_to_find] => :environment do |task, args|
emails = args[:emails_to_find].split
emails.each do |email|
if User.find_by(email:email)
puts "#{email}"
end
end
end
I can pass in a list using rake:find_users_in_array[email1 email2 email3]

Converting a string to integer in CSV import on Rails

I've a rake task where I import CSV data into a database via Rails.
I want a specific column (specifically, row[6] below) to be rendered as an integer. However, everything I try returns that value as a string.
Below is the rake task:
require 'csv'
namespace :import_site_csv do
task :create_sites => :environment do
CSV.foreach('../sites.csv', :headers => true) do |row|
row[6] = row[6].to_i
Site.create!(row.to_hash)
end
end
end
Does anyone have an idea how I might do this? Thanks!
You are making one small (but important) mistake here.
When you call CSV.foreach('../sites.csv') each of the rows will be an array of the values in that particular row. That would allow you to access the data you need, in the way you do it now - row[6].
But, when you add the :headers => true option to CSV.foreach, you will not get an array of values (row will not be an array). Instead, it will be a CSV::Row object (docs). As you can read in the documentation:
A CSV::Row is part Array and part Hash. It retains an order for the fields and allows duplicates just as an Array would, but also allows you to access fields by name just as you could if they were in a Hash.
For example, if you have a column with the name Title in the CSV, to get the title in each of the rows, you need to do something like:
CSV.foreach('file.csv', :headers => true) do |row|
puts row['Title']
end
Since I do not know the structure of your CSV, I cannot tell you which key you should use to get the data and convert it to an Integer, but I think that this should give you a good idea of how to proceed.

Running a 1 time task to enter values into database in Ruby

I have now added a new column to my table in database. I want to add some values to some rows in this new column. I know the logic and all. But actually I dont know the way to add this, or write a 1 time task to do this in ruby on rails. Can any one help me. I just need some idea.
data = Model.where(#your_condition)
if the value is same for all
data.update_all(:new_column => "new value")
if the value is different for all
data.each do |d|
d.update_attributes(:new_column => "some value")
end
You can create a rake task for this, and run it once.
Create a file lib/tasks/my_namespace.rake
namespace :my_namespace do
desc "My Task description"
task :my_task => :environment do
# Code to make db change goes here
end
end
You can invoke the task from the command line in project root folder like
rake my_namespace:my_task RAILS_ENV=production

Passing a parameter or two to a Rake task

I have a rake task that I want to pass parameters to. For example, I want to issue a command like
<prompt> rake db:do_something 1
and inside the rake task:
...
cust = Customer.find( the_id_passed_in )
# do something with this customer record, etc...
...
Pretty straightforward, right?
The way rake commands accept and define arguments is, well, not pretty.
Call your task this way:
<prompt> rake db:do_something[1,2]
I've added a second parameter to show that you'll need the comma, but omit any spaces.
And define it like this:
task :do_something, :arg1, :arg2 do |t, args|
args.with_defaults(:arg1 => "default_arg1_value", :arg2 => "default_arg2_value")
# args[:arg1] and args[:arg2] contain the arg values, subject to the defaults
end
While passing parameters, it is better option is an input file, can this be a excel a json or whatever you need and from there read the data structure and variables you need from that including the variable name as is the need.
To read a file can have the following structure.
namespace :name_sapace_task do
desc "Description task...."
task :name_task => :environment do
data = ActiveSupport::JSON.decode(File.read(Rails.root+"public/file.json")) if defined?(data)
# and work whit yoour data, example is data["user_id"]
end
end
Example json
{
"name_task": "I'm a task",
"user_id": 389,
"users_assigned": [389,672,524],
"task_id": 3
}
Execution
rake :name_task

postgresql nextval generating existing values

I had to migrate from a mySql based ruby on rails app to using postgresql. No problems but one so far, and I don't know how to solve it.
The migration of data brought ids along with it, and postgresql is now having problems with existing ids: it's not clear to me where it gets the value that it uses to determine the base for nextval: it certainly isn't the highest value in the column, although you might think that would be a good idea. In any case, it's now colliding with existing id values. id column, created from a standard RoR migration is defined as
not null default nextval('geopoints_id_seq'::regclass)
Is there some place that the value it uses as a base can be hacked? This problem could now arise in any of 20 or so tables: I could use
'select max(id) from <table_name>'
but that seems to make the idea of an autoincrement column pointless.
How is this best handled?
There is a reset_pk_sequences! method on the Postgres adapter. You can call it and it will set it to max(id) + 1, which is probably what you want.
In some projects I get data ETL'ed in often enough to warrant a rake task to do this for all models, or for a specified model. Here's the task - include it in some Rakefile or in it's own under lib/tasks:
desc "Reset all sequences. Run after data imports"
task :reset_sequences, :model_class, :needs => :environment do |t, args|
if args[:model_class]
classes = Array(eval args[:model_class])
else
puts "using all defined active_record models"
classes = []
Dir.glob(RAILS_ROOT + '/app/models/**/*.rb').each { |file| require file }
Object.subclasses_of(ActiveRecord::Base).select { |c|
c.base_class == c}.sort_by(&:name).each do |klass|
classes << klass
end
end
classes.each do |klass|
next if klass == CGI::Session::ActiveRecordStore::Session && ActionController::Base.session_store.to_s !~ /ActiveRecordStore/
puts "reseting sequence on #{klass.table_name}"
ActiveRecord::Base.connection.reset_pk_sequence!(klass.table_name)
end
end
Now you can run this either for all models (defined under RAIS_ROOT/app/models) using rake reset_sequences, or for a specific model by passing in a class name.
The rails 3 version looks like this:
namespace :db do
desc "Reset all sequences. Run after data imports"
task :reset_sequences, :model_class, :needs => :environment do |t, args|
if args[:model_class]
classes = Array(eval args[:model_class])
else
puts "using all defined active_record models"
classes = []
Dir.glob(RAILS_ROOT + '/app/models/**/*.rb').each { |file| require file }
ActiveRecord::Base.subclasses.select { |c|c.base_class == c}.sort_by(&:name).each do |klass|
classes << klass
end
end
classes.each do |klass|
puts "reseting sequence on #{klass.table_name}"
ActiveRecord::Base.connection.reset_pk_sequence!(klass.table_name)
end
end
end
https://gist.github.com/909032
with that definition, the column will get the next value from the geopoints_id_seq sequence.
That sequence is not directly attached to the table. If you're migrating data, you have to create or update that sequence so its starting point is larger than the current max id in your table.
You should be able to set its new value with e.g.
ALTER SEQUENCE geopoints_id_seq RESTART with 1692;
Or whatever select max(id) from table_name; yields
PG uses sequences :
Make it's current value 1 higher than the highest value in your table like this.
SELECT setval('geopoints_id_seq', 999999999, true);
Also see these
http://www.postgresql.org/docs/8.4/interactive/datatype-numeric.html#DATATYPE-SERIAL
http://www.postgresql.org/docs/8.4/interactive/functions-sequence.html
Use setval() to set the starting value for the sequence.

Resources