I've been getting sick of seeing the "helpful" comments in the db/schema.rb after each migration. I've figured out how to disable it.
To stop this from happening create the following file in lib/tasks/schema_cleanup.rake (note the .rake):
namespace :db do
def cleanup_schema
filename = "db/schema.rb"
schema = File.read(filename)
.split("\n")
.select { |l| not l.strip.starts_with? "#" }
.slice(1..-1)
.push("")
.join("\n")
File.open(filename, 'w') do |file|
file.write(schema)
end
end
task :migrate do
cleanup_schema
end
end
Now each time you run the command the comments will be removed.
Related
I have a simple rails application where I import data from csv into my rails app which is functioning properly, but I have no idea where to start with testing this rake task, as well as where in a modular rails app. Any help would be appreciated. Thanks!
Hint
My Rails structure is a little different from traditional rails structures, as I have written a Modular Rails App. My structure is in the picture below:
engines/csv_importer/lib/tasks/web_import.rake
The rake task that imports from csv..
require 'open-uri'
require 'csv'
namespace :web_import do
desc 'Import users from csv'
task users: :environment do
url = 'http://blablabla.com/content/people.csv'
# I forced encoding so avoid UndefinedConversionError "\xC3" from ASCII-8BIT to UTF-8
csv_string = open(url).read.force_encoding('UTF-8')
counter = 0
duplicate_counter = 0
user = []
CSV.parse(csv_string, headers: true, header_converters: :symbol) do |row|
next unless row[:name].present? && row[:email_address].present?
user = CsvImporter::User.create row.to_h
if user.persisted?
counter += 1
else
duplicate_counter += 1
end
end
p "Email duplicate record: #{user.email_address} - #{user.errors.full_messages.join(',')}" if user.errors.any?
p "Imported #{counter} users, #{duplicate_counter} duplicate rows ain't added in total"
end
end
Mounted csv_importer in my parent structure
This makes the csv_importer engine available in the root of the application.
Rails.application.routes.draw do
mount CsvImporter::Engine => '/', as: 'csv_importer'
end
To correctly migrate in the root of the application, I added initializer
/engines/csv_importer/lib/csv_importer/engine.rb
module CsvImporter
class Engine < ::Rails::Engine
isolate_namespace CsvImporter
# This enables me to be able to correctly migrate the database from the parent application.
initializer :append_migrations do |app|
unless app.root.to_s.match(root.to_s)
config.paths['db/migrate'].expanded.each do |p|
app.config.paths['db/migrate'] << p
end
end
end
end
end
So with this explanation am able to run rails app like every other rails application. I explained this so anyone who will help will understand what to help me with as regards writing test for the rake task inside the engine.
What I have done as regards writing TEST
task import: [:environment] do
desc 'Import CSV file'
task test: :environment do
# CSV.import 'people.csv'
Rake::Task['app:test:db'].invoke
end
end
How do someone write test for a rake task in a modular app? Thanks!
I haven't worked with engines, but is there a way to just put the CSV importing logic into it's own class?
namespace :web_import do
desc 'Import users from csv'
task users: :environment do
WebImport.new(url: 'http://blablabla.com/content/people.csv').call
end
end
class WebImport # (or whatever name you want)
def initialize(url) ... end
def call
counter, CSV parse, etc...
end
end
That way you can bump into the Rails console to do the WebImport and you can also do a test isolating WebImport. When you do Rake tasks and Jobs (Sidekiq etc), you want to make the Rake task act as as thin a wrapper as possible around the actual meat of the code (which is in this case CSV parsing). Separate the "trigger the csv parse" code from the "actually parse the csv" code into their own classes or files.
I am trying to import data from CSV into the database using Classes so that I can easily write Test Case for the csv import rake task I created
However, my solution does not work.
And I also feel:
It doesn't make sense
Aside feeling its not a good solution that connotes Ruby mastery, it doesn't work.
Here is what I came up with in my engines/csv_importer/lib/tasks/csv_import.rake
require 'open-uri'
require 'csv'
namespace :csv_import do
desc 'Import users from csv'
task users: :environment do
WebImport.new(url: 'http://blablabla.com/details/people.csv').call.answers
end
end
class WebImport
def initialize(url)
#csv_string = url
end
def call
CSV.parse(#csv_string, headers: true, header_converters: :symbol) do |row|
next unless row[:name].present? && row[:email_address].present?
end
CsvImporter::User.create row.to_h
end
def self.answers
user = []
counter = 0
duplicate_counter = 0
user.persisted? ? counter + 1 : duplicate_counter + 1
p "Email duplicate record: #{user.email_address} - #{user.errors.full_messages.join(',')}" if user.errors.any?
p "Imported #{counter} users, #{duplicate_counter} duplicate rows ain't added in total"
end
end
Error when I run rake csv_import:users
$ rake csv_import:users
rake aborted!
NoMethodError: private method `gets' called for {:url=>"http://blablabla.com/details/people.csv"}:Hash
How do I make this work and write TEST for this at the long run?
You are getting this error because you are passing a hash to CSV.parse while that method accepts a string.
To fix that you need to change argument from a hash to a string: WebImport.new('http://blablabla.com/details/people.csv') and read a remote CSV file before passing it to CSV.parse, for example: CSV.parse(open(url)).
You can try to use
rake db:seed
to import the data to your database using seed file as
require 'csv'
puts "Importing data..."
CSV.foreach(Rails.root.join("file_name.csv"), headers: true) do |row|
Model_name.create! do |model_name|
model_name.name = row[0]
model_name.email_address = row[1]
end
end
csv file should be in your project root folder
I'm very new to the concept of importing data into a SQL database with CSV. I've followed some stackoverflow posts but I'm getting an error. The error states, Errno::ENOENT: No such file or directory # rb_sysopen - products.csv after running rake import:data. I have csv required in my application.rb as well as I have created a csv file and placed it in TMP. Here is my code so far. I understand I may be asking for a lot from the community but if someone were to answer this question, can you provide some more insight into CSV and rake functions. Thanks so much!!!
<b>import.rake</b>
namespace :import do
desc "imports data from a csv file"
task :data => :environment do
require 'csv'
CSV.foreach('tmp/products.csv') do |row|
name = row[0]
price = row[1].to_i
Product.create( name: name, price: price )
end
end
end
Specify the full path to the CSV file.
For example, if the file is in /tmp/ use:
CSV.foreach('/tmp/products.csv') do |row|
If the products.csv file is in your application's tmp directory use:
CSV.foreach(Rails.root.join('tmp', 'products.csv')) do |row|
I ran into something similar, it was forgetting to put both parenthesis with the braces so you might want to try going from:
Product.create( name: name, price: price )
to:
Product.create({ name: name, price: price })
Check out the smarter_csv Gem.
In it's simplest form you can do this:
SmarterCSV.process('tmp/products.csv').each do |hash|
Product.create( hash )
end
Add smarter_csv to your Gemfile, so it's auto-loaded when you require the environment in your Rake task
This gives you:
namespace :import do
desc 'imports data from given csv file'
task :data, [:filename] => :environment do |t, args|
fail "File not found" unless File.exists? args[:filename]
options = {} # add options if needed
SmarterCSV.process( args[:filename], options).each do |hash|
Product.create( hash )
end
end
end
Call it like this:
rake import:data['/tmp/products.csv']
See also: https://github.com/tilo/smarter_csv
I've got a rake task that changes data on the homepage every few hours. I've tested it out and it works fine in development. But it doesn't work in production. What do I have to do to get the changes I want to see? Should I add a command that restarts the server? Would that make the server acknowledge the change? Is there a smarter way to do this?
The rake task is below. It'll be run by heroku's scheduler add on, so it's currently in the lib/tasks/scheduler.rake file.
desc 'changes the meta tags'
task :mixup_meta_tags => :environment do
regex = /#meta_tag/
file = File.open('app/controllers/site_controller.rb', 'r')
lines = []
file.each_line do |line|
(line =~ regex) ? (lines << replace_line(line)) : (lines << line)
end
file.close
file = File.open('app/controllers/site_controller.rb', 'w')
lines.each{|line| file.write line}
file.close
end
def replace_line(line)
meta_tags = MetaTag.all.map { |tag| tag["tag"] }
new_tag = meta_tags.sample(1)[0]
line = " #meta_tag = \"#{new_tag}\" \n" # added the newline
end
Yes, changes to your Rails application in Production require a restart for them to get picked up by the server. To get this to work on the fly you might want to try the solution mentioned in this post why-does-code-need-to-be-reloaded-in-rails-3
I'm working on a rake task that changes the meta tags for certain pages. These tags are defined in the controller. The rake task is meant to open the controller file, find any line with the phrase "#meta_tag" in it, and then re-write that line per the replace_line(line) method. When I run the rake task, I don't get any errors, but it also doesn't make any changes.
I think my problem is in the 'r+' line. Do I need to open the file in one line and write to it in another line of code?
require 'rake'
namespace :meta_tags do
desc 'changes the meta tags'
task :update => :environment do
regex = /#meta_tag/
found = false
file = File.open('app/controllers/site_controller.rb', 'r+')
file.each_line do |line|
replace_line(line) if(found)
found = true if(line =~ regex)
end
end
def replace_line(line)
meta_tags = MetaTag.all.map { |tag| tag["tag"] }
new_tag = meta_tags.sample(1)[0]
line = "#meta_tag = #{new_tag}"
end
end
If you see what I'm doing wrong, please let me know.
Well you don't actually write to the file anywhere. each_line does just what it says, it iterates over each line in the file (actually it reads until there is a newline and then yields this line to the block you provide).
But just using file.write now isn't a really good idea since inplace file-writing doesn't work like you would expect. Because files are byte/character based, a replacementline would have to be exactly as long as the old one.
So you should go with the normally used practice of reading then writing. Also your current code would alter the line after the
#meta_tag occurrence, although your question suggests that this is not what
you want.
Here is an example applied to your situation:
require 'rake'
namespace :meta_tags do
desc 'changes the meta tags'
task :update => :environment do
regex = /#meta_tag/
# only 'r' since you will only read the file,
# although you could use 'r+' and just change the lineno
# back to 0 when finished reading...
file = File.open('app/controllers/site_controller.rb', 'r')
lines = []
file.each_line do |line|
# i don't think you need the found variable,
# it is simple if-then/else
(line =~ regex) ? (lines << replace_line(line)) : (lines << line)
end
file.close
file = File.open('app/controllers/site_controller.rb', 'w')
# you could also join the array beforehand,
# and use one big write-operation,
# i don't know which approach would be faster...
lines.each{|line| file.write line}
file.close
end
def replace_line(line)
meta_tags = MetaTag.all.map { |tag| tag["tag"] }
new_tag = meta_tags.sample(1)[0]
line = "#meta_tag = #{new_tag}\n" # added the newline
end
end