I'm trying to import CSV data into an already existing Rails table.
My CSV has columns of lec_exam, location, meeting_days, begin_time, and end_time. My table looks as follows:
create_table "courses", force: true do |t|
t.string "lec_exam"
t.string "location"
t.string "meeting_days"
t.time "begin_time"
t.time "end_time"
t.datetime "created_at"
t.datetime "updated_at"
t.string "status"
end
The status (i.e. taken vs open) field is something I want to update based on the current time versus the presence of a course occurring at that time.
Every time I import the CSV data the last column (end_time) does not get properly imported because each course has an end_time of nil, when a simple glance at the CSV shows otherwise.
I have tried
CSV.foreach(file.path, headers: true) do |row|
#course_hash = row.to_hash # exclude the price field
#course = Course.where(id: course_hash["id"])
row = Course.create!({
:lec_exam => row[0],
:location => row[1],
:meeting_days => row[2],
:begin_time => row[3],
:end_time => row[4]
})
as well as the to_hash method. Any help towards a solution would be great. Thanks!
To import a CSV data into an existing Rails table add:
require 'csv'
csv_text = File.read('...')
csv = CSV.parse(csv_text, :headers => true)
csv.each do |row|
Moulding.create!(row.to_hash)
end
In the rake task, or in a controller action.
Source: Ruby on Rails - Import Data from a CSV file
Hope this helps!
Solved.
Model had an unneccessary
attr_accessor:
tag. Bleh.
Related
I am using the Globalize 3 gem as seen in Ryan Bates railscasts, and everything works fine. I need to know how to seed the data though. Currently, I created a table called monthly_post_translations with the following schema
schema.rb
create_table "monthly_post_translations", :force => true do |t|
t.integer "monthly_post_id"
t.string "locale"
t.text "body"
t.datetime "created_at"
t.datetime "updated_at"
end
I need to add seed data to this table, but it doesn't have a model to interact with, so how do I do it?
Here is my currents seeds.rb that isn't working
seeds.rb
# Monthly Posts
MonthlyPost.delete_all
monthlypost = MonthlyPost.create(:body => "Monthly Post Text")
#Monthly Posts Spanish Translation
monthlytranslation = MonthlyPostTranslation.create(:body => "Spanish Translation of monthly post text",
:monthly_post_id => monthlypost.id,
:locale => "es" )
But the monthly_post_translation table doesn't have a model that I can interact with, so I get the error
uninitialized constant MonthlyPostTranslation
Any thoughts on how I can add this seed data properly?
As from documentation by typing translates :<attribute_name_here> you get generated model named MonthlyPost::Translation. So the answer will be: use instance collection to create or list all translations for entity:
monthlypost = MonthlyPost.create(:body => "Monthly Post Text")
#Monthly Posts Spanish Translation
monthlytranslation = monthlypost.translations.create(:body => "Spanish Translation of monthly post text",
:locale => "es" )
I have just changed a column (called time) from t.string to t.datetime and then dropped and re-created the database and run the migration.
I have a script set to run every minute that scrapes information from a remote website and then adds the information to new records based on the time column that I adjusted to be a datetime rather than string.
# Add each row to a new call record
page = agent.page.search("table tbody tr").each do |row|
next if (!row.at('td'))
time, source, destination, duration = row.search('td').map{ |td| td.text.strip }
call = Call.find_or_create_by_time(time)
call.update_attributes({:time => time, :source => source, :destination => destination, :duration => duration})
end
Since changing the time column to integer the script doesn't seem to be importing any information at all. I wondered if there is an extra step that I need to do to make this work again?
My schema looks like this:
create_table "calls", :force => true do |t|
t.string "source"
t.string "duration"
t.datetime "time"
t.string "destination"
t.string "recording"
t.string "cost"
t.datetime "created_at"
t.datetime "updated_at"
end
In this part
time, source, destination, duration = row.search('td').map{ |td| td.text.strip }
call = Call.find_or_create_by_time(time)
you get time variable as a string, and trying to find_by it. I think smth like
call = Call.find_or_create_by_time(Time.parse(time))
should do the trick
I have imported a .csv file of 10,000 locations and I need to loop through the database and geocode_by a few fields rather than the usual "geocode_by :address"
I am using the geocoder gem.
My database scheme looks like this
create_table "locations", :force => true do |t|
t.string "Address"
t.string "City"
t.string "State"
t.string "Zip"
t.float "latitude"
t.float "longitude"
t.datetime "created_at"
t.datetime "updated_at"
end
Can I do this in a controller action rather than on validation?
Should I do something like this:
def index
#locations = Location.all
#locations.each do |l|
new_address = "#{l.Address} #{l.City} #{l.State}"
geocode_by = :new_address
end
end
But yea, any help is greatly appreciated, thank you!
I was able to get things working with this controller code:
def index
if params[:search].present?
#locations = Location.near(params[:search], 20, :order => :distance)
else
#locations = Location.all
#locations.each do |l|
if l.latitude.nil?
new_location = "#{l.HP_Address_1} #{l.HP_City} #{l.HP_State}"
s = Geocoder.search(new_location)
l.latitude = s[0].latitude
l.longitude = s[0].longitude
l.save
end
end
end
end
It loops through every entry in the database, and if the latitude and longitude values have not been encoded it calls the correct geocoder function, and stores the returned lat / long values in the database.
I have 9k entries in my database, so I can only encode 2k per day due to the limit on the google maps api. But for folks used to encoding entries during creating using geocoded_by in their models, this will work in controllers.
This is the kind of thing I'd do with workers (delayed job or resque), since it might take some time to complete. But I'm not sure I understood the question, so this might not be the kind of answer you're expecting.
There is a rake task for this if you want to do every record in a class.
rake geocode:all CLASS=YourModel sleep=0.25
I have created Rails database using the following schema:
ActiveRecord::Schema.define(:version => 20090807141407) do
create_table "trunks", :force => true do |t|
t.integer "npa"
t.integer "nxxFrom"
t.integer "nxxTo"
t.string "trnk"
t.datetime "created_at"
t.datetime "updated_at"
end
end
In my CSV file, I only have the first four columns (npa, nxxFrom, nxxTo, and trnk).
How can I import the CSV data while also updating the last two columns?
Thanks always
To use the csv module which is part of the standard Ruby library:
require 'csv'
# row will be an array with the fields in the order they appear in the file
CSV.open('myfile.csv', 'r') do |row|
# assuming the fields in the CSV file are in order npa, nxxFrom, nxxTo, trnk
# create and save a Trunk model for each row
Trunk.create!(:npa => row[0], :nxxFrom => row[1], :nxxTo => row[2], :trnk => row[3])
end
I haven't used fastercsv but looking at its documentation the approach seems the same.
Use the FasterCSV gem.
A nice example is at http://espndev.com/blog/csv-imports-using-fastercsv/
The last two columns will be updated by ActiveRecord. If they exist in the schema then ActiveRecord will add a value for the created_at and updated_at upon creation of the object. Any modifications to the object will cause the updated_at column to be auto updated.
So when importing from a CSV just map the first 4 columns and save the object. When you retrieve the object you will see the other two values have also been set automatically.
I am trying to upload a CSV file into an existing database and receiving the following
error: ActiveRecord::RecordInvalid in UploadsController#import! Validation failed: Email has already been taken.
Controller:
class UploadsController < ApplicationController
def index
#uploads = Upload.all
end
def import
Upload.import(params[:file])
redirect_to uploads_path, notice: "Employee data imported!"
end
end
Model:
class Upload < ActiveRecord::Base
require 'csv'
def self.import(file)
CSV.foreach(file.path, headers: true) do |row|
Employee.create! row.to_hash
end
end
end
Table:
create_table "employees", force: :cascade do |t|
t.string "last_name"
t.string "first_name"
t.string "employee_code"
t.string "email"
t.string "level"
t.string "dept"
t.datetime "created_at", null: false
t.datetime "updated_at", null: false
end
I checked and and my database headers for employees do match my first row of the CSV file (beginning row 1, col 1). Any thoughts?
The keys of the hash you supply to Employee.create! (the CSV headers in this case) need to match exactly with the names of attributes on your model. Your problem is probably caused by one or more of the following:
There is a column where the header is not all lowercase. E.g., the CSV column name may be "Email" or "EMAIL" but should be "email".
There is a column where the header has spaces in it. E.g., the CSV column name may be "first name" or "last name" but should be "first_name" or "last_name", respectively.
There is a column in the CSV in which the column header is misspelled or otherwise doesn't belong to your table schema in the db at all. (Though you have said all of your CSV column headers match the database table columns, maybe check again if the problem is not solved by the first two bullets.)
Any one of these will cause the ActiveModel::UnknownAttributeError you are seeing. If the problem is being caused by one or multiple of the above then you can obviously solve the problem by editing your CSV file to make sure every header is lowercase with underscores instead of spaces. Alternatively, you could pass in the :header_converters => :symbol option to CSV.foreach like this:
CSV.foreach(file.path, headers: true, header_converters: :symbol) do |row|
Employee.create! row.to_hash
end
This option will transform your headers by converting them to lowercase, replacing spaces with underscores, and then converting them to symbols. This may be a more reliable option if you have a lot of column headers to tidy up, or if you will frequently receive the files with "slightly bad" headers suffering from one of the problems described above.
EDIT
I see you updated your question with more detail about the error. Rails is telling you that there is a column with header "last_name" in your CSV but there is no such column in your database (i.e. no such attribute on your Employee model). As Muhamad suggested, you could create a migration to add the last_name column to Employee. Can you verify that your employees table in the database has this column?