I'm building a marketplace app in Rails 4 where sellers can list items to sell. I have a csv import feature so sellers can bulk load products. The import code worked fine on small files but I ran into timeout issues with larger files. So I want to use delayed_job to process these files in the background.
I set up delayed_job up to the point where the job is queued (i see the job in the delayed_job table). But when I run the job, I get an error saying that the file to be imported is not found. It is looking for the file in a temp folder which doesn't exist when the job is run.
How do I save (or without saving) the file in a location where delayed_job can access it? And how to I tell delayed_job where the file is located?
my listings controller:
def import
Listing.import(params[:file], params[:user_id])
redirect_to seller_url, notice: "Products are being imported."
end
my listing model:
class Listing < ActiveRecord::Base
require 'csv'
require 'open-uri'
class << self
def importcsv(file_path)
CSV.foreach(file_path, headers: true, skip_blanks: true) do |row|
#some model processing
end
end
handle_asynchronously :importcsv
end
# My importer as a class method
def self.import(file, user_id)
Listing.importcsv file.path
end
end
Here is form view:
<%= form_tag import_listings_path, multipart: true do %>
<%= file_field_tag :file %>
<%= hidden_field_tag :user_id, current_user.id %>
<%= submit_tag "Import CSV" %>
<% end %>
Presumably the file is a form upload. I think those files only persist while the web request is running. My recommendation is to use FileUtils.copy to copy the file into some location that will exist when your job runs.
So, probably you don't want to handle_asynchronously importcsv, but instead copy the files then call a private method on your model (which will be handled asynchronously) with the new file paths.
Related
I am trying to figure out how to import a CSV file to create new records and make the appropriate associations.
I have 3 models:
1) Farmer (has_many :farms)
2) Farm (has_many :crops, belongs_to :farmer)
3) Crop (belongs_to :farm)
The goal is to have the user create each farm manually, and then import the crops though a CSV file. I cannot figure out how to pass the farm_id to the crops getting uploaded, creating the association.
The import action on the Crops controller:
def import
Crop.import(params[:file], params[:farm_id])
redirect_to root_url, notice: "Crops imported."
end
The import method in the Crop model:
def self.import(file, farm)
CSV.foreach(file.path, headers: true, encoding:'iso-8859-1:utf-8') do |row|
row["farm_id"] = farm
Crop.create! row.to_hash
end
end
The form for the upload on the farms#show page:
##farm = Farm.find(params[:id])
<%= form_tag import_crops_path, multipart: true do %>
<%= file_field_tag :file %>
<%= submit_tag "import CSV" %>
How do you send over the #farm.id on the farms#show page to import#crops along with the file? Is it best to do this through the form - or some other way?
You could do this a couple ways, it depends on how your routes are set up.
If you have something like
resources :farms do
resources :crops
end
import_crops_path(#farm, #crop) will generate a url along the lines of /farms/:farm_id/crops/new
with both options you can get the farm id with params[:farm_id]
Another option is to use a hidden field.
f.hidden_field(:farm, :id)
this will create a html field
<input type="hidden" id="farm_id" name"farm[id]" value="#{#farm.id}" />
With both options you can get the farm id with params[:farm_id]
References
https://apidock.com/rails/ActionView/Helpers/FormHelper/hidden_field
http://guides.rubyonrails.org/routing.html
Hello i have a question about active record and importing XML.
I was asked if i could import a xml file into are website to add orders.
So say a client has a bulk order.
His system outputs already a XML file and i need to import it. I found a gem called importer.
It seems very good, as it does both XML and CSV imports ideal for what i need. However in the DOC's it says for imports
Product.import(path_to_file)
But i would like to make it into a button were they may select a file to import. I think i would use something like paperclip to do this. However if you have any method to do it. Even if it does not include importer. Maybe Nokugiri. Then please help.
Thank you for your time, I will be monitoring this post closely.
Not really. Unless you want to save the file, all you need is a tool that reads it and converts the results to something that can be inserted into the database. Since you already found one, you just need to import it:
Add to your view a form for file insertion:
<%= form_tag import_products_path, multipart: true do %>
<%= file_field_tag :file %>
<%= submit_tag "Import" %>
<% end %>
Add a custom route action:
resources :product do
collection { post :import }
end
Put the import method in the controller:
def import
Product.import(params[:file])
redirect_to root_url, notice: "Products imported."
end
You do the processing in the model. For example:
require 'nokogiri'
def self.import(file)
doc = Nokogiri::XML.parse(file)
#your processing code goes here
end
I am using rails 4 and I have this code.
I created a controller and called it upload (upload_controller.rb)
I put this code in it:
class UploadController < ApplicationController
def index
render :file => 'app\views\upload\uploadfile.rhtml.erb'
end
def uploadFile
post = DataFile.save(params[:upload])
render :text => "File has been uploaded successfully"
end
end
My model was called data_file.rb. The code is the following:
class DataFile < ActiveRecord::Base
attr_accessor :upload
def self.save(upload)
name = upload['datafile'].original_filename
directory = "public/data"
# create the file path
path = File.join(directory, name)
# write the file
File.open(path, "wb") { |f| f.write(upload['datafile'].read) }
end
end
My view file was called uploadFile.html.erb. My code is the following:
<h1>File Upload</h1>
<%= form_tag({:action => 'uploadFile'}, :multipart => true) do %>
<p><label for="upload_file">Select File</label>
<%= file_field 'upload', 'datafile' %></p>
<%= submit_tag "Upload" %>
<% end %>
My main goal of having this code is so that the user can upload a file to a specified location. The file type has to be anything not just images but doc, excel sheets, etc. Once I write this code and I do bundle install and then I do rake db:migrate and I go to tools and run the development. Once I do that I go to firefox and type in localhost:3000 and the page cannot show. (directs me to yahoo search or whatever).
I don't know what I am doing wrong here. Am I suppose to add a gem or change a certain line or something? I have been stuck on this issue for days now and I just want to move on to the next part of my webpage. Please help me. Thank you.
I'm trying to build an app in Rails which will take audio file uploads and read metadata off them to populate a database. I'm using the Taglib-ruby gem to handle various file types. The uploads seem to be working on their own, but Taglib considers any file given to it as null.
Here's my controller:
class UploadsController < ApplicationController
require 'taglib'
def new
end
def create
file = params[:upload]
TagLib::FileRef.open(file) do |fileref|
unless fileref.null?
tag = fileref.tag
# properties = fileref.audio_properties
#song = Song.new(title: tag.title, artist: tag.artist, album: tag.album,
year: tag.year, track: tag.track, genre: tag.genre)
if #song.save
redirect_to songs_path
else
render 'new'
end
else
raise "file was null"
end
end
end
end
and my view for form submission:
<h1> Upload </h1>
<%= form_tag(url: { action: :create }, html: { multipart: true }) do %>
<%= label_tag :upload, "Scan your song:" %>
<%= file_field_tag :upload, multiple: true %>
<br />
<%= submit_tag "Submit" %>
<% end %>
Taglib itself seems to be working - adding "require 'taglib'" removed the error I had been getting in regards to that, and a mock-up I made of this outside of rails worked fine (so the files I'm using are also not the problem). Every time I run this, the control flow hits my raise command, and no record is saved. It's clear that fileref.null? is returning true, which suggests to me that there's something wrong with the upload process... but I'm not sure what.
Ideally, I'd like to use the multiple uploads option and run this process on each file sequentially, but I can't even get a single upload to register as anything but null.
I have the following in my view:
<%= form_tag import_list_path, multipart: true do %>
<%= file_field_tag(:file) %>
<%= submit_tag(:Submit) %>
<% end %>
I have this in my controller:
def import
require 'csv'
csv = CSV.load params[:file].tempfile.read
CSV.new(csv.tempfile, :col_sep => ",", :return_headers => false).each do |column|
name_array << column[5]
end
redirect_to(:index)
end
I'm just trying to store a temporary CSV file in memory and do some actions on it, essentially using it to pull in information to be used in consuming a web service later.
This is the error I receive:
cannot load such file -- Column1,Column2,Column3,Column4,Column5,Column6,Column7,etc....
How can I change my controller to not throw this error?
That should do it.
def import
require 'csv'
CSV.new(params[:file].tempfile, :col_sep => ",", :return_headers => false).each do |column|
name_array << column[5]
end
redirect_to(:index)
end
Another notice: Dont put your logic in your controller it belongs in the model ;)
//
That menas you should write a method in your model that deals with the data and only refere the path to the csv file as a parameter of the method. The model is there as an interface between you app and the database and for the things done in the app. The View is there to display your stuff and the controller is the thing that connects both.