How to parse the contents of a uploaded file in RoR - ruby-on-rails

I am new to Rails.
In my project where users have to upload a file, I store it
then I have to parse the file contents and show it in new form.
I have successfully done the file uploading portion,
now how should I read the contents of it?

Try something like this:
upload = params[:your_upload_form_element]
content = upload.is_a?(StringIO) ? upload.read : File.read(upload.local_path)
Very small files can be passed as strings instead of uploaded files, therefore you should check for that and handle it accordingly.

You can open files and read their contents in Ruby using the File class, as this simple example demonstrates:
# Open a file in read-only mode and print each line to the console
file = File.open('afile.txt', 'r') do |f|
f.each do |line|
puts line
end
end

Complete Example
Take, for example, uploading an import file containing contacts. You don't need to store this import file, just process it and discard it.
Routes
routes.rb
resources :contacts do
collection do
get 'import/new', to: :new_import # import_new_contacts_path
post :import, on: :collection # import_contacts_path
end
end
Form
views/contacts/new_import.html.erb
<%= form_for #contacts, url: import_contacts_path, html: { multipart: true } do |f| %>
<%= f.file_field :import_file %>
<% end %>
Controller
controllers/contacts_controller.rb
def new_import
end
def import
begin
Contact.import( params[:contacts][:import_file] )
flash[:success] = "<strong>Contacts Imported!</strong>"
redirect_to contacts_path
rescue => exception
flash[:error] = "There was a problem importing that contacts file.<br>
<strong>#{exception.message}</strong><br>"
redirect_to import_new_contacts_path
end
end
Contact Model
models/contact.rb
def import import_file
File.foreach( import_file.path ).with_index do |line, index|
# Process each line.
# For any errors just raise an error with a message like this:
# raise "There is a duplicate in row #{index + 1}."
# And your controller will redirect the user and show a flash message.
end
end
Hope that helps others!
JP

Related

Upload File parameter not coming through to controller

I am fairly new to Rails and have been making steady progress on a Mobile Web App I am working on for our local high school but have run into an issue which I am stumped on. I am hoping the collective knowledge here will point me in the right direction.
I have a model for the school athletes (first name, last name, height, weight, graduation years, - standard stuff) which is working (CRUD via standard scaffold generation) and now I want to add the ability to import records via CSV upload.
In an effort to not reinvent the wheel, I am following this example from Rich on Rails. To get familiar with it, I created a separate Rail project and followed the example and it all works as expected. Great. Now to integrate into my existing project.
Everything seems to integrate fine with one exception - the CSV file is never passed to my model in the params I cannot figure out why. I am sure it is something obvious but I have stared at this problem for several hours and am unable to see what I am doing wrong.
Here is a portion of my Athletes controller:
class AthletesController < ApplicationController
before_action :set_athlete, only: [:show, :edit, :update, :destroy]
# GET /athletes
# GET /athletes.json
def index
#athletes = Athlete.all.order(:lastname, :firstname)
end
# POST /athletes/import
# POST /athletes/import.json
def import
logger.info(params.to_yaml)
begin
Athlete.import(params[:file])
redirect_to page_path('admin'), notice: "Athletes imported."
rescue
redirect_to page_path('admin'), notice: "Invalid CSV file format."
end
end
# GET /athletes/1
# GET /athletes/1.json
def show
end
# GET /athletes/new
def new
#athlete = Athlete.new
end
# GET /athletes/1/edit
def edit
end
My model looks like this:
class Athlete < ActiveRecord::Base
# an athlete can be on more than one team
has_and_belongs_to_many :teams, through: :athletes
require 'csv'
## CSV import
def self.import(file)
CSV.foreach(file.path, headers: true) do |row|
athlete_hash = row.to_hash # exclude the ? field
athlete = Athlete.where(id: athlete_hash["id"])
if athlete.count == 1
athlete.first.update_attributes
else
Athlete.create!(athlete_hash)
end # end if !athlete.nil?
end # end CSV.foreach
end # end self.import(file)
I've added this onto my index view for testing, later on it will be in an admin area:
<div>
<h3>Import a CSV File</h3>
<%= form_tag import_athletes_path, multipart: true do %>
<%= file_field_tag :file %>
<%= submit_tag "Import CSV" %>
<% end %>
</div>
No matter what I do, I never get the value of the file_field_tag to come through to the controller. If I add other fields using text_field_tag they come through as expected but the file_field_tag value never does.
--- !ruby/hash:ActionController::Parameters
utf8: "✓"
authenticity_token: it3yBxBnzA4UQ/NILP5GNoYJeO5dyg+Z+VfhE/C6p7k=
commit: Import CSV
action: import
controller: athletes
Redirected to http://localhost:3000/
Completed 302 Found in 8ms (ActiveRecord: 0.0ms)
I am stumped - if anyone has any ideas as to what I might be doing wrong, I would be grateful. I have about 300 athletes which I want to import and have no desire to type them in.
It turns out because I am using jQuery Mobile for my framework, I need to add "data-ajax=false" to my form tag. This change to my form allowed the file parameter to be visible in the controller:
<h3>Import a CSV File</h3>
<%= form_tag(import_athletes_path, { :multipart => true, :'data-ajax' => false }) do %>
<%= file_field_tag :file %>
<%= submit_tag "Import CSV" %>
<% end %>
</div>
A short while ago I recalled reading something about file uploads and jQuery Mobile not working by default. It is due to the standard AJAX navigation employed by jQM.

Ruby Rails: Upload File

I am trying to follow this tutorial. It has written in previous version of Rails and I am using Rails 4. I am trying to upload file but I am getting following error:
NoMethodError in UploadController#uploadfile
undefined method `[]' for nil:NilClass
Extracted source (around line #3):
class DataFile < ActiveRecord::Base
def self.save(upload)
name = upload['datafile'].original_filename
directory = "public/data"
# create the file path
path = File.join(directory, name)
Rails.root: C:/Ruby193/mylibrary
Application Trace | Framework Trace | Full Trace
app/models/data_file.rb:3:in `save'
app/controllers/upload_controller.rb:6:in `uploadfile'
Here is data_file.rb
class DataFile < ActiveRecord::Base
def self.save(upload)
name = upload['datafile'].original_filename
directory = "public/data"
# create the file path
path = File.join(directory, name)
# write the file
File.open(path, "wb") { |f| f.write(upload['datafile'].read) }
end
end
Here is controller upload_controller.rb
class UploadController < ApplicationController
def index
render :file => 'app\views\upload\uploadfile.html'
end
def uploadfile
post = DataFile.save(params[:upload])
render :text => "File has been uploaded successfully"
end
end
Here is uploadfile.html
<h1>File Upload</h1>
<%= form_tag({:action => 'uploadfile'}, :multipart => true) do %>
<p><label for="upload_file">Select File</label>
<%= file_field 'upload', 'datafile' %></p>
<%= submit_tag "Upload" %>
<% end %>
What should I do? Thanks in advance
It looks like params[:upload] isn't what you think it is. You forgot to set the form to be multipart. If fixing that doesn't make it work, start inspecting params to see what you're actually getting.
def uploadfile
puts params.inspect # Add this line to see what's going on
post = DataFile.save(params[:upload])
render :text => "File has been uploaded successfully"
end
Also, it's not a great "answer," but I've had good success using paperclip to handle file uploads. If you just want something that works (rather than learning how to do it yourself), check into that.

Use rubyzip to download paperclip attachments within nested model

I have the following model setup:
assignments belong to a user and assignments have many submissions
submissions belong to users and also belong to assignments
submissions have attached files (using paperclip).
I want the assignment user (creator) to be able to download the files (submissions) that belong to the particular assignment.
My routes are structured as follows:
resources :assignments do
resources :submissions
end
So, I think I need to define a download action in my assignments controller, which creates a zip archive of all the submissions belonging to an assignment and then redirects directly to that file url for a download.
def download
#submissions = #assignment.submissions.all
input_filenames = #submissions.file_file_name.all
Zip::File.open(assignment.file.url+"/archive.zip", Zip::File::CREATE) do |zipfile|
input_filenames.each do |filename|
zipfile.add(filename, assignment_submission_file_path(#assignment.id, #submission.id)+ '/' + filename)
end
end
respond_to do |format|
format.html { redirect_to assignment.file.url }
format.json { head :no_content }
end
end
Then in my assignment show page, I have the following:
<%= link_to 'Download', #assignment.file.url(:original, false) %>
But when clicked I get an error returning that the file is missing:
No route matches [GET] "/files/original/missing.png"
So the zip archive file is not being created, and thus my routing to the file doesn't work. It's possible I've done something wrong that is very basic, or that the whole thing needs to be structured differently.
Or my other thought was: do I need to create an empty zip archive in the create action of the assignment controller, so that there is an empty zip archive with a viable path to refer to when I want to add stuff into it? If so, how can I do that with the rubyzip gem?
Thanks!
Here's the answer to my own questions:
create an action in the controller called download and then refer to it properly in the show page:
def download
#assignment = Assignment.find(params[:assignment])
#submissions = #assignment.submissions.all
file = #assignment.file.url(:original, false)
Zip::ZipFile.open(file, create=nil) do |zipfile|
#submissions.each do |filename|
zipfile.add(filename.file_file_name, filename.file.url(:original, false))
end
end
And this is the call to that download action in the show page:
<%= link_to "Download", {:controller => "assignments", :action => "download", :assignment => #assignment.id }%>

Unable to pass uploaded file to model

I am relatively new to rails cannot get the following code to work. I am trying to upload a data file (Excel or csv), copy it to a temp folder and create a record in a Datafiles model which holds basic file information, such as filename, type, and date. Then I want to read the file and use the data to create or update records in several other models. If all goes well, move the file to a permanent location and write the new path in the Datafiles record.
Controller:
def new
#datafile = Datafile.new
respond_to do |format|
format.html # new.html.erb
format.xml { render :xml => #datafile }
end
end
def create
#datafile = Datafile.new(params[:upload])
#datafile.save!
redirect_to datafile_path(#datafile), :notice => "Successfully imported datafile"
rescue => e
logger.error( 'Upload failed. ' + e.to_s )
flash[:error] = 'Upload failed. Please try again.'
render :action => 'new'
end
View:
<%= form_for #datafile, :html => {:multipart => true} do |f| %>
<p>
<%= f.label(:upload, "Select File:") %>
<%= f.file_field :upload %>
</p>
<p> <%= f.submit "Import" %> </p>
<% end %>
Model:
require 'spreadsheet'
class Datafile < ActiveRecord::Base
attr_accessor :upload
attr_accessible :upload
before_create :upload_file
def upload_file
begin
File.open(Rails.root.join('uploads/temp', upload.original_filename), 'wb') do |file|
file.write(upload.read)
self.filename = upload.original_filename
Spreadsheet.client_encoding = 'UTF-8'
#book = Spreadsheet.open(file.path)
self.import
end
rescue => e
#upload_path = Rails.root.join('uploads/temp', upload.original_filename)
File.delete(#upload_path) if File::exists?(#upload_path)
raise e
end
end
def import
case #book.worksheet(0).row(0)[0]
when "WIP Report - Inception to Date"
self.report_type = 'WIP'
puts 'report_type assigned'
self.import_wip
else
self.report_type = 'Unknown'
end
end
def import_wip
self.end_date = #book.worksheet(0).row(0)[3]
puts 'end_date assigned'
end
def formatted_end_date
end_date.strftime("%d %b, %Y")
end
end
However, it fails and the rails server window says
Started POST "/datafiles" for 127.0.0.1 at 2011-05-24 16:05:25 +0200
Processing by DatafilesController#create as HTML
Parameters: {"utf8"=>"✓", "datafile"=>{"upload"=>#<ActionDispatch::Http::UploadedFile:0xa0282d0 #original_filename="wip.xls", #content_type="application/vnd.ms-excel", #headers="Content-Disposition: form-data; name=\"datafile[upload]\"; filename=\"wip.xls\"\r\nContent-Type: application/vnd.ms-excel\r\n", #tempfile=#<File:/tmp/RackMultipart20110524-14236-1kcu3hm>>}, "commit"=>"Import"}
Upload failed. undefined method `original_filename' for nil:NilClass
Rendered datafiles/new.html.erb within layouts/application (54.5ms)
Completed 200 OK in 131ms (Views: 56.3ms | ActiveRecord: 0.0ms)
I have rspec model tests that pass and controller tests that fail to redirect after saving. I can post them if it would be useful.
I inserted the raise #datafile.to_yaml and got the following in the terminal.
ERROR RuntimeError: --- !ruby/object:Datafile
attributes:
filename:
report_type:
import_successful:
project:
begin_date:
end_date:
created_at:
updated_at:
attributes_cache: {}
changed_attributes: {}
destroyed: false
marked_for_destruction: false
persisted: false
previously_changed: {}
readonly: false
I notice that :upload is not listed. Can I set model instance variables from the form? :upload is an instance variable, not an attribute, because I do not want to keep the uploaded file in the database (just its path to the local directory). If I cannot set instance variables in the view's form, any suggestions? Does it make sense (in terms of MVC) to upload the file to a temp folder in the controller, then create a model record by passing it the temp file's path?
Hello I am pretty new to Rails and was strugling with this as well I found a solution though it probably isn't the best. It does work though.
in you model make a public function called import_upload
def import_upload( upload )
#uploaded_file = upload
end
now in your controller you can explicitly pass it. I don't know why this doesn't happen automatically if you make an attr_accsessor with the same name as the file_field but this was the solution that worked for me.
def new
foo = Foo.new( params[:foo] )
foo.import_upload( params[:foo][:uploaded_file] ) #This is were the majic happens
#Do your saving stuff and call it a day
end

Not losing paperclip attachment when model cannot be saved due to validation error

The scenario is a normal model that contains a paperclip attachment along with some other columns that have various validations. When a form to to create an object cannot be saved due to a validation error unrelated to the attachment, columns like strings are preserved and remain prefilled for the user, but a file selected for uploading is completely lost and must be reselected by the user.
Is there a standard approach to preserving the attachment in the case of a model validation error? This seems like a very common use case.
It seems inelegant to hack up a solution where the file is saved without an owner and then reconnected to the object after it's successfully saved so I'm hoping to avoid this.
Switch to using CarrierWave. I know this was in a comment, but I just spent all day making the transition so my answer may be helpful still.
First you can follow a great railscast about setting up carrier wave: http://railscasts.com/episodes/253-carrierwave-file-uploads
To get it to preserve the image between posts, you need to add a hidden field with the suffix 'cache':
<%= form_for #user, :html => {:multipart => true} do |f| %>
<p>
<label>My Avatar</label>
<%= f.file_field :avatar %>
<%= f.hidden_field :avatar_cache %>
</p>
<% end %>
For Heroku
And if you're deploying to Heroku like I am, you need to make some changes to get it to work, since the caching works by temporarily saving uploads in a directory called public/uploads. Since the filesystem is readonly in Heroku, you need to have it use the tmp folder instead, and have rack serve static files from there.
Tell carrierwave to use the tmp folder for caching.
In your config/initializers/carrierwave.rb (feel free to create if not there), add:
CarrierWave.configure do |config|
config.root = Rails.root.join('tmp')
config.cache_dir = 'carrierwave'
end
Configure rack to serve static files in from the tmp/carrierwave folder
In your config.ru file, add:
use Rack::Static, :urls => ['/carrierwave'], :root => 'tmp'
For an example of a fully functional barebones rails/carrierwave/s3/heroku app, check out:
https://github.com/trevorturk/carrierwave-heroku (no affiliation, just was useful).
Hope this helps!
I had to fix this on a recent project using PaperClip. I've tried calling cache_images() using after_validation and before_save in the model but it fails on create for some reason that I can't determine so I just call it from the controller instead.
model:
class Shop < ActiveRecord::Base
attr_accessor :logo_cache
has_attached_file :logo
def cache_images
if logo.staged?
if invalid?
FileUtils.cp(logo.queued_for_write[:original].path, logo.path(:original))
#logo_cache = encrypt(logo.path(:original))
end
else
if #logo_cache.present?
File.open(decrypt(#logo_cache)) {|f| assign_attributes(logo: f)}
end
end
end
private
def decrypt(data)
return '' unless data.present?
cipher = build_cipher(:decrypt, 'mypassword')
cipher.update(Base64.urlsafe_decode64(data).unpack('m')[0]) + cipher.final
end
def encrypt(data)
return '' unless data.present?
cipher = build_cipher(:encrypt, 'mypassword')
Base64.urlsafe_encode64([cipher.update(data) + cipher.final].pack('m'))
end
def build_cipher(type, password)
cipher = OpenSSL::Cipher::Cipher.new('DES-EDE3-CBC').send(type)
cipher.pkcs5_keyivgen(password)
cipher
end
end
controller:
def create
#shop = Shop.new(shop_params)
#shop.user = current_user
#shop.cache_images
if #shop.save
redirect_to account_path, notice: 'Shop created!'
else
render :new
end
end
def update
#shop = current_user.shop
#shop.assign_attributes(shop_params)
#shop.cache_images
if #shop.save
redirect_to account_path, notice: 'Shop updated.'
else
render :edit
end
end
view:
= f.file_field :logo
= f.hidden_field :logo_cache
- if #shop.logo.file?
%img{src: #shop.logo.url, alt: ''}
Following the idea of #galatians , i got this solution (and worked beautfully )
Created a repo to that example:
* https://github.com/mariohmol/paperclip-keeponvalidation
The first thing to do is put some methods in your base active record, so every model that uses attach you can make it work
In config/initializers/active_record.rb
module ActiveRecord
class Base
def decrypt(data)
return '' unless data.present?
cipher = build_cipher(:decrypt, 'mypassword')
cipher.update(Base64.urlsafe_decode64(data).unpack('m')[0]) + cipher.final
end
def encrypt(data)
return '' unless data.present?
cipher = build_cipher(:encrypt, 'mypassword')
Base64.urlsafe_encode64([cipher.update(data) + cipher.final].pack('m'))
end
def build_cipher(type, password)
cipher = OpenSSL::Cipher::Cipher.new('DES-EDE3-CBC').send(type)
cipher.pkcs5_keyivgen(password)
cipher
end
#ex: #avatar_cache = cache_files(avatar,#avatar_cache)
def cache_files(avatar,avatar_cache)
if avatar.queued_for_write[:original]
FileUtils.cp(avatar.queued_for_write[:original].path, avatar.path(:original))
avatar_cache = encrypt(avatar.path(:original))
elsif avatar_cache.present?
File.open(decrypt(avatar_cache)) {|f| assign_attributes(avatar: f)}
end
return avatar_cache
end
end
end
After that , include in your model and attached field, the code above
In exemple, i included that into /models/users.rb
has_attached_file :avatar, PaperclipUtils.config
attr_accessor :avatar_cache
def cache_images
#avatar_cache=cache_files(avatar,#avatar_cache)
end
In your controller, add this to get from cache the image (just before the point where you save the model)
#user.avatar_cache = params[:user][:avatar_cache]
#user.cache_images
#user.save
And finally include this in your view, to record the location of the current temp image
f.hidden_field :avatar_cache
If you want to show in view the actual file, include it:
<% if #user.avatar.exists? %>
<label class="field">Actual Image </label>
<div class="field file-field">
<%= image_tag #user.avatar.url %>
</div>
<% end %>
As of Sept 2013, paperclip has no intention of "fixing" the losing of attached files after validation. "The problem is (IMHO) more easily and more correctly avoided than solved"
https://github.com/thoughtbot/paperclip/issues/72#issuecomment-24072728
I'm considering the CarrierWave solution proposed in John Gibb's earlier solution
Also check out refile (newer option)
Features:
Configurable backends, file system, S3, etc...
Convenient integration with ORMs
On the fly manipulation of images and other files
Streaming IO for fast and memory friendly uploads
Works across form redisplays, i.e. when validations fail, even on S3
Effortless direct uploads, even to S3
Support for multiple file uploads
https://gorails.com/episodes/file-uploads-with-refile
If the image isn't required why not split the form into two stages, the first one creates the object, the second page lets you add optional information (like a photo).
Alternatively you could validate the form as the user enters the information so that you don't have to submit the form to find out your data is invalid.
save your picture first than try the rest
lets say you have a user with a paperclip avatar:
def update
#user = current_user
unless params[:user][:avatar].nil?
#user.update_attributes(avatar: params[:user][:avatar])
params[:user].delete :avatar
end
if #user.update_attributes(params[:user])
redirect_to edit_profile_path, notice: 'User was successfully updated.'
else
render action: "edit"
end
end
In view file just put if condition that should accept only the record which had valid id.
In my scenario this is the code snippet
<p>Uploaded files:</p>
<ul>
<% #user.org.crew.w9_files.each do |file| %>
<% if file.id.present? %>
<li> <%= rails code to display value %> </li>
<% end %>
<% end %>
</ul>

Resources