I am creating a very simple web app that allows the user to upload a .zip file that I temporarily save in the tmp folder inside my application, parse the contents using zipfile and then delete the file after I'm done.
I managed to upload the file and copy it to the tmp folder, I can successfully parse it and get the results I want, but when I try to delete the file I get a permission denied error.
here's my view:
<%= form_tag({action: :upload}, multipart: true) do %>
<%= file_field_tag :software %>
<br/><br/>
<%= submit_tag("UPLOAD") %>
<% end %>
And here's my controller:
def upload
#file = params[:software]
#name = #file.original_filename
File.open(Rails.root.join('tmp', #name), 'wb') do |file|
file.write(#file.read)
end
parse
File.delete("tmp/#{#name}")
render action: "show"
end
I have tried using FileUtils.rm ("tmp/#{#name}") as well, and I also tried setting File.chmod(0777, "tmp/#{#name}") before deletion but to no avail. Changing the deletion path to Rails.root.join('tmp', #name) like the File.open block also doesn't fix it. I can totally delete the file via console so I don't know what can be the matter.
EDIT: The parse method:
def parse
require 'zip'
Zip::File.open("tmp/#{#nome}") do |zip_file|
srcmbffiles = File.join("**", "src", "**", "*.mbf")
entry = zip_file.glob(srcmbffiles).first
#stream = entry.get_input_stream.read
puts #stream
end
end
The issue was that for some reason my file was not being closed for deletion in either the File.open block or the Zip::File.open block. My solution was to close it manually and avoid using open blocks, changing this snippet:
File.open(Rails.root.join('tmp', #name), 'wb') do |file|
file.write(#file.read)
end
into this:
f = File.open(Rails.root.join('tmp', #nome), 'wb+')
f.write(#file.read)
f.close
and changing my parse method from this:
def parse
require 'zip'
Zip::File.open("tmp/#{#nome}") do |zip_file|
srcmbffiles = File.join("**", "src", "**", "*.mbf")
entry = zip_file.glob(srcmbffiles).first
#stream = entry.get_input_stream.read
puts #stream
end
end
to this:
def parse
require 'zip'
zf = Zip::File.open("tmp/#{#nome}")
srcmbffiles = File.join("**", "src", "**", "*.mbf")
entry = zf.glob(srcmbffiles).first
#stream = zf.read(entry)
puts #stream
zf.close()
end
Notice that I changed the way I populate #stream because apparently entry.get_input_stream also locks the file you're accessing.
The writing process may be still locking the file. You may have to wait until that process is complete.
'"tmp/#{#name}"' is not right path. Just use 'Rails.root.join('tmp', #name)'
Related
I am building a very simple web tool where user can upload a CSV file, which is then processed and the result CSV can be immediately downloaded.
The upload form:
<%= form_tag '/upload', multipart: true do %>
<%= file_field_tag :csv %>
<%= submit_tag 'Import CSV' %>
<% end %>
The upload and download actions:
def upload
original_csv = params[:csv]
p original_csv.path # /var/folders/71/chp2vrc92_19b3jt2fcwhvp80000gn/T/RackMultipart20181025-11469-25guh5.csv
redirect_to result_path(file_path: original_csv.path)
end
def result
p params[:file_path] # /var/folders/71/chp2vrc92_19b3jt2fcwhvp80000gn/T/RackMultipart20181025-11469-25guh5.csv
output_csv = CSV.generate do |csv|
CSV.foreach(params[:file_path], headers: true) do |row|
#############################################
# "No such file or directory # rb_sysopen" #
# exception is thrown #
#############################################
# each row data is being processed here
csv << row
end
end
# Download the file into user's computer
send_data output_csv
end
As you can see from the comments, this method doesn't work because the temp file path no longer exists in result action. How can I go about this without touching db at all.
Uploaded files are stored as temp files by the application. That means once the request has ended the temp file is automatically deleted. Therefore it doesn't exist anymore when the next page is requested.
One option would be to copy the file by yourself to another location and make it a "real" file in the file system that isn't deleted automatically anymore. But that has downsides too: Now you are responsible to manage and delete these files by yourself too. That means you need to generate unique files names and pass them to the next request and you need to ensure that the file is deleted after it was downloaded otherwise these files would slowly consume all space on your server's disk. Furthermore, this doesn't scale to multiple servers and will only work for small applications running on one server.
A better option might be to just do the upload, the processing and the download in one request, without any redirect. As long as the processing can be done in a reasonable time and in memory this might be a good option to avoid complexity.
def upload
original_csv = params[:csv]
output_csv = CSV.generate do |csv|
CSV.foreach(original_csv.path, headers: true) do |row|
# process data
csv << row
end
end
send_data output_csv
end
Try this:
def upload
result(params[:csv])
end
def result(fpath=params[:file_path])
output_csv = CSV.generate do |csv|
CSV.foreach(fpath, headers: true) do |row|
csv << row
end
end
# Download the file into user's computer
send_data output_csv
end
I installed the camt_parser gem in my rails app.
My goal is to upload and parse camt_file (.xml).
It works perfectly when I parse from a local file this way:
require 'camt_parser'
require 'camt'
camt = CamtParser::File.parse 'CAMT053_140518.xml'
puts camt.group_header.creation_date_time
camt.statements.each do |statement|
statement.entries.each do |entry|
# Access individual entries/bank transfers
# puts "->"
puts entry.description
puts entry.debit
p entry.transactions[0].iban
p entry.transactions[0].transaction_id
puts entry.transactions[0].debitor.iban
end
end
But when I try to upload it from my view as a file using this code:
<%= form_tag '/patient_page/import_camt', :multipart => true do %>
<label for="file">Upload text File</label> <%= file_field_tag "file" %>
<%= submit_tag %>
<% end %>
and
the corresponding method:
def import_camt
uploaded_file = params[:file]
parsed_file = uploaded_file.read
camt = CamtParser::File.parse uploaded_file
puts camt.group_header.creation_date_time
camt.statements.each do |statement|
statement.entries.each do |entry|
puts entry.description
puts entry.debit
p entry.transactions[0].iban
p entry.transactions[0].transaction_id
puts entry.transactions[0].debitor.iban
end
end
end
I get the following error
"no implicit conversion of ActionDispatch::Http::UploadedFile into String"
at the line when I try to parse the uploaded file.
Any hints?
Thx!
CamtParser::File.parse is expecting a file path but you are passing an ActionDispatch::Http::UploadedFile object.
When you upload a file in rails the file is wrapped in an instance of ActionDispatch::Http::UploadedFile. To access the file itself there is an attribute called tempfile.
This will return the Tempfile that represents the actual file that is being uploaded. A Tempfile has a path method which is the path to the file itself so since CamtParser::File.parse is expecting a file path it can be called as follows
CamtParser::File.parse(uploaded_file.tempfile.path)
CamtParser also has a String class that can parse an appropriate string so you could call this as
CamtParser::String.parse(uploaded_file.read)
This works because ActionDispatch::Http::UploadedFile exposes the method read which is the same as calling uploaded_file.tempfile.read
I'm trying to save the Paperclip uploaded images throw a Sidekiq worker.
The user select the images into a image[] and passes it to the controller (as params[:image]).
View:
<%= file_field_tag "image[]", type: :file, multiple: true %>
When I pass it to another var (files), it works in the controller, but when I pass it to the sidekiq_worker, it turns into a hash of strings
Controller:
file = params[:image]
SidekiqWorker.perform_async("import_images", file)
SidekiqWorker
def perform(type, file)
case type
when "import_images"
file.each do |picture|
puts picture.class
puts picture.original_filename
end
Product.import_images(file)
puts "SIDEKIQ_WORKER: IMPORTING IMAGES"
end
end
How can I pass a hash of image-hashes? Or how can I achieve what I want to do?
After that, the images are processed into a model, but the hash already turned into a string and it does not works.
def self.import_images(file)
file.each do |picture|
#product = Product.where(code: File.basename(picture.original_filename, ".*"))
if(!#product.nil?)
#product.update(:image=> picture)
end
end
end
Thank you for your help :)
So, what I just did to make it happen...
After the user uploaded the files, it saves them in a folder and a variable in controller gets the name of each image.
when "Import images"
file = Array.new
params[:image].each do |picture|
File.open(Rails.root.join('public/system/products', 'uploaded', picture.original_filename), 'wb') do |f|
f.write(picture.read)
end
file.push picture.original_filename
end
SidekiqWorker.perform_async("import_images", file,0,0)
redirect_to products_url, notice: "#{t 'controllers.products.images'}"
After that, it passes to my sidekiq_worker and passes to my model, where I search for the image and search for the product where the code equals the name of the image. After processing, deletes the file of the uploaded image :)
def self.import_images(file)
file.each do |image|
uploaded = open("public/system/products/uploaded/"+image)
prd = Product.where(code: File.basename(uploaded, '.*'))
if !prd.nil?
prd.update(image: uploaded)
end
File.delete("public/system/products/uploaded/"+image)
end
end
I am trying to upload a file to Box.com using its API REST call and the httmultiparty gem. The code is working and uploads to Box.com but does that after writing the uploaded file to the server file system as in f.write(data.read) then capturing the file path for the written file as the input parameter to the Box.com API REST call as in :filename => File.new(path). The app will be running on Heroku, so we can't save any files (read only) on Heroku's server so I would like to directly upload the file to Box.com while bypassing the writing of the file on the server but can't figure that out given that the Box.com REST call requires an object of type "File". Any help is appreciated. Thanks.
The model and view code is:
###
#The Model
###
class BoxUploader
require 'httmultiparty'
include HTTMultiParty
#base_uri 'https://api.box.com/2.0'
end
class File < ActiveRecord::Base
attr_accessible :file
attr_accessor :boxResponse
FILE_STORE = File.join Rails.root, 'public', 'files'
API_KEY = #myBoxApiKey
AUTH_TOKEN = #myBoxAuthToken
def file=(data) #uploaded file
filename = data.original_filename
path = File.join FILE_STORE, filename
#### would like to bypass the file writing step
File.open(path, "wb") do |f|
f.write(data.read)
end
#############
File.open(path, "wb") do |f|
boxResponse = BoxUploader.post('https://api.box.com/2.0/files/content',
:headers => { 'authorization' => 'BoxAuth api_key={API_KEY&auth_token=AUTH_TOKEN' },
:body => { :folder_id => '911', :filename => File.new(path)}
)
end
end
###
# The View
###
<!-- Invoke the Controller's "create" action -->
<h1>File Upload</h1>
<%= form_for #file, :html => {:multipart=>true} do |f| %>
<p>
<%= f.label :file %>
<%= f.file_field :file %>
</p>
<p>
<%= f.submit 'Create' %>
<% end %>
To upload a file from memory with HTTMultiParty, you need to supply it with an UploadIO object in place of the File object you'd normally give it. The UploadIO object can be populated using StringIO. It seems HTTMultiParty handles UploadIO objects in a special way, so you can't use the StringIO directly:
class Uploader
include HTTMultiParty
base_uri "http://foo.com"
end
string_io = StringIO.new('some stuff that pretends to be in a file')
upload_io = UploadIO.new(string_io, 'text/plain', 'bar.txt')
Uploader.post("/some/path", query: {file: upload_io})
You are aiming at a non-common use pattern, so your best shot could be to extend the existant gem, to provide the functionality you need.
There is a gem ruby-box to use with Box service at the 2.0 version of their API.
The gem is well supported and pretty easy to use.
You'll need to dig on the source code and create a new upload method.
I have a form that is attempting to read in a JSON file for parsing/actions/etc. I'm having problems getting it to read in the controller.
View:
<%= form_tag({:controller => :admins, :action => :upload_json}, {:multipart => true, :method => :post}) do |f| %>
<%= file_field_tag 'datafile' %>
<%= submit_tag "Upload" %>
Controller:
def upload_json
file_data = params[:datafile]
File.read(file_data) do |file|
file.each do |line|
## does stuff here....
end
end
end
A similar function works in my seed.rb file when I'm seeding data - just can't get it to read in an uploaded file.
The error I'm getting is: can't convert ActionDispatch::Http::UploadedFile into String.
Thanks in advance for the help!
Figured it out. Needed to change:
file_data = params[:datafile]
to
file_data = params[:datafile].tempfile
And decided to use the .open function to change:
File.read(file_data) do |file|
to
File.open(file_data, 'r') do |file|
params[:datafile] is an instance of ActionDispatch::Http::UploadedFile class with tempfile attached with that.To open the tempfile
You try something like
File.open(params[:datafile].path) do |file|
#your stuff goes here
end
Open the uploaded file using path.
params[:datafile] is an instance of ActionDispatch::Http::UploadedFile class and you'll need to get at the stored file by calling path to properly process it.
Additionally, File.read will not get you the line-by-line processing you're looking for. You need to change that to File.open.
Try this:
Controller
def upload_json
uploaded_datafile = params[:datafile]
File.open( uploaded_datafile.path ) do |file|
file.each_line do |line|
# Do something with each line.
end
end
end
Alternative Style
def upload_json
File.foreach( params[:datafile].path ) do |line|
# Do something with each line.
end
# FYI: The above method block returns `nil` when everything goes okay.
end