Save Paperclip image with Sidekiq - ruby-on-rails

I'm trying to save the Paperclip uploaded images throw a Sidekiq worker.
The user select the images into a image[] and passes it to the controller (as params[:image]).
View:
<%= file_field_tag "image[]", type: :file, multiple: true %>
When I pass it to another var (files), it works in the controller, but when I pass it to the sidekiq_worker, it turns into a hash of strings
Controller:
file = params[:image]
SidekiqWorker.perform_async("import_images", file)
SidekiqWorker
def perform(type, file)
case type
when "import_images"
file.each do |picture|
puts picture.class
puts picture.original_filename
end
Product.import_images(file)
puts "SIDEKIQ_WORKER: IMPORTING IMAGES"
end
end
How can I pass a hash of image-hashes? Or how can I achieve what I want to do?
After that, the images are processed into a model, but the hash already turned into a string and it does not works.
def self.import_images(file)
file.each do |picture|
#product = Product.where(code: File.basename(picture.original_filename, ".*"))
if(!#product.nil?)
#product.update(:image=> picture)
end
end
end
Thank you for your help :)

So, what I just did to make it happen...
After the user uploaded the files, it saves them in a folder and a variable in controller gets the name of each image.
when "Import images"
file = Array.new
params[:image].each do |picture|
File.open(Rails.root.join('public/system/products', 'uploaded', picture.original_filename), 'wb') do |f|
f.write(picture.read)
end
file.push picture.original_filename
end
SidekiqWorker.perform_async("import_images", file,0,0)
redirect_to products_url, notice: "#{t 'controllers.products.images'}"
After that, it passes to my sidekiq_worker and passes to my model, where I search for the image and search for the product where the code equals the name of the image. After processing, deletes the file of the uploaded image :)
def self.import_images(file)
file.each do |image|
uploaded = open("public/system/products/uploaded/"+image)
prd = Product.where(code: File.basename(uploaded, '.*'))
if !prd.nil?
prd.update(image: uploaded)
end
File.delete("public/system/products/uploaded/"+image)
end
end

Related

Parse a xml uploaded with camt_parser rails gem

I installed the camt_parser gem in my rails app.
My goal is to upload and parse camt_file (.xml).
It works perfectly when I parse from a local file this way:
require 'camt_parser'
require 'camt'
camt = CamtParser::File.parse 'CAMT053_140518.xml'
puts camt.group_header.creation_date_time
camt.statements.each do |statement|
statement.entries.each do |entry|
# Access individual entries/bank transfers
# puts "->"
puts entry.description
puts entry.debit
p entry.transactions[0].iban
p entry.transactions[0].transaction_id
puts entry.transactions[0].debitor.iban
end
end
But when I try to upload it from my view as a file using this code:
<%= form_tag '/patient_page/import_camt', :multipart => true do %>
<label for="file">Upload text File</label> <%= file_field_tag "file" %>
<%= submit_tag %>
<% end %>
and
the corresponding method:
def import_camt
uploaded_file = params[:file]
parsed_file = uploaded_file.read
camt = CamtParser::File.parse uploaded_file
puts camt.group_header.creation_date_time
camt.statements.each do |statement|
statement.entries.each do |entry|
puts entry.description
puts entry.debit
p entry.transactions[0].iban
p entry.transactions[0].transaction_id
puts entry.transactions[0].debitor.iban
end
end
end
I get the following error
"no implicit conversion of ActionDispatch::Http::UploadedFile into String"
at the line when I try to parse the uploaded file.
Any hints?
Thx!
CamtParser::File.parse is expecting a file path but you are passing an ActionDispatch::Http::UploadedFile object.
When you upload a file in rails the file is wrapped in an instance of ActionDispatch::Http::UploadedFile. To access the file itself there is an attribute called tempfile.
This will return the Tempfile that represents the actual file that is being uploaded. A Tempfile has a path method which is the path to the file itself so since CamtParser::File.parse is expecting a file path it can be called as follows
CamtParser::File.parse(uploaded_file.tempfile.path)
CamtParser also has a String class that can parse an appropriate string so you could call this as
CamtParser::String.parse(uploaded_file.read)
This works because ActionDispatch::Http::UploadedFile exposes the method read which is the same as calling uploaded_file.tempfile.read

Rails: Permission denied for File.delete

I am creating a very simple web app that allows the user to upload a .zip file that I temporarily save in the tmp folder inside my application, parse the contents using zipfile and then delete the file after I'm done.
I managed to upload the file and copy it to the tmp folder, I can successfully parse it and get the results I want, but when I try to delete the file I get a permission denied error.
here's my view:
<%= form_tag({action: :upload}, multipart: true) do %>
<%= file_field_tag :software %>
<br/><br/>
<%= submit_tag("UPLOAD") %>
<% end %>
And here's my controller:
def upload
#file = params[:software]
#name = #file.original_filename
File.open(Rails.root.join('tmp', #name), 'wb') do |file|
file.write(#file.read)
end
parse
File.delete("tmp/#{#name}")
render action: "show"
end
I have tried using FileUtils.rm ("tmp/#{#name}") as well, and I also tried setting File.chmod(0777, "tmp/#{#name}") before deletion but to no avail. Changing the deletion path to Rails.root.join('tmp', #name) like the File.open block also doesn't fix it. I can totally delete the file via console so I don't know what can be the matter.
EDIT: The parse method:
def parse
require 'zip'
Zip::File.open("tmp/#{#nome}") do |zip_file|
srcmbffiles = File.join("**", "src", "**", "*.mbf")
entry = zip_file.glob(srcmbffiles).first
#stream = entry.get_input_stream.read
puts #stream
end
end
The issue was that for some reason my file was not being closed for deletion in either the File.open block or the Zip::File.open block. My solution was to close it manually and avoid using open blocks, changing this snippet:
File.open(Rails.root.join('tmp', #name), 'wb') do |file|
file.write(#file.read)
end
into this:
f = File.open(Rails.root.join('tmp', #nome), 'wb+')
f.write(#file.read)
f.close
and changing my parse method from this:
def parse
require 'zip'
Zip::File.open("tmp/#{#nome}") do |zip_file|
srcmbffiles = File.join("**", "src", "**", "*.mbf")
entry = zip_file.glob(srcmbffiles).first
#stream = entry.get_input_stream.read
puts #stream
end
end
to this:
def parse
require 'zip'
zf = Zip::File.open("tmp/#{#nome}")
srcmbffiles = File.join("**", "src", "**", "*.mbf")
entry = zf.glob(srcmbffiles).first
#stream = zf.read(entry)
puts #stream
zf.close()
end
Notice that I changed the way I populate #stream because apparently entry.get_input_stream also locks the file you're accessing.
The writing process may be still locking the file. You may have to wait until that process is complete.
'"tmp/#{#name}"' is not right path. Just use 'Rails.root.join('tmp', #name)'

Audio files always null using Taglib-ruby in rails

I'm trying to build an app in Rails which will take audio file uploads and read metadata off them to populate a database. I'm using the Taglib-ruby gem to handle various file types. The uploads seem to be working on their own, but Taglib considers any file given to it as null.
Here's my controller:
class UploadsController < ApplicationController
require 'taglib'
def new
end
def create
file = params[:upload]
TagLib::FileRef.open(file) do |fileref|
unless fileref.null?
tag = fileref.tag
# properties = fileref.audio_properties
#song = Song.new(title: tag.title, artist: tag.artist, album: tag.album,
year: tag.year, track: tag.track, genre: tag.genre)
if #song.save
redirect_to songs_path
else
render 'new'
end
else
raise "file was null"
end
end
end
end
and my view for form submission:
<h1> Upload </h1>
<%= form_tag(url: { action: :create }, html: { multipart: true }) do %>
<%= label_tag :upload, "Scan your song:" %>
<%= file_field_tag :upload, multiple: true %>
<br />
<%= submit_tag "Submit" %>
<% end %>
Taglib itself seems to be working - adding "require 'taglib'" removed the error I had been getting in regards to that, and a mock-up I made of this outside of rails worked fine (so the files I'm using are also not the problem). Every time I run this, the control flow hits my raise command, and no record is saved. It's clear that fileref.null? is returning true, which suggests to me that there's something wrong with the upload process... but I'm not sure what.
Ideally, I'd like to use the multiple uploads option and run this process on each file sequentially, but I can't even get a single upload to register as anything but null.

How to persist file upload fields after a rails validation error

I have a form with multiple file uploads, The issue is when i am submitting the form and an validation error occurs, the file input field gets reset.
I basically wanted to persist those files inside the file input field for the complete process.
I have also gone through few links
How can I "keep" the uploaded image on a form validation error?
Please let me know what are the various options in such cases that one can follow.
Carrierwave is a great tool for handling file uploads and can handle this for you
https://github.com/jnicklas/carrierwave#making-uploads-work-across-form-redisplays
I took a completely different approach to the other solutions on offer here, as I didn't fancy switching to CarrierWave or using yet another gem to implement a hack to get around this.
Basically, I define placeholders for validation error messages and then make an AJAX call to the relevant controller. should it fail validation I simply populate the error message placeholders - this leaves everything in place client side including the file input ready for resubmission.
Example follows, demonstrating an organisation with nested address model and a nested logo model (that has a file attachment) - this has been cut for brevity :
organisations/_form.html.erb
<%= form_for #organisation, html: {class: 'form-horizontal', role: 'form', multipart: true}, remote: true do |f| %>
<%= f.label :name %>
<%= f.text_field :name %>
<p class='name error_explanation'></p>
<%= f.fields_for :operational_address do |fa| %>
<%= fa.label :postcode %>
<%= fa.text_field :postcode %>
<p class='operational_address postcode error_explanation'></p>
<% end %>
<%= f.fields_for :logo do |fl| %>
<%= fl.file_field :image %>
<p class='logo image error_explanation'></p>
<% end %>
<% end %>
organisations_controller.rb
def create
if #organisation.save
render :js => "window.location = '#{organisations_path}'"
else
render :validation_errors
end
end
organisations/validation_errors.js.erb
$('.error_explanation').html('');
<% #organisation.errors.messages.each do |attribute, messages| %>
$('.<%= attribute %>.error_explanation').html("<%= messages.map{|message| "'#{message}'"}.join(', ') %>");
<% end %>
Created a repo with a example of using Paperclip on rails and mainting your files when validation error occurs
https://github.com/mariohmol/paperclip-keeponvalidation
I had to fix this on a recent project using the Paperclip Gem. It's a bit hacky but it works. I've tried calling cache_images() using after_validation and before_save in the model but it fails on create for some reason that I can't determine so I just call it from the controller instead. Hopefully this saves someone else some time!
model:
class Shop < ActiveRecord::Base
attr_accessor :logo_cache
has_attached_file :logo
def cache_images
if logo.staged?
if invalid?
FileUtils.cp(logo.queued_for_write[:original].path, logo.path(:original))
#logo_cache = encrypt(logo.path(:original))
end
else
if #logo_cache.present?
File.open(decrypt(#logo_cache)) {|f| assign_attributes(logo: f)}
end
end
end
private
def decrypt(data)
return '' unless data.present?
cipher = build_cipher(:decrypt, 'mypassword')
cipher.update(Base64.urlsafe_decode64(data).unpack('m')[0]) + cipher.final
end
def encrypt(data)
return '' unless data.present?
cipher = build_cipher(:encrypt, 'mypassword')
Base64.urlsafe_encode64([cipher.update(data) + cipher.final].pack('m'))
end
def build_cipher(type, password)
cipher = OpenSSL::Cipher::Cipher.new('DES-EDE3-CBC').send(type)
cipher.pkcs5_keyivgen(password)
cipher
end
end
controller:
def create
#shop = Shop.new(shop_params)
#shop.user = current_user
#shop.cache_images
if #shop.save
redirect_to account_path, notice: 'Shop created!'
else
render :new
end
end
def update
#shop = current_user.shop
#shop.assign_attributes(shop_params)
#shop.cache_images
if #shop.save
redirect_to account_path, notice: 'Shop updated.'
else
render :edit
end
end
view:
= f.file_field :logo
= f.hidden_field :logo_cache
- if #shop.logo.file?
%img{src: #shop.logo.url, alt: ''}
Well - I thought of taking a different approach to this; Instead of temporarily storing the file on the server, why not serve it back to the client to be resubmitted when the user fixes the validation issues.
This might still need a bit of refinement but it's the general concept:
# in the controller - save the file and its attributes to params
def create
# ...
if params[:doc] # a regular file uploaded through the file form element
# when the form re-renders, it will have those additional params available to it
params[:uploaded_file] = params[:doc].read # File contents
params[:uploaded_file_original_filename] = params[:doc].original_filename
params[:uploaded_file_headers] = params[:doc].headers
params[:uploaded_file_content_type] = params[:doc].content_type
elsif params[:uploaded_file] # a file coming through the form-resubmit
# generate an ActionDispatch::Http::UploadedFile
tempfile = Tempfile.new("#{params[:uploaded_file_original_filename]}-#{Time.now}")
tempfile.binmode
tempfile.write CGI.unescape(params[:uploaded_file]) #content of the file / unescaped
tempfile.close
# merge into the params
params.merge!(doc:
ActionDispatch::Http::UploadedFile.new(
:tempfile => tempfile,
:filename => params[:uploaded_file_original_filename],
:head => params[:uploaded_file_headers],
:type => params[:uploaded_file_content_type]
)
)
end
#...
# params (including the UploadedFile) can be used to generate and save the model object
end
# in the form (haml)
- if !params[:uploaded_file].blank?
# file contents in hidden textarea element
= text_area_tag(:uploaded_file, CGI.escape(params[:uploaded_file]), style: 'display: none;') #escape the file content
= hidden_field_tag :uploaded_file_headers, params[:uploaded_file_headers]
= hidden_field_tag :uploaded_file_content_type, params[:uploaded_file_content_type]
= hidden_field_tag :uploaded_file_original_filename, params[:uploaded_file_original_filename]
A workaround for this rather than an outright solution is to use client side validation so that the file isn't lost because the whole form persists.
The few users that don't have JavaScript enabled will lose the files between requests, but perhaps this % is so low for you as to make it an acceptable compromise. If this is the route you decide to go down I'd recommend this gem
https://github.com/bcardarella/client_side_validations
Which makes the whole process really simple and means you don't have to rewrite your validation in JavaScript
Browsers block against setting the value attribute on input of file
type for security reasons so that you can't upload a file without the
user's selected any file itself.
Pre-Populate HTML form file input
You can use carrierwave: https://github.com/carrierwaveuploader/carrierwave
Or validate the model via js request.
I found a way to keep files without using gems, it can probably be improved but I am still a young dev :)
I draw the whole thing from solution 2 contained in this article: https://medium.com/earthvectors/validations-and-file-caching-using-activestorage-e16418060f8f
First of all, you need to add an hidden_field within your form that contains the signed_id of the attachment:
<%= f.hidden_field :file_signed_id, value: #model.file.signed_id if #model.file.attached? %>
<%= f.input :file %>
The problem when validations fail (in my case), it keeps the file in memory but do not send anymore the ActionDispatch object as parameters. To override it, I did the following in my controller:
if file = params.dig(:model, :file)
blob = ActiveStorage::Blob.create_and_upload!(
io: File.open(file.tempfile),
filename: file.original_filename
)
#model.file.attach(blob)
elsif file_signed_id = params.dig(:model, file_signed_id)
blob = ActiveStorage::Blob.find_signed(file_signed_id)
#model.file.attach(blob)
end
You then can display your file when rendering your view again:
<%= link_to #model.file.filename, url_for(#model.file) if #model.file.attached? %>
The only problem I see with this workaround it is that it will create a blob object even if validations fail.
I hope it will help someone!

Reading in file contents rails

I have a form that is attempting to read in a JSON file for parsing/actions/etc. I'm having problems getting it to read in the controller.
View:
<%= form_tag({:controller => :admins, :action => :upload_json}, {:multipart => true, :method => :post}) do |f| %>
<%= file_field_tag 'datafile' %>
<%= submit_tag "Upload" %>
Controller:
def upload_json
file_data = params[:datafile]
File.read(file_data) do |file|
file.each do |line|
## does stuff here....
end
end
end
A similar function works in my seed.rb file when I'm seeding data - just can't get it to read in an uploaded file.
The error I'm getting is: can't convert ActionDispatch::Http::UploadedFile into String.
Thanks in advance for the help!
Figured it out. Needed to change:
file_data = params[:datafile]
to
file_data = params[:datafile].tempfile
And decided to use the .open function to change:
File.read(file_data) do |file|
to
File.open(file_data, 'r') do |file|
params[:datafile] is an instance of ActionDispatch::Http::UploadedFile class with tempfile attached with that.To open the tempfile
You try something like
File.open(params[:datafile].path) do |file|
#your stuff goes here
end
Open the uploaded file using path.
params[:datafile] is an instance of ActionDispatch::Http::UploadedFile class and you'll need to get at the stored file by calling path to properly process it.
Additionally, File.read will not get you the line-by-line processing you're looking for. You need to change that to File.open.
Try this:
Controller
def upload_json
uploaded_datafile = params[:datafile]
File.open( uploaded_datafile.path ) do |file|
file.each_line do |line|
# Do something with each line.
end
end
end
Alternative Style
def upload_json
File.foreach( params[:datafile].path ) do |line|
# Do something with each line.
end
# FYI: The above method block returns `nil` when everything goes okay.
end

Resources