Parse a xml uploaded with camt_parser rails gem - ruby-on-rails

I installed the camt_parser gem in my rails app.
My goal is to upload and parse camt_file (.xml).
It works perfectly when I parse from a local file this way:
require 'camt_parser'
require 'camt'
camt = CamtParser::File.parse 'CAMT053_140518.xml'
puts camt.group_header.creation_date_time
camt.statements.each do |statement|
statement.entries.each do |entry|
# Access individual entries/bank transfers
# puts "->"
puts entry.description
puts entry.debit
p entry.transactions[0].iban
p entry.transactions[0].transaction_id
puts entry.transactions[0].debitor.iban
end
end
But when I try to upload it from my view as a file using this code:
<%= form_tag '/patient_page/import_camt', :multipart => true do %>
<label for="file">Upload text File</label> <%= file_field_tag "file" %>
<%= submit_tag %>
<% end %>
and
the corresponding method:
def import_camt
uploaded_file = params[:file]
parsed_file = uploaded_file.read
camt = CamtParser::File.parse uploaded_file
puts camt.group_header.creation_date_time
camt.statements.each do |statement|
statement.entries.each do |entry|
puts entry.description
puts entry.debit
p entry.transactions[0].iban
p entry.transactions[0].transaction_id
puts entry.transactions[0].debitor.iban
end
end
end
I get the following error
"no implicit conversion of ActionDispatch::Http::UploadedFile into String"
at the line when I try to parse the uploaded file.
Any hints?
Thx!

CamtParser::File.parse is expecting a file path but you are passing an ActionDispatch::Http::UploadedFile object.
When you upload a file in rails the file is wrapped in an instance of ActionDispatch::Http::UploadedFile. To access the file itself there is an attribute called tempfile.
This will return the Tempfile that represents the actual file that is being uploaded. A Tempfile has a path method which is the path to the file itself so since CamtParser::File.parse is expecting a file path it can be called as follows
CamtParser::File.parse(uploaded_file.tempfile.path)
CamtParser also has a String class that can parse an appropriate string so you could call this as
CamtParser::String.parse(uploaded_file.read)
This works because ActionDispatch::Http::UploadedFile exposes the method read which is the same as calling uploaded_file.tempfile.read

Related

Save Paperclip image with Sidekiq

I'm trying to save the Paperclip uploaded images throw a Sidekiq worker.
The user select the images into a image[] and passes it to the controller (as params[:image]).
View:
<%= file_field_tag "image[]", type: :file, multiple: true %>
When I pass it to another var (files), it works in the controller, but when I pass it to the sidekiq_worker, it turns into a hash of strings
Controller:
file = params[:image]
SidekiqWorker.perform_async("import_images", file)
SidekiqWorker
def perform(type, file)
case type
when "import_images"
file.each do |picture|
puts picture.class
puts picture.original_filename
end
Product.import_images(file)
puts "SIDEKIQ_WORKER: IMPORTING IMAGES"
end
end
How can I pass a hash of image-hashes? Or how can I achieve what I want to do?
After that, the images are processed into a model, but the hash already turned into a string and it does not works.
def self.import_images(file)
file.each do |picture|
#product = Product.where(code: File.basename(picture.original_filename, ".*"))
if(!#product.nil?)
#product.update(:image=> picture)
end
end
end
Thank you for your help :)
So, what I just did to make it happen...
After the user uploaded the files, it saves them in a folder and a variable in controller gets the name of each image.
when "Import images"
file = Array.new
params[:image].each do |picture|
File.open(Rails.root.join('public/system/products', 'uploaded', picture.original_filename), 'wb') do |f|
f.write(picture.read)
end
file.push picture.original_filename
end
SidekiqWorker.perform_async("import_images", file,0,0)
redirect_to products_url, notice: "#{t 'controllers.products.images'}"
After that, it passes to my sidekiq_worker and passes to my model, where I search for the image and search for the product where the code equals the name of the image. After processing, deletes the file of the uploaded image :)
def self.import_images(file)
file.each do |image|
uploaded = open("public/system/products/uploaded/"+image)
prd = Product.where(code: File.basename(uploaded, '.*'))
if !prd.nil?
prd.update(image: uploaded)
end
File.delete("public/system/products/uploaded/"+image)
end
end

Rails: Permission denied for File.delete

I am creating a very simple web app that allows the user to upload a .zip file that I temporarily save in the tmp folder inside my application, parse the contents using zipfile and then delete the file after I'm done.
I managed to upload the file and copy it to the tmp folder, I can successfully parse it and get the results I want, but when I try to delete the file I get a permission denied error.
here's my view:
<%= form_tag({action: :upload}, multipart: true) do %>
<%= file_field_tag :software %>
<br/><br/>
<%= submit_tag("UPLOAD") %>
<% end %>
And here's my controller:
def upload
#file = params[:software]
#name = #file.original_filename
File.open(Rails.root.join('tmp', #name), 'wb') do |file|
file.write(#file.read)
end
parse
File.delete("tmp/#{#name}")
render action: "show"
end
I have tried using FileUtils.rm ("tmp/#{#name}") as well, and I also tried setting File.chmod(0777, "tmp/#{#name}") before deletion but to no avail. Changing the deletion path to Rails.root.join('tmp', #name) like the File.open block also doesn't fix it. I can totally delete the file via console so I don't know what can be the matter.
EDIT: The parse method:
def parse
require 'zip'
Zip::File.open("tmp/#{#nome}") do |zip_file|
srcmbffiles = File.join("**", "src", "**", "*.mbf")
entry = zip_file.glob(srcmbffiles).first
#stream = entry.get_input_stream.read
puts #stream
end
end
The issue was that for some reason my file was not being closed for deletion in either the File.open block or the Zip::File.open block. My solution was to close it manually and avoid using open blocks, changing this snippet:
File.open(Rails.root.join('tmp', #name), 'wb') do |file|
file.write(#file.read)
end
into this:
f = File.open(Rails.root.join('tmp', #nome), 'wb+')
f.write(#file.read)
f.close
and changing my parse method from this:
def parse
require 'zip'
Zip::File.open("tmp/#{#nome}") do |zip_file|
srcmbffiles = File.join("**", "src", "**", "*.mbf")
entry = zip_file.glob(srcmbffiles).first
#stream = entry.get_input_stream.read
puts #stream
end
end
to this:
def parse
require 'zip'
zf = Zip::File.open("tmp/#{#nome}")
srcmbffiles = File.join("**", "src", "**", "*.mbf")
entry = zf.glob(srcmbffiles).first
#stream = zf.read(entry)
puts #stream
zf.close()
end
Notice that I changed the way I populate #stream because apparently entry.get_input_stream also locks the file you're accessing.
The writing process may be still locking the file. You may have to wait until that process is complete.
'"tmp/#{#name}"' is not right path. Just use 'Rails.root.join('tmp', #name)'

File upload to box.com without writing to file system

I am trying to upload a file to Box.com using its API REST call and the httmultiparty gem. The code is working and uploads to Box.com but does that after writing the uploaded file to the server file system as in f.write(data.read) then capturing the file path for the written file as the input parameter to the Box.com API REST call as in :filename => File.new(path). The app will be running on Heroku, so we can't save any files (read only) on Heroku's server so I would like to directly upload the file to Box.com while bypassing the writing of the file on the server but can't figure that out given that the Box.com REST call requires an object of type "File". Any help is appreciated. Thanks.
The model and view code is:
###
#The Model
###
class BoxUploader
require 'httmultiparty'
include HTTMultiParty
#base_uri 'https://api.box.com/2.0'
end
class File < ActiveRecord::Base
attr_accessible :file
attr_accessor :boxResponse
FILE_STORE = File.join Rails.root, 'public', 'files'
API_KEY = #myBoxApiKey
AUTH_TOKEN = #myBoxAuthToken
def file=(data) #uploaded file
filename = data.original_filename
path = File.join FILE_STORE, filename
#### would like to bypass the file writing step
File.open(path, "wb") do |f|
f.write(data.read)
end
#############
File.open(path, "wb") do |f|
boxResponse = BoxUploader.post('https://api.box.com/2.0/files/content',
:headers => { 'authorization' => 'BoxAuth api_key={API_KEY&auth_token=AUTH_TOKEN' },
:body => { :folder_id => '911', :filename => File.new(path)}
)
end
end
###
# The View
###
<!-- Invoke the Controller's "create" action -->
<h1>File Upload</h1>
<%= form_for #file, :html => {:multipart=>true} do |f| %>
<p>
<%= f.label :file %>
<%= f.file_field :file %>
</p>
<p>
<%= f.submit 'Create' %>
<% end %>
To upload a file from memory with HTTMultiParty, you need to supply it with an UploadIO object in place of the File object you'd normally give it. The UploadIO object can be populated using StringIO. It seems HTTMultiParty handles UploadIO objects in a special way, so you can't use the StringIO directly:
class Uploader
include HTTMultiParty
base_uri "http://foo.com"
end
string_io = StringIO.new('some stuff that pretends to be in a file')
upload_io = UploadIO.new(string_io, 'text/plain', 'bar.txt')
Uploader.post("/some/path", query: {file: upload_io})
You are aiming at a non-common use pattern, so your best shot could be to extend the existant gem, to provide the functionality you need.
There is a gem ruby-box to use with Box service at the 2.0 version of their API.
The gem is well supported and pretty easy to use.
You'll need to dig on the source code and create a new upload method.

Reading in file contents rails

I have a form that is attempting to read in a JSON file for parsing/actions/etc. I'm having problems getting it to read in the controller.
View:
<%= form_tag({:controller => :admins, :action => :upload_json}, {:multipart => true, :method => :post}) do |f| %>
<%= file_field_tag 'datafile' %>
<%= submit_tag "Upload" %>
Controller:
def upload_json
file_data = params[:datafile]
File.read(file_data) do |file|
file.each do |line|
## does stuff here....
end
end
end
A similar function works in my seed.rb file when I'm seeding data - just can't get it to read in an uploaded file.
The error I'm getting is: can't convert ActionDispatch::Http::UploadedFile into String.
Thanks in advance for the help!
Figured it out. Needed to change:
file_data = params[:datafile]
to
file_data = params[:datafile].tempfile
And decided to use the .open function to change:
File.read(file_data) do |file|
to
File.open(file_data, 'r') do |file|
params[:datafile] is an instance of ActionDispatch::Http::UploadedFile class with tempfile attached with that.To open the tempfile
You try something like
File.open(params[:datafile].path) do |file|
#your stuff goes here
end
Open the uploaded file using path.
params[:datafile] is an instance of ActionDispatch::Http::UploadedFile class and you'll need to get at the stored file by calling path to properly process it.
Additionally, File.read will not get you the line-by-line processing you're looking for. You need to change that to File.open.
Try this:
Controller
def upload_json
uploaded_datafile = params[:datafile]
File.open( uploaded_datafile.path ) do |file|
file.each_line do |line|
# Do something with each line.
end
end
end
Alternative Style
def upload_json
File.foreach( params[:datafile].path ) do |line|
# Do something with each line.
end
# FYI: The above method block returns `nil` when everything goes okay.
end

file upload without activerecord

How do you handle file upload in rail without attaching them to active record ?
I just want to write the files to the disk.
Thanks,
If I understand correctly what you need then the most simple example would be this:
The controller:
class UploadController < ApplicationController
def new
end
def create
name = params[:upload][:file].original_filename
path = File.join("public", "images", "upload", name)
File.open(path, "wb") { |f| f.write(params[:upload][:file].read) }
flash[:notice] = "File uploaded"
redirect_to "/upload/new"
end
end
The view:
<% flash.each do |key, msg| %>
<%= content_tag :div, msg, :class => [key, " message"], :id => "notice_#{key}" %>
<% end %>
<% form_tag '/upload/create', { :multipart => true } do %>
<p>
<%= file_field_tag 'upload[file]' %>
</p>
<p>
<%= submit_tag "Upload" %>
</p>
<% end %>
This would let you upload any file without any checks or validations which in my opinion isn't that usefull.
If I would do it myself then I would use something like validatable gem or tableless gem just tableless is not supported anymore. These gems would allow you to validate what you're uploading to make it more sane.
You can just move the temporary file to destiny path using FileUtils
tmp = params[:my_file_field].tempfile
destiny_file = File.join('public', 'uploads', params[:my_file_field].original_filename)
FileUtils.move tmp.path, destiny_file
The Tempfile documentation shows an example that's equivalent to Rytis's code, which is fine most of the time. But when you call tempfile.read, Ruby is reading the whole file as a single chunk into memory, which is sub-optimal.
However, FileUtils provides a copy_stream method, and IO, at least in Ruby 2.0, provides a copy_stream implementation that handles writing directly to a filepath (FileUtils.copy_stream requires File-like objects on both sides, or so say the docs).
In my case, I was initiating a large multi-file upload via AJAX, and wanted to avoid reading the whole file(s) into Ruby's memory before writing to disk.
In the example below, params[:files] is an Array of ActionDispatch::Http::UploadedFile instances, and local_filepath is a string pointing to a non-existing file in an existing directory. For brevity, I'll assume I'm only uploading one file:
IO.copy_stream(params[:files][0].tempfile, local_filepath)
The ActionDispatch::Http::UploadedFile instance has a .tempfile field that's just a regular Tempfile instance.
I'm not actually sure that Ruby still isn't reading the whole file into memory—I didn't benchmark anything—but it's a lot more possible than it is with the localfile.write(tempfile.read) syntax.
tl;dr: IO.copy_stream(your_tempfile, your_disk_filepath) is more concise, if not faster.
You could try using the Rails plugin Attachment_fu to handle file uploads. It allows you to save uploads to the file system instead of the database.

Resources