I'm giving user opportunity to upload their files. However, application is not saving those files into the database, it only needs to get informations out of it.
So from a form which looks like this:
= simple_form_for #channel, method: :post do |f|
= f.input :name
= f.input :configuration_file, as: :file
= f.submit
come params[:channel][:configuration_file]
#<ActionDispatch::Http::UploadedFile:0xc2af27c #original_filename="485.csv", #content_type="text/csv", #headers="Content-Disposition: form-data; name=\"channel[configuration_file]\"; filename=\"485.csv\"\r\nContent-Type: text/csv\r\n", #tempfile=#<File:/tmp/RackMultipart20140822-6972-19sqxq2>>
How exactly can i read from this thing? I tried simply
File.open(params[:channel][:configuration_file])
but it returns error
!! #<TypeError: can't convert ActionDispatch::Http::UploadedFile into String>
PS
Additional solutions for xml and csv would be much appreciated!
According to the Rails docs:
http://api.rubyonrails.org/classes/ActionDispatch/Http/UploadedFile.html
an uploaded file supports the following instance methods, among others:
open()
path()
read(length=nil, buffer=nil)
you could try:
my_data = params[:channel][:configuration_file].read
to get a string of the file contents?
or even:
my_data = File.read params[:channel][:configuration_file].path
Also, if the file can be long, you may want to open the file and read line by line. A few solutions here:
How to read lines of a file in Ruby
If you want to read a CSV file, you could try:
require 'csv'
CSV.foreach(params[:channel][:configuration_file].path, :headers => true) do |row|
row_hash = row.to_hash
# Do something with the CSV data
end
Assuming you have headers in your CSV of course.
For XML I recommend the excellent Nokogiri gem:
http://nokogiri.org/
At least partly because it uses an efficient C library for navigating the XML. (This can be a problem if you're using JRuby). Its use is probably out of scope of this answer and adequately explained in the Nokogiri docs.
From the documentation
The actual file is accessible via the tempfile accessor, though some
of its interface is available directly for convenience.
You can change your code to:
file_content = params[:channel][:configuration_file].read
or if you want to use the File API:
file_content = File.read params[:channel][:configuration_file].path
Related
I'm trying to make a request to the PDFTables API, and save what is returned (an xml doc) in a new file. I have this code:
result = RestClient.post "https://pdftables.com/api?key=nn123450hsn", :myfile => File.new("./lib/assets/PeterValleyHexacoResults.pdf", "rb")
File.open('./lib/assets/test.xml', "w") do |f|
f.puts result
end`
When I view the newly saved file, it looks like a bunch of random symbols and characters in the editor. I'm not entirely sure what I'm doing wrong. Any help is appreciated.
You are getting the result in XLSX format. You need to specify XML in your request:
result = RestClient.post "https://pdftables.com/api?key=nn123450hsn&format=xml", :myfile => File.new("./lib/assets/PeterValleyHexacoResults.pdf", "rb")
This question already has answers here:
How do you access the raw content of a file uploaded with Paperclip / Ruby on Rails?
(7 answers)
Closed 5 years ago.
I have paperclip running to upload and store files in a non-public directory on the server. Now I want to be able to read the files directly and or feed them into a gem such as axlsx. I'm struggling with even simply looping threw a text file and think I'm missing something basic (as is usually the case).
Here is my first attempt at opening the file:
Paperclip config in application.rb:
config.paperclip_defaults = {:storage => :fog, :fog_credentials => {:provider => "Local", :local_root => "#{Rails.root}/secured_storage"}, :fog_directory => "", :fog_host => "localhost"}
Model:
class Census < ActiveRecord::Base
has_attached_file :censusfile
validates_attachment_content_type :censusfile,
:content_type => ["application/octet-stream", "text/plain"]
end
In the Controller:
def processcensus
#census=Census.find(params[:id])
#file=#census.censusfile.path
end
In the View:
<% File.readlines(#file).read do |line| %>
<%= line %>
<% end %>
This fails because the 'path' returned by Paperclip is the path relative to its sotrage path, not the full path.
UPDATE: If I add the directory (in this case "secured_storage" in from of the path it work as expected. For example:
#file="secured_storage/" + #census.censusfile.path
Not sure if this is at all the way to address this. If it is, is there a way to ask Paperclip where it is storing the files??
I've read where I could use:
Paperclip.io_adapters.for(#census.censusfile).path
But that seems to read the file into an array unless I'm missing something completely. My goal is to be able to loop threw a text file as well as feed an Excel file to axlsx for processing. I would also like to be able to eventually feed these files directly to a user somehow to allow for secure downloads.
I have looked hard for some documentation on all this and haven't found anything that really explains it yet. I'm to that point of just randomly throwing code here or there and hoping something works, which rarely does. Any help/direction that could be provided would be GREATLY appreciated!!!
Mark
I think io adapter can support read
Paperclip.io_adapters.for(#census.censusfile).read
so
<% Paperclip.io_adapters.for(#census.censusfile).read %>
<%= line %>
<% end %>
Use the copy_to_local_file method. This returns a file object on which you can read like on a normal file`.
For the life of my I can't understand how the basic paperclip example works. There's only one line included in the controller, and that's
#user = User.create( params[:user] )
I simply don't understand how that's all that is needed to upload an image to s3. I've changed the example quite a bit because i wanted to use jquery file uploader rather than the default rails form helper, so I'm at the point where an image is being POSTed to my controller, but I can't figure out how I'm supposed to take the image from the params and assign it as an attachment. Here's what I'm seeing the logs:
Parameters: {"files"=>[#<ActionDispatch::Http::UploadedFile:0x132263b98 #tempfile=#<File:/var/folders/5d/6r3qnvmx0754lr5t13_y1vd80000gn/T/RackMultipart20120329-71039-1b1ewde-0>, #headers="Content-Disposition: form-data; name=\"files[]\"; filename=\"background.png\"\r\nContent-Type: image/png\r\n", #content_type="image/png", #original_filename="background.png">], "id"=>"385"}
My JS is very simple:
` $('#fileupload').fileupload({
dataType: 'json',
url: '/my_url',
done: function (e, data) {
console.log('done');
}
});`
What would be helpful for me to know is how I can strip the file data from the POSTed parameters given above and pass it to paperclip. I'm sure that I'll have to assign the attachment attribute a value of File.open(...), but I dont know what source of my file is.
I've spent a ridiculous amount of time trying to figure this out and I can't seem to get it. I've tried uploading directly to s3, but the chain of events was terribly confusing, so I want to get this simple pass-through example completed first. Thanks so much for any help you cna give!
You need a few more pieces and it will help if you can show the exact code you're using.
Paperclip can post to S3 by using:
http://rubydoc.info/gems/paperclip/Paperclip/Storage/S3
When your controller creates a User model, it is sending along all the params. This is called "mass assignment" (be sure to read about attr_accessible).
When your model receives the params, it uses the Paperclip AWS processor, which uploads it.
You need the AWS gem, a valid bucket on S3, and a config file.
Try this blog post and let us know if it helps you:
http://blog.trydionel.com/2009/11/08/using-paperclip-with-amazon-s3/
UPDATE 2013-04-03: Pleases see Chloe's comment below-- you may need an additional parameter, and the blog post may be outdated.
If you want to do it manually, approach it like this:
# In order to get contents of the POST request with the photo,
# you need to read contents of request
upload = params[:file].is_a(String)
file_name = upload ? params[:file] : params[:file].original_filename
extension = file_name.split('.').last
# We have to create a temp file which is going to be used by Paperclip for
# its upload
tmp_file = "#{Rails.root}/tmp/file.#{extension}"
file_id = 0
# Check if file with the name exists and generate unique path
while File.exists?(tmp_file) do
tmp_file_path = "#{Rails.root}/tmp/file#{file_id}.#{extension}"
id += 1
end
# Let's write the file from post request to unique location
File.open(tmp_file_path, 'wb') do |f|
if upload
f.write request.body.read
else
f.write params[:file].read
end
end
# Now that file is saved in temp location, we can use Paperclip to mimic one file
# upload
#photo = Photo.new :photo => File.open(tmp_file_path)
# We'll return javascript to say that the file is uploaded and put its thumbnail in
# HTML or whatever else you wanted to do with it
respond_to do |format|
if #photo.save
render :text => "Success"
else
render :text => #photo.errors
end
end
You can rewrite your create or whatever you use as the url to which you are POSTing the form.
This bit:
"files"=>[#<ActionDispatch::Http::UploadedFile:0x132263b98 #tempfile=# <File:/var/folders/5d/6r3qnvmx0754lr5t13_y1vd80000gn/T/RackMultipart20120329-71039-1b1ewde-0>
is the part (I think) that holds the file contents that are posted in the form.
In Rails, the User model will have a helper: has_attached_file
Passing the [:params] to the User.create method allows that helper to pick up the file contents, do any processing on them (eg resizing etc based on attributes supplied to the helper) and then push the image(s) to your storage (eg S3 or whatever - S3 credentials are passed to the helper).
Hopefully that explains the 'how does it do it?' question
re the jQuery bit.. not sure what the code should be there, but why not use the Rails form with :remote => true and handle the response in jquery?
I may end up figuring this out later, but i thought I'd try.
Can someone help combine send_data and CSV.open
According to the docs, you can CSV.open filename, mode(whatever that is) and basically a file will save to your current path. However, if you want to send that file to a user through his broswer, as most of us who give the option of CSV files to download, do,do... then can we combine the CSV.open with send_data?
thoughts?
examples welcome if you do something like this too.
I don't think you want to combine those two things.
CSV.open will save the data to a file, which you would need to read back in in order to send it through send_data.
But you can do something like:
csv = []
csv << ["titles", "for", "csv"]
csv << ["data", "for", "csv"]
send_data(csv.collect{|s| s.join(",")}.join("\n"),
:type => 'text/csv; charset=utf-8; header=present',
:filename => "mytitle.csv")
Which should prompt the user to download the csv file.
How do I parse a document in a table and send it across as a JSON file to another db.
Detailed Desc:
I have crawled and taken data into a table from websites using anemone. I need to now parse it and transfer it as a JSON file to another server. I think, I will have to first convert the document in the table into nokogiri document which can be parsed and converted to json file. Any idea how can I convert the doc into nokogiri document or if anyone has any other idea to parse it and send it as a json file ?
Nokogiri is your best bet for the HTML parsing, but as for converting it to JSON you're on your own from what I can tell.
Once you have it parsed via Nokogiri it shouldn't be terribly hard to extract the elements you need and generate JSON that represents them. What you're doing isn't a very common task, so you'll have to bridge the gap between Nokogiri and whichever gem you're using to generate the JSON.
Okay I found the answer long time back, I basically made use of REST to send message form one application to another, i sent it across as a hash. And the obvious one, I used nokogiri for parsing the table.
def post_me
#page_hash = page_to_hash
res = Net::HTTP.post_form(URI.parse('http://127.0.0.1:3007/element_data/save.json'),#page_hash)
end
For sending the hash from one application to another using net/http.
def page_to_hash
require 'rubygems'
require 'nokogiri'
require 'open-uri'
require 'domainatrix'
#page = self.page.sub(/^<!DOCTYPE html(.*)$/, '<!DOCTYPE html>')
hash={}
doc = Nokogiri::HTML(self.page)
doc.search('*').each do |n|
puts n.name
end
Using Nokogiri for parsing the page table in my model. page table had the whole body of a webpage.
file_type = []
file_type_data=doc.xpath('//a/#href[contains(. , ".pdf") or contains(. , ".doc")
or contains(. , ".xls") or contains(. , ".cvs") or contains(. , ".txt")]')
file_type_data.each do |href|
if href[1] == "/"
href = "http://" + website_url + href
end
file_type << href
end
file_type_str = file_type.join(",")
hash ={:head => head,:title => title, :body => self.body,
:image => images_str, :file_type => file_type_str, :paragraph => para_str, :description => descr_str,:keyword => key_str,
:page_url=> self.url, :website_id=>self.parent_request_id, :website_url => website_url,
:depth => self.depth, :int_links => #int_links_arr, :ext_links => #ext_links_arr
}
A simple parsing example and how i formed my hash.