I've been trying to manipulate a file that's base64 encoded that I'm recieving from my client.
I'm currently using https://github.com/zdavatz/spreadsheet/blob/master/GUIDE.md to manipulate it, however, there doesn't appear to be any way to open a file directly from the base64 blob, or should I write it and then read from it? can't that a potential security threat for the server?
for example, if I recieve a file :
file = params[:file] with contents:
data:application/vnd.ms-excel;base64,0M8R4KGxGuEAAAAAAAAAAAAAAAAAAAAAOwADAP7
(should I remove the data:application/vnd.ms-excel;base64, ?)
I'd like to open it with this:
Spreadsheet.client_encoding = 'UTF-8'
book = Spreadsheet.open "#{Rails.root}/app/assets/spreadsheet/event.xls"
(or with a blob or temp fle)
Sorry if it's pretty obvious, been looking for hours and there's not much info about it available, tried creating a temp file first but I don't think that's supported and there's not much I can get from the docs.
Shot in the dark: Maybe decode it, write to binary-enabled tempfile, and then feed that to Spreadsheet?
tmpfile = Tempfile.new.binmode
tmpfile << Base64.decode64(params[:file])
tmpfile.rewind
book = Spreadsheet.open(tmpfile)
Related
In my Rails app, I managed to stream large CSV files directly from Postgres based on solutions mentioned in this SO post. My working code looks somewhat like so:
query = <A Long SQL Query String>
response.headers["Cache-Control"] = "no-cache"
response.headers["Content-Type"] = "text/csv; charset=utf-8"
response.headers["Content-Disposition"] =
%(attachment; filename="#{csv_filename}")
response.headers["Last-Modified"] = Time.now.ctime.to_s
conn = ActiveRecord::Base.connection.raw_connection
conn.copy_data("COPY (#{query}) TO STDOUT WITH (FORMAT CSV, HEADER TRUE, FORCE_QUOTE *, ESCAPE E'\\\\');") do
while row = conn.get_copy_data
response.stream.write row
end
end
response.stream.close
end
Some of the columns (VARCHAR) being queried have values as either English or Chinese strings. The CSV file resulting from the above code doesn’t show the Chinese characters as is. Instead, I get something like this:
大大 文文
Am I supposed to change the way I’m using the copy_data function, or is there something I could do to the CSV file to solve this? I’ve tried saving the file as UTF-8 .txt file, as well as trying the convert_to function mentioned in the copy_data documentation, but to no avail.
This depends of the original encoding included in the CSV file.
Do this on Linux :
file -i you_file
Are you sure it's not UTF-16 or GB 18030 ?
And also in what kind of encoding is setup your database ?
do a \l in psql to see this.
So it boiled down to my MS Excel not being able to render the Chinese chars correctly. On MacOS, opening the same .csv file using the Numbers app (or even Atom, for that matter) resolved this issue for me.
I'd like to create a file object from an image located at a specific url. I'm downloading the file with Net Http:
img = Net::HTTP.get_response(URI.parse('https://prium-solutions.com/wp-content/uploads/2016/11/rails-1.png'))
file = File.read(img.body)
However, I get ArgumentError: string contains null byte when trying to read the file and store in into the file variable.
How can I do this without having to store it locally ?
Since File deals with reading from storage, it's really not applicable here. The read method is expecting you to hand it a location to read from, and you're passing in binary data.
If you have a situation where you need to interface with a library that expects an object that is streaming, you can wrap the string body in a StringIO object:
file = StringIO.new(img)
# you can now call file.read, file.seek, file.rewind, etc.
I get image files sent from an Android app to my Rails API. I decode the images using this:
StringIO.new(Base64.decode64(image[1]))
The issue is that it takes too much time; on heroku it takes even longer.
Is there another way to do this that's faster and more efficient?
You can also use this for decode base64:
# this method for decode base64 code to file
def parse_image_data(image[1])
base64_file = image[1]
ext, string = base64_file.split(',')
ext = MIME::Types[base64_file].first.preferred_extension if ext.include?("base64")
tempfile = Tempfile.new(["#{DateTime.now.to_i}", ".#{ext}"])
tempfile.binmode
tempfile.write Base64.decode64(string)
tempfile.rewind
tempfile
end
I have an application where an admin can upload an image. I save the image in a file and also base64 encode (using Base64.strict_encode method of ruby) & save in my DB. This is so that when later someone deleted the physical file from the HDD/Server, I can still generate it back by base64 decoding it (Base64.decode method) and save in a file.
But the encoding and decoding didn't go well as the image get damaged and I'm unable to view it after save.
I checked the output of the Base64.strict_encode against the result when I used http://www.base64-image.de/ to encode the file, they were different.
Can anyone help me with this? What am I doing wrong? What am I not doing?
ENCODING THE IMAGE DURING UPLOAD:
imageLoc = image.image.to_s
logger.info '>>>>>>' + (Base64.strict_encode64(open(imageLoc).read)).to_s
image_data = Base64.strict_encode64(File.open(imageLoc, 'rb').read)
CategoryImage.update_image_data(image.id,image_data)
DECODING WHEN IMAGE FILE IS LOST:
File.open(File.join(APP_CONFIG['image_storage_location'], image[:image]), 'wb') { |f|
content = image[:image_data]
content.gsub!('\\r', "\r")
content.gsub!('\\n', "\n")
f.write(Base64.decode64(content))
f.close
}
ENCODED IMAGE FROM THE SITE (base64-image.de): https://shrib.com/cYLKfEe1?v=nc
ENCODED IMAGE FROM MY CODE: https://shrib.com/CODE-encoded%20image?v=nc
EDIT
When I replaced the encoded image data in my DB with the one I generated from the above named website, my image was regenerated and viewable. So the real is with the encoding.
Had once a similar issue, solved it by replacing the File.read method with IO.binread(imageLoc). Hope it helps. :)
I'm having some problems reading a file from S3. I want to be able to load the ID3 tags remotely, but using open-URI doesn't work, it gives me the following error:
ruby-1.8.7-p302 > c=TagLib2::File.new(open(URI.parse("http://recordtemple.com.s3.amazonaws.com/music/745/original/The%20Stranger.mp3?1292096514")))
TypeError: can't convert Tempfile into String
from (irb):8:in `initialize'
from (irb):8:in `new'
from (irb):8
However, if i download the same file and put it on my desktop (ie no need for open-URI), it works just fine.
c=TagLib2::File.new("/Users/momofwombie/Desktop/blah.mp3")
is there something else I should be doing to read a remote file?
UPDATE: I just found this link, which may explain a little bit, but surely there must be some way to do this...
Read header data from files on remote server
Might want to check out AWS::S3, a Ruby Library for Amazon's Simple Storage Service
Do an AWS::S3:S3Object.find for the file and then an use about to retrieve the metadata
This solution assumes you have the AWS credentials and permission to access the S3 bucket that contains the files in question.
TagLib2::File.new doesn't take a file handle, which is what you are passing to it when you use open without a read.
Add on read and you'll get the contents of the URL, but TagLib2::File doesn't know what to do with that either, so you are forced to read the contents of the URL, and save it.
I also noticed you are unnecessarily complicating your use of OpenURI. You don't have to parse the URL using URI before passing it to open. Just pass the URL string.
require 'open-uri'
fname = File.basename($0) << '.' << $$.to_s
File.open(fname, 'wb') do |fo|
fo.print open("http://recordtemple.com.s3.amazonaws.com/music/745/original/The%20Stranger.mp3?1292096514").read
end
c = TagLib2::File.new(fname)
# do more processing...
File.delete(fname)
I don't have TagLib2 installed but I ran the rest of the code and the mp3 file downloaded to my disk and is playable. The File.delete would clean up afterwards, which should put you in the state you want to be in.
This solution isn't going to work much longer. Paperclip > 3.0.0 has removed to_file. I'm using S3 & Heroku. What I ended up doing was copying the file to a temporary location and parsing it from there. Here is my code:
dest = Tempfile.new(upload.spreadsheet_file_name)
dest.binmode
upload.spreadsheet.copy_to_local_file(:default_style, dest.path)
file_loc = dest.path
...
CSV.foreach(file_loc, :headers => true, :skip_blanks => true) do |row|}
This seems to work instead of open-URI:
Mp3Info.open(mp3.to_file.path) do |mp3info|
puts mp3info.tag.artist
end
Paperclip has a to_file method that downloads the file from S3.