I have working Paperclip gem in my app for documents (pdf, doc). I need to pass the document to some other third party application via post request.
I tried to convert the paperclip attachment via Base64 but it throws error:
no implicit conversion of Tempfile into String
Here is how I did it:
# get url from the paperclip file
url = document.doc.url # https://s3-ap-southeast-1.amazonaws.com/xx-eng/documents/xx/000/000/xx/original/doc.pdf
file_data = open(url)
# Encode the bytes to base64 - this line throw error
base_64_file = Base64.encode64(file_data)
Do you have any suggestion how to avoid the Tempfile error?
You need to read file first.
base_64_file = Base64.encode64(file_data.read)
Here is working example:
$ bundle exec rails c
=> file = open("tmp/file.pdf")
#> #<File:tmp/receipts.pdf>
=> base_64 = Base64.encode64(file)
#> TypeError: no implicit conversion of File into String
=> base_64 = Base64.encode64(file.read)
#> "JVBERi0xLjQKMSAwIG9iago8PAovVGl0b/BBQEPgQ ......J0ZgozMDM0OQolJUVPRgo=\n"
The answer from #3елёный didn't work to me - maybe because it's the S3 file.
However I managed to find a way with Paperclip method:
file_data = Paperclip.io_adapters.for(url).read
base_64_file = Base64.encode64(file_data)
Related
I am using Libreconv gem to convert word to doc but it's not working with S3
bucket = Aws::S3::Bucket.new('bucket-name')
object = bucket.object file.attachment.blob.key
path = object.presigned_url(:get)
Libreconv.convert(path, "public/test.pdf")
If I try to convert this path to PDF using Libreconv then it's give me filename too long error. I have wrriten this code under ActiveJobs. So kindly provide me solutions as per ActiveJobs.
Can someone please suggest me how can I convert word file to pdf.
Here path is https://domain.s3.amazonaws.com/Bf5qPUP3znZGCHCcTWHcR5Nn?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIZ6RZ7J425ORVUYQ%2F20181206%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20181206T051240Z&X-Amz-Expires=900&X-Amz-SignedHeaders=host&X-Amz-Signature=b89c47a324b2aa423bf64dfb343e3b3c90dce9b54fa9fe1bc4efa9c248e912f9
and error I am getting is
Error: source file could not be loaded
*** Errno::ENAMETOOLONG Exception: File name too long # rb_sysopen - /tmp/Bf5qPUP3znZGCHCcTWHcR5Nn?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIZ6RZ7J425ORVUYQ%2F20181206%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20181206T051240Z&X-Amz-Expires=900&X-Amz-SignedHeaders=host&X-Amz-Signature=b89c47a324b2aa423bf64dfb343e3b3c90dce9b54fa9fe1bc4efa9c248e912f9.pd
It seems that you PDF is created with all the params needed to fetch docx from S3.
I suppose it happens in this line:
target_tmp_file = "#{target_path}/#{File.basename(#source, ".*")}.#{File.basename(#convert_to, ":*")}"
#source is https://domain.s3.amazonaws.com/Bf5qPUP3znZGCHCcTWHcR5Nn?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIZ6RZ7J425ORVUYQ%2F20181206%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20181206T051240Z&X-Amz-Expires=900&X-Amz-SignedHeaders=host&X-Amz-Signature=b89c47a324b2aa423bf64dfb343e3b3c90dce9b54fa9fe1bc4efa9c248e912f9 and
> File.basename(#source, ".*")
=> "Bf5qPUP3znZGCHCcTWHcR5Nn?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIZ6RZ7J425ORVUYQ%2F20181206%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20181206T051240Z&X-Amz-Expires=900&X-Amz-SignedHeaders=host&X-Amz-Signature=b89c47a324b2aa423bf64dfb343e3b3c90dce9b54fa9fe1bc4efa9c248e912f9"
As a result Libreconv gem tries to create a tmp file with this long name and it's too long - that's why an error is raised.
Possible solution: split the process into separate steps of fetching file and converting it. Something like:
require "open-uri"
bucket = Aws::S3::Bucket.new('bucket-name')
object = bucket.object file.attachment.blob.key
path = object.presigned_url(:get)
doc_file = open(path)
begin
Libreconv.convert(doc_file.path, "public/test.pdf")
ensure
doc_file.delete
end
following is the answer using combine pdf gem
tape = Tape.new(file)
result = tape.preview
tempfile = Tempfile.new(['foo', '.pdf'])
File.open(tempfile, 'wb') do |f|
f.write result
end
path = tempfile.path
combine_pdf(path)
and for load file for S3 I have used
object = #bucket.object object_key
path = object.presigned_url(:get)
response = Net::HTTP.get_response(URI.parse(path)).body
I'm having trouble calling image.write with MiniMagick on a decoded base64 image in Rails. Every line seems to be working properly except for image.write. The code below is in my Rails API ImageController, which my React frontend is hitting through a POST request with the encoded image.
def create
uploaded_io = params["image_io"]["base64"] # base64 string + metadata
metadata = uploaded_io.split(',/')[0] + "," # "data:image/jpeg;base64,"
filetype = metadata.split("/")[1].split("base64")[0][0...-1] # "jpeg"
base64_string = uploaded_io[metadata.size..-1] # base64 string w/o metadata
blob = Base64.decode64(base64_string)
image = MiniMagick::Image.read(blob)
image.write `#{Time.new.to_i}.#{filetype}`
storage = Google::Cloud::Storage.new(
project_id: ENV['GOOGLE_CLOUD_PROJECT'],
credentials: JSON.parse(File.read('config/google_cloud_credentials.json'))
)
bucket = storage.bucket "auto-stock-189103.appspot.com"
bucket.create_file image,`test/#{Time.new.to_i}.jpg`
end
I added comments to the first few lines in the code describing their value. base64_string was too long to comment, so here is its value:
"/9j/4QAYRXhpZgAASUkqAAgAAAAAAAAAAAAAAP/sABFEdWNreQABAAQAAABkAAD/4QMtaHR0cDovL25zLmFkb2JlLmNvbS94YXAvMS4wLwA8P3hwYWNrZXQgYmVnaW49Iu+7vyIgaWQ9Ilc1TTBNcENlaGlIenJlU3pOVGN6a2M5ZCI/PiA8eDp4bXBtZXRhIHhtbG5zOng9ImFkb2JlOm5zOm1ldGEvIiB4OnhtcHRrPSJBZG9iZSBYTVAgQ29yZSA1LjMtYzAxMSA2Ni4xNDU2NjEsIDIwMTIvMDIvMDYtMTQ6NTY6MjcgICAgICAgICI+IDxyZGY6UkRGIHhtbG5zOnJkZj0iaHR0cDovL3d3dy53My5vcmcvMTk5OS8wMi8yMi1yZGYtc3ludGF4LW5zIyI+IDxyZGY6RGVzY3JpcHRpb24gcmRmOmFib3V0PSIiIHhtbG5zOnhtcD0iaHR0cDovL25zLmFkb2JlLmNvbS94YXAvMS4wLyIgeG1sbnM6eG1wTU09Imh0dHA6Ly9ucy5hZG9iZS5jb20veGFwLzEuMC9tbS8iIHhtbG5zOnN0UmVmPSJodHRwOi8vbnMuYWRvYmUuY29tL3hhcC8xLjAvc1R5cGUvUmVzb3VyY2VSZWYjIiB4bXA6Q3JlYXRvclRvb2w9IkFkb2JlIFBob3Rvc2hvcCBDUzYgKE1hY2ludG9zaCkiIHhtcE1NOkluc3RhbmNlSUQ9InhtcC5paWQ6MTNDQjQyRjlEQTAxMTFFN0E4N0VBNzdGODEwREFGMTYiIHhtcE1NOkRvY3VtZW50SUQ9InhtcC5kaWQ6MTNDQjQyRkFEQTAxMTFFN0E4N0VBNzdGODEwREFGMTYiPiA8eG1wTU06RGVyaXZlZEZyb20gc3RSZWY6aW5zdGFuY2VJRD0ieG1wLmlpZDoxM0NCNDJGN0RBMDExMUU3QTg3RUE3N0Y4MTBEQUYxNiIgc3RSZWY6ZG9jdW1lbnRJRD0ieG1wLmRpZDoxM0NCNDJGOERBMDExMUU3QTg3RUE3N0Y4MTBEQUYxNiIvPiA8L3JkZjpEZXNjcmlwdGlvbj4gPC9yZGY6UkRGPiA8L3g6eG1wbWV0YT4gPD94cGFja2V0IGVuZD0iciI/Pv/uAA5BZG9iZQBkwAAAAAH/2wCEAAEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQECAgICAgICAgICAgMDAwMDAwMDAwMBAQEBAQEBAgEBAgICAQICAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDA//AABEIAAoACgMBEQACEQEDEQH/xABNAAEBAAAAAAAAAAAAAAAAAAAACQEBAQEAAAAAAAAAAAAAAAAAAAkKEAEAAAAAAAAAAAAAAAAAAAAAEQEAAAAAAAAAAAAAAAAAAAAA/9oADAMBAAIRAxEAPwCMKU7f4AAA/9k="
Testing it here renders the correct image (a red square), but when I run the image.write line it returns the following error:
bin/rails: No such file or directory - 1513397345.jpeg
*** NoMethodError Exception: undefined method `write' for nil:NilClass
nil
Here's the return value of image = MiniMagick::Image.read(blob) for reference:
#<MiniMagick::Image:0x00007f9ba76ba1f8 #path="/var/folders/pf/xhvv11092_j08hw47q6rt9z80000gn/T/mini_magick20171215-26353-l2lcyu", #tempfile=#<Tempfile:/var/folders/pf/xhvv11092_j08hw47q6rt9z80000gn/T/mini_magick20171215-26353-l2lcyu (closed)>, #info=#<MiniMagick::Image::Info:0x00007f9ba76ba1d0 #path="/var/folders/pf/xhvv11092_j08hw47q6rt9z80000gn/T/mini_magick20171215-26353-l2lcyu", #info={}>>
Ultimately, my goal is to upload the image to Google Cloud so please let me know if there's a better way to go about this. I'm following this answer from a similar question, which is why I have it structured this way.
I think your problem is that you're using backticks where you mean to use double quotes:
image.write `#{Time.new.to_i}.#{filetype}`
# ----------^----------------------------^
Backticks will attempt to execute their contents in the shell. You don't have an executable file named 1513397345.jpeg (which is what #{Time.new.to_i}.#{filetype} evaluates to) so you get an error.
You just want to use plain old double quotes to get the string interpolation you're expecting:
image.write "#{Time.new.to_i}.#{filetype}"
and again a few lines below that:
bucket.create_file image, "test/#{Time.new.to_i}.jpg"
Furthermore, you probably want to store that filename in a variable because Time.new.to_i isn't guaranteed to be the same in both invocation:
name = "#{Time.new.to_i}.#{filetype}
image.write name
#...
bucket.create_file image, name
I want to generate pdf file from DB record. Encode it to Base64 string and store it to DB. Which works fine. Now I want reverse action, How can I decode Base64 string and generate pdf file again?
here is what I tried so far.
def data_pdf_base64
begin
# Create Prawn Object
my_pdf = Prawn::Document.new
# write text to pdf
my_pdf.text("Hello Gagan, How are you?")
# Save at tmp folder as pdf file
my_pdf.render_file("#{Rails.root}/tmp/pdf/gagan.pdf")
# Read pdf file and encode to Base64
encoded_string = Base64.encode64(File.open("#{Rails.root}/tmp/pdf/gagan.pdf"){|i| i.read})
# Delete generated pdf file from tmp folder
File.delete("#{Rails.root}/tmp/pdf/gagan.pdf") if File.exist?("#{Rails.root}/tmp/pdf/gagan.pdf")
# Now converting Base64 to pdf again
pdf = Prawn::Document.new
# I have used ttf font because it was giving me below error
# Your document includes text that's not compatible with the Windows-1252 character set. If you need full UTF-8 support, use TTF fonts instead of PDF's built-in fonts.
pdf.font Rails.root.join("app/assets/fonts/fontawesome-webfont.ttf")
pdf.text Base64.decode64 encoded_string
pdf.render_file("#{Rails.root}/tmp/pdf/gagan2.pdf")
rescue => e
return render :text => "Error: #{e}"
end
end
Now I am getting below error:
Encoding ASCII-8BIT can not be transparently converted to UTF-8.
Please ensure the encoding of the string you are attempting to use is
set correctly
I have tried How to convert base64 string to PNG using Prawn without saving on server in Rails but it gives me error:
"\xFF" from ASCII-8BIT to UTF-8
Can anyone point me what I am missing?
The answer is to decode the Base64 encoded string and either send it directly or save it directly to disk (naming it as a PDF file, but without using prawn).
The decoded string is a binary representation of the PDF file data, so there's no need to use Prawn or to re-calculate the content of the PDF data.
i.e.
raw_pdf_str = Base64.decode64 encoded_string
render :text, raw_pdf_str # <= this isn't the correct rendering pattern, but it's good enough as an example.
EDIT
To clarify some of the information given in the comments:
It's possible to send the string as an attachment without saving it to disk, either using render text: raw_pdf_str or the #send_data method (these are 4.x API versions, I don't remember the 5.x API style).
It's possible to encode the string (from the Prawn object) without saving the rendered PDF data to a file (save it to a String object instead). i.e.:
encoded_string = Base64.encode64(my_pdf.render)
The String data could be used directly as an email attachment, similarly to the pattern provided here only using the String directly instead of reading any data from a file. i.e.:
# inside a method in the Mailer class
attachments['my_pdf.pdf'] = { :mime_type => 'application/pdf',
:content => raw_pdf_str }
I get image files sent from an Android app to my Rails API. I decode the images using this:
StringIO.new(Base64.decode64(image[1]))
The issue is that it takes too much time; on heroku it takes even longer.
Is there another way to do this that's faster and more efficient?
You can also use this for decode base64:
# this method for decode base64 code to file
def parse_image_data(image[1])
base64_file = image[1]
ext, string = base64_file.split(',')
ext = MIME::Types[base64_file].first.preferred_extension if ext.include?("base64")
tempfile = Tempfile.new(["#{DateTime.now.to_i}", ".#{ext}"])
tempfile.binmode
tempfile.write Base64.decode64(string)
tempfile.rewind
tempfile
end
The code below resizes my image. But I am not sure how to write it out to a temp file or blob so I can upload it to s3.
origImage = MiniMagick::Image.open(myPhoto.tempfile.path)
origImage.resize "200x200"
thumbKey = "tiny-#{key}"
obj = bucket.objects[thumbKey].write(:file => origImage.write("tiny.jpg"))
I can upload the original file just fine to s3 with the below command:
obj = bucket.objects[key].write('data')
obj.write(:file => myPhoto.tempfile)
I think I want to create a temp file, read the image file into it and upload that:
thumbFile = Tempfile.new('temp')
thumbFile.write(origImage.read)
obj = bucket.objects[thumbKey].write(:file => thumbFile)
but the origImage class doesn't have a read command.
UPDATE: I was reading the source code and found this out about the write command
# Writes the temporary file out to either a file location (by passing in a String) or by
# passing in a Stream that you can #write(chunk) to repeatedly
#
# #param output_to [IOStream, String] Some kind of stream object that needs to be read or a file path as a String
# #return [IOStream, Boolean] If you pass in a file location [String] then you get a success boolean. If its a stream, you get it back.
# Writes the temporary image that we are using for processing to the output path
And the s3 api docs say you can stream the content using a code block like:
obj.write do |buffer, bytes|
# writing fewer than the requested number of bytes to the buffer
# will cause write to stop yielding to the block
end
How do I change my code so
origImage.write(s3stream here)
http://docs.aws.amazon.com/AWSRubySDK/latest/AWS/S3/S3Object.html
UPDATE 2
This code successfully uploads the thumbnail file to s3. But I would still love to know how to stream it up. It would be much more efficient I think.
#resize image and upload a thumbnail
smallImage = MiniMagick::Image.open(myPhoto.tempfile.path)
smallImage.resize "200x200"
thumbKey = "tiny-#{key}"
newFile = Tempfile.new("tempimage")
smallImage.write(newFile.path)
obj = bucket.objects[thumbKey].write('data')
obj.write(:file => newFile)
smallImage.to_blob ?
below code copy from https://github.com/probablycorey/mini_magick/blob/master/lib/mini_magick.rb
# Gives you raw image data back
# #return [String] binary string
def to_blob
f = File.new #path
f.binmode
f.read
ensure
f.close if f
end
Have you looked into the paperclip gem? The gem offers direct compatibility to s3 and works great.