I have a Rails 3.2 app that uses gem 'wicked_pdf', and gem 'combine_pdf'.
They both work and I can create PDFs which get emailed.
But, I have run into a situation where the email would be too big.
So, I'm trying to save the created pdf to Amazon S3. The app already has the gem 'aws-sdk'.
This is my code:
def self.saveallpdf
#costprojects = Costproject.where("client_id = 2")
pdf = CombinePDF.new
#costprojects.each do |costproject|
#costproject = costproject
controller = CostprojectsController.new
controller.instance_variable_set(:"#costproject", #costproject)
pdf2 = controller.render_to_string(pdf: "Captital Projects.pdf",
template: "costprojects/viewproject",
encoding: "UTF-8")
pdf << CombinePDF.parse(pdf2)
end
#s3 = AWS::S3.new
#bucket = #s3.buckets['ndeavor3-pdf']
#obj = #bucket.objects['filename'].write(pdf, acl: :public_read)
end
The error I'm getting is:
:data must be provided as a String, Pathname, File, or an object that responds to #read and #eof?
/app/vendor/bundle/ruby/1.9.1/gems/aws-sdk-1.8.3.1/lib/aws/s3/data_options.rb:125:in `validate_data!'
/app/vendor/bundle/ruby/1.9.1/gems/aws-sdk-1.8.3.1/lib/aws/s3/data_options.rb:32:in `compute_write_options'
/app/vendor/bundle/ruby/1.9.1/gems/aws-sdk-1.8.3.1/lib/aws/s3/s3_object.rb:594:in `write'
/app/app/models/costproject.rb:167:in `saveallpdf
'
I guess was-sdk doesn't like the "pdf" as the file??
PS - I can email the "pdf" - if it was smaller in size.
Thanks for your help!
At this point in time:
#obj = #bucket.objects['filename'].write(pdf, acl: :public_read)
pdf is a CombinePDF object and not a File, String, or Pathname
pdf.to_s might work, or you will have to create a new file from the CombinePDF object
File.new(CombinePDF) # pseduo code only
Related
I have a data backup system for customers of my app. I gather up all associated csv files and zip them. Once that zip file is complete, I attach it in an email. This process breaks on heroku due to their file system. I thought since heroku-16 we could write to the app/tmp directory and that this process might occur within the same transaction and the files would be fine, but that doesn't seem to be the case. I don't even seem to be writing the files to the tmp directory in production (in Dev I am).
So, what I would like to do instead is just write the csv files directly to S3, then Zip those files and also save the .zip to S3...then, pull that file as an email attachment. To do this, I need to generate the csv files and write them to S3 from inside ActiveJob. I use S3 already as part of ActiveStorage, but this process will not utilize ActiveStorage.
Is there's a command for me to manually direct upload to an S3 bucket. I've been digging around in the docs, etc but don't see what I'm after.
The Job (using /tmp)
def perform(company_id, recipient_id)
company = Company.find(company_id)
source_folder = "#{ Rails.root }/tmp"
zipfile_name = "company_#{ company.id }_archive.zip"
zipfile_path = "#{ Rails.root }/tmp/#{ zipfile_name }"
input_filenames = []
# USERS: create a new empty csv file,
# ... then add rows to it
# ... and, add the file name to the list of files array
users_file_name = "#{ company.name.parameterize.underscore }_users_list.csv"
input_filenames << users_file_name
users_csv_file = File.new("#{ Rails.root.join('tmp') }/#{ users_file_name }", 'w')
users_csv_file << company.users.to_csv
users_csv_file.close
...
# gather up the created files and zip them
Zip::File.open(zipfile_path, create: true) do |zipfile|
input_filenames.uniq.each do |filename|
zipfile.add(filename, File.join(source_folder, filename))
end
end
puts "attaching data_export".colorize(:red)
company.data_exports.attach(
io: StringIO.new("#{ Rails.root }/tmp/company_14_#{ Time.current.to_date.to_s }_archive.zip"),
filename: 'company_14_archive.zip',
content_type: 'application/zip'
)
last_id = company.data_exports.last.id
puts "sending mail using company.id: #{ company.id }, recipient_id: #{ recipient_id }, company.data_exports.last.id: #{ last_id }".colorize(:red)
CompanyMailer.mail_data_export(
company.id,
recipient_id,
last_id
)
end
You can upload file like this on S3
key = "file_name.zip"
file_path = "tmp/file_name.zip"
new_s3_client = Aws::S3::Resource.new(region: 'eu-west-1', access_key_id: '123', secret_access_key: '456')
new_bucket = new_s3_client.bucket('public')
obj = new_bucket.object(key)
obj.upload_file(file_path)
In my task table I can access pdf file #task&.assignable&.pdf_file that is stored on s3 bucket.
I can access to its url using #task&.assignable&.pdf_file.file.url
I want to add a new page in the same pdf file so I am using combine_pdf gem. It is getting path of file1 that I have generated with wicked_pdf but on line
pdf << CombinePDF.load(#task&.assignable&.pdf_file.file.url)
It is showing error:
No such file or directory # rb_sysopen - https://patientiq1.s3-us-east-2.amazonaws.com/uploads/ < hashed_path >.pdf? < access_key >
file1_path = Rails.root.join('tmp', filename)
pdf = WickedPdf.new.pdf_from_string(
render_to_string('patient_engagements/signed_page', layout: 'pdf', locals: { :#signed_pdf => #signed_pdf })
)
File.open(file1_path, 'wb') do |file|
file << pdf
end
pdf = CombinePDF.new
pdf << CombinePDF.load(file1_path) # one way to combine, very fast.
pdf << CombinePDF.load(#task&.assignable&.pdf_file.file.url)
pdf.save "combined.pdf"
You can not use load with remote files, use parse instead
require 'combine_pdf'
require 'net/http'
url = "https://example.com/my.pdf"
pdf = CombinePDF.parse Net::HTTP.get_response(URI.parse(url)).body
Extracted from Loading and parsing DPF data at CombinePDF docs.
How to rotate image in S3 storage, I'm using Attachment_fu and Amazon S3 , I want to implement a feature that lets users edit existing images by rotating it 90 degrees. I was wondering if anyone had played with this, or had any ideas. I'm currently using rmagick
to rotate the image in Rails directory. But in AWS S3 storage cannot rotate the image.
require 'RMagick'
def rotate
photo = Photo.find(params[:id])
image = Magick::ImageList.new(photo.file)
image = image.rotate(90)
image.write(photo.file)
end
I'm using this gem:
gem 'pothoven-attachment_fu'
gem 'aws-s3'
gem 'rmagick'
I solved this problem.
def update_photo
require 'RMagick'
require 'base64'
# get the image data in Photo table
#photo = Photo.where(id: params[:photo_id]).first
# get image path in S3 url and encode to Base64 binary
img_url = #photo.s3_url()
img_uri = URI.parse(img_url)
base64_image = Base64.encode64(open(img_uri) { |io| io.read })
image_file_name = File.basename(img_uri.path,File.extname(img_uri.path))
filename_path = "#{Rails.root.to_s}/tmp/#{image_file_name}"
# create tempfile in rails root/tmp folder and decode the Base64 binary into image file, then processing RMagick to rotate the image into desire image rotation.
#tempfile = Tempfile.new(filename_path)
#tempfile.binmode
#tempfile.write Base64.decode64(base64_image)
#tempfile.close
# rotate image using RMagick
Magick::Image.read(img_uri).first.rotate!(params[:photo_degree].to_i).write(#tempfile.path)
# for security we want the actual content type, not just what was passed in
# get mime type of image and passed into ActionDispatch type
content_type = `file --mime -b #{#tempfile.path}`.split(";")[0]
# we will also add the extension ourselves based on the above
# if it's not gif/jpeg/png, it will fail the validation in the upload model
extension = content_type.match(/gif|jpeg|png/).to_s
extension = "jpg" if extension == "jpeg"
# append filename to prevent browser image caching
filename_path += "_#{params[:photo_degree]}"
filename_path += ".#{extension}" if extension
# passed the data into Rails core ActionDispatch instead of file input object
file = ActionDispatch::Http::UploadedFile.new({
tempfile: #tempfile,
type: content_type,
filename: filename_path
})
# delete the existing image
#photo.destroy
# create new image
img_rotate = Photo.new()
img_rotate.id = #photo.id
img_rotate.uploaded_data = file
img_rotate.save
ensure
clean_tempfile
end
def clean_tempfile
if #tempfile
#tempfile.close
#tempfile.unlink
end
end
I have a Rails 3 app that uses these gems:
gem 'paperclip'
gem 'wicked_pdf'
gem 'combine_pdf'
I'm using wicked_pdf to open a pdf for a costproject. The costproject has an HTML page called viewproject.pdf.erb.
I'm trying to combine the wicked pdf with the costproject attachments into a single pdf.
This is my controller code:
def viewproject
#costproject = Costproject.find(params[:costproject_id])
respond_to do |format|
format.html
format.pdf do
pdf = CombinePDF.new
pdf2 = render_to_string pdf: "Costproject.pdf", template: "costprojects/viewproject", encoding: "UTF-8"
pdf << CombinePDF.new(pdf2)
#costproject.attachments.each do |attachment|
pdf << CombinePDF.new(attachment.attach.path)
end
send_data pdf.to_pdf, :disposition => 'inline', :type => "application/pdf"
end
end
end
The line pdf << CombinePDF.new(pdf2) is giving me:
string contains null byte
If I look at pdf2, it starts like this - so it looks like a pdf:
>> pdf2
=> "%PDF-1.4\n1 0 obj\n<<\n/Title (\xFE\xFF)\n/Producer (wkhtmltopdf)\n/CreationDate (D:20150405202628)\n>>\nendobj\n4 0 obj\n<<\n/Type /ExtGState\n/SA true\n/SM 0.02\n/ca 1.0\n/CA 1.0\n/AIS false\n/SMask /None>>\nendobj\n5 0 obj\n[/Pattern /DeviceRGB]\nendobj\n8 0 obj\n<<\n/Type /XObject\n/Subtype /Image\n/Width 71\n/Height 75\n/BitsPerComponent 8\n/ColorSpace /DeviceRGB\n/Length 9 0 R\n/Filter
I also tried pdf << CombinePDF.new(pdf2.to_pdf)
Thanks for the help!
UPDATE1
As a test, to see if pdf2 is working, I did this successfully:
def viewproject
#costproject = Costproject.find(params[:costproject_id])
respond_to do |format|
format.html
format.pdf do
pdf2 = render_to_string pdf: "Costproject.pdf", template: "costprojects/viewproject", encoding: "UTF-8"
send_data pdf2, :disposition => 'inline', :type => "application/pdf"
end
end
end
UPDATE2
Myst was correct about using parse. Thanks!
I am now using this line in the controller code:
pdf << CombinePDF.new(attachment.attach.url)
I get this error:
No such file or directory - http://s3.amazonaws.com/ ...
But, if I copy the http address and paste into the browser the pdf shows up.
I am editing this answer to reflect the issue of remotely stored PDF files.
I should point out that without a persistent connection to the S3 storage and without using the S3 API, the following solution WILL effect performance*.
As I pointed out, the CombinePDF.new method is the same as the CombinePDF.load method. It accepts a file name and attempts to open the file. The CombinePDF.parse method will accept raw PDF data and parses it into a PDF Object.
In the following code I use Net::HTTP.get(URI.parse(url)) to get the raw PDF data.
I recommend replacing this solution with a S3 native solution, so that the whole application can share one or more persistent connections. This is a performance issue that may or may not be important for you.
require 'net/http'
def viewproject
#costproject = Costproject.find(params[:costproject_id])
respond_to do |format|
format.html
format.pdf do
pdf = CombinePDF.new
pdf2 = render_to_string pdf: "Costproject.pdf", template: "costprojects/viewproject", encoding: "UTF-8"
pdf << CombinePDF.parse(pdf2)
#costproject.attachments.each do |attachment|
pdf << CombinePDF.parse( Net::HTTP.get( URI.parse( attachment.attach.url ) ) )
end
send_data pdf.to_pdf, :disposition => 'inline', :type => "application/pdf"
end
end
end
* The performance hit is dependent on the amount of PDF attachments you have, on the number of users your app has, on network traffic, on your framework (single/multi-thread) and other factors.
A persistent connection should reduce the performance hit in a dramatic way, mainly due to the fact that establishing connections is an expensive action.
I'm having an invalid encoding error that doesn't let me save the image to a carrierwave uploader.
require 'rqrcode_png'
img = RQRCode::QRCode.new( 'test', :size => 4, :level => :h ).to_img.to_s
img.valid_encoding?
=> false
I'm not sure if this is what you're looking for, in my case I needed to associate the generated QR code with a Rails model using carrierwave, what I ended up doing was saving the image to a temp file, associating that file with the model and afterwards deleting the temp file, here's my code:
def generate_qr_code!
tmp_path = Rails.root.join('tmp', "some-filename.png")
tmp_file = RQRCode::QRCode.new(self.hash_value).to_img.resize(200,200).save(tmp_path)
# Stream is handed closed, we need to reopen it
File.open(tmp_file.path) do |file|
self.qr_code = file
end
File.delete(tmp_file.path)
self.save!
end