Paperclip processor timeout on S3 - ruby-on-rails

I'm inserting a digital signature into a pdf file inside my processor, but keep getting an AWS::S3::ERRORS::Requestimeout error. What is this timeout? Is there any way i could keep the connection open until the file gets uploaded ?
Your socket connection to the server was not read from or written to
within the timeout period. Idle connections will be closed.
Here is my code :
model:
...
has_attached_file :receipt_file,
:storage => :s3,
:s3_credentials => "#{Rails.root}/config/s3.yml",
:path => "/:style/:id/:filename",
:s3_protocol => "https",
:styles => {dummy:""},
processors: [:SignPdf]
#process_in_background :receipt_file
...
Processor
module Paperclip
class SignPdf < Processor
attr_accessor :receipt_id,:style
S3_CONFIG = YAML.load_file("#{::Rails.root}/config/s3.yml")[Rails.env]
ORIGAMIDIR = "/ruby/1.9.1/gems/origami-1.2.4/lib"
def initialize(file, options = {}, attachment = nil)
#file = file
#current_format = File.extname(#file.path)
#basename = File.basename(#file.path, #current_format)
#attachment = attachment
end
def make
signPdf(#file)
#file
end
end
end
begin
require 'origami'
rescue LoadError
$: << ORIGAMIDIR
require 'origami'
end
include Origami
def signPdf(file)
certFile = "#{::Rails.root}/lib/assets/Cert.pem"
rsakeyFile = "#{::Rails.root}/lib/assets/pk.pem"
passphrase = "o2Receipts"
key4pem=File.read rsakeyFile
key = OpenSSL::PKey::RSA.new key4pem, passphrase
cert = OpenSSL::X509::Certificate.new(File.read certFile)
pdf = PDF.read(file)
page = pdf.get_page(1)
# Add signature annotation (so it becomes visibles in pdf document)
sigannot = Annotation::Widget::Signature.new
sigannot.Rect = Rectangle[:llx => 89.0, :lly => 386.0, :urx => 190.0, :ury => 353.0]
page.add_annot(sigannot)
# Sign the PDF with the specified keys
pdf.sign(cert, key,
:method => 'adbe.pkcs7.sha1',
:annotation => sigannot,
:location => "Portugal",
:contact => "email#email.pt",
:reason => "Proof of Concept"
)
# Save the resulting file
pdf.save(file.path)
file
end

I have worked around this, by using the after save. See my answer related to this subject here

What you're looking for isn't in the documentation as of today. You'll need to create an AWS::S3::Client
I'm referencing: https://github.com/thoughtbot/paperclip/blob/master/lib/paperclip/storage/s3.rb#L263
config.paperclip_defaults = {
storage: :s3,
s3_credentials: "#{Rails.root}/config/s3.yml",
s3_region: ENV['AWS_REGION'],
s3_protocol: :https,
s3_options: {
client: Aws::S3::Client.new(
access_key_id: ENV['S3_KEY'],
secret_access_key: ENV['S3_SECRET'],
http_open_timeout: 10,
http_read_timeout: 5,
http_idle_timeout: 20
)
}
}

Related

missing region; use :region option or export region name to ENV['AWS_REGION']

I understand there are other questions that are the same, but they have no solved my problem.
I keep on receiving the error: Aws::Errors::MissingRegionError in BooksController#create,
missing region; use :region option or export region name to ENV['AWS_REGION']. However, this is my configuration in my
Development.rb:
config.paperclip_defaults = {
storage: :s3,
s3_host_name: "s3-us-west-2.amazonaws.com",
s3_credentials: {
bucket: ENV['AWS_BUCKET'],
access_key_id: ENV['AWS_ACCESS_KEY_ID'],
secret_access_key: ENV['AWS_SECRET_ACCESS_KEY'],
s3_region: ENV['us-west-2']
}
}
Production.rb:
config.paperclip_defaults = {
storage: :s3,
s3_host_name: "s3-us-west-2.amazonaws.com",
s3_credentials: {
bucket: ENV['AWS_BUCKET'],
access_key_id: ENV['AWS_ACCESS_KEY_ID'],
secret_access_key: ENV['AWS_SECRET_ACCESS_KEY'],
s3_region: ENV['us-west-2']
}
}
And Application.rb:
config.paperclip_defaults = {
storage: :s3,
s3_host_name: "s3-us-west-2.amazonaws.com",
s3_credentials: {
bucket: ENV['AWS_BUCKET'],
access_key_id: ENV['AWS_ACCESS_KEY_ID'],
secret_access_key: ENV['AWS_SECRET_ACCESS_KEY'],
s3_region: ENV['us-west-2']
}
}
However, it keeps coming up with the error. I have followed other people's advice from other questions. Hope someone can help.
Ben
You should either set the ENV['AWS_REGION'] env variable to "us-west-2" and use it as
s3_region: ENV['AWS_REGION']
Or use a string:
s3_region: 'us-west-2'
Also, s3_region option should me moved out of credentials hash in config/environments/{development|production}:
config.paperclip_defaults = {
storage: :s3,
s3_host_name: "s3-us-west-2.amazonaws.com",
s3_region: 'us-west-2', # or ENV['AWS_REGION']
s3_credentials: {
bucket: ENV['AWS_BUCKET'],
access_key_id: ENV['AWS_ACCESS_KEY_ID'],
secret_access_key: ENV['AWS_SECRET_ACCESS_KEY'],
}
}
Rails 5 Way:
according to updated documentation region is necessary. where as it was necessary for non us-region. and the recommended way to setup paperclip with s3 is:
put your secret info in yml file let say aws.yml (must be git
ignored)
put your global configurations in environmental files i.e
development/production.rb in config/environments/
put your files related setting in model. in my case I am defining
image properties in User model
example step-1: (config/aws.yml)
development:
access_key_id: AWS_ACCESS_KEY_ID # without quotation marks
secret_access_key: AWS_SECRET_KEY_ID # without quotation marks
production:
access_key_id: <%= ENV["AWS_ACCESS_KEY_ID"] %> # get it from terminal environment
secret_access_key: <%= ENV["AWS_SECRET_KEY_ID"] %> # get it from terminal environment
example step-2: (config/environments/development.rb)
# US West (N. California) us-west-2 apigateway.us-west-2.amazonaws.com HTTPS
config.paperclip_defaults = {
:storage => :s3,
:s3_region => 'us-west-2',
:bucket => 'production-bucket',
:path => '/:class/:attachment/:id_partition/:style/:filename',
:s3_credentials => "#{Rails.root}/config/aws.yml",
}
example step-3: (app/models/user.rb)
has_attached_file :avatar, styles: { medium: "300x300>", thumb: "100x100>" }
validates_attachment_content_type :avatar, content_type: /\Aimage\/.*\z/
finally you can upload file:
def upload_to_s3
response = {JSON_KEY_STATUS_MESSAGE: "OK", server_time: DateTime.current.to_i}
response[:user] = []
response[:status] = '0'
unless params[:avatar].present?
response[:message] = 'either user avatar image file [avatar]'
render json: response and return
end
begin
user = User.new # create new user
user.avatar = params[:avatar] # params[:avatar] is a file posted by form in mutli-part true over json api
s = user.save # save it # will through error if you have more than one required attributes
if(s != false)
response[:message] = 'file Successfully upload to s3'
else
response[:message] = 'fail to upload file to s3'
end
rescue => e
response[:message] = e.message # this guy will help debug a lot!
end
render json: response and return
end
create a file called:
config/initializers/paperclip.rb
And add the following:
Paperclip::Attachment.default_options[:url] = ':s3_domain_url'
Paperclip::Attachment.default_options[:path] = '/:class/:attachment/:id_partition/:style/:filename'
Paperclip::Attachment.default_options[:s3_host_name] = 's3-eu-west-1.amazonaws.com'
And in the 3rd line where I have eu-west-1 replace that depending on the region you are in.

Rails google api client load P12/JSON file method

Following a sample which is here: https://gist.github.com/joost/5344705
The latest version of the google-api-client gem throws an error on:
key = Google::APIClient::PKCS12.load_key(key_file, 'notasecret')
#=> Uninitialized constant Google::APIClient(name error)
What is the new method for loading the P12/JSON file?
FYI, refactored error-prone code:
# you need to set this according to your situation/needs
SERVICE_ACCOUNT_EMAIL_ADDRESS = 'xxx#developer.gserviceaccount.com' # looks like 12345#developer.gserviceaccount.com
PATH_TO_KEY_FILE = '/home/arjun/rails/learn/xxx.p12' # the path to the downloaded .p12 key file
PROFILE = 'ga:xxx' # your GA profile id, looks like 'ga:12345'
require 'google/apis/analytics_v3'
require 'googleauth/stores/file_token_store'
Analytics = Google::Apis::AnalyticsV3
# set up a client instance
client = Analytics::AnalyticsService.new
client.authorization = Signet::OAuth2::Client.new(
:token_credential_uri => 'https://accounts.google.com/o/oauth2/token',
:audience => 'https://accounts.google.com/o/oauth2/token',
:scope => 'https://www.googleapis.com/auth/analytics.readonly',
:issuer => SERVICE_ACCOUNT_EMAIL_ADDRESS,
# :signing_key => OpenSSL::PKey.load_key(PATH_TO_KEY_FILE, 'notasecret')
# Not working = throws []': no implicit conversion of Symbol into Integer (TypeError)
:signing_key => Google::Auth::Stores::FileTokenStore.new(PATH_TO_KEY_FILE.to_i)
).tap { |auth| auth.fetch_access_token! }
api_method = client.discovered_api('analytics','v3').data.ga.get
# make queries
result = client.execute(:api_method => api_method, :parameters => {
'ids' => PROILE,
'start-date' => Date.new(1970,1,1).to_s,
'end-date' => Date.today.to_s,
'dimensions' => 'ga:pagePath',
'metrics' => 'ga:pageviews',
'filters' => 'ga:pagePath==/url/to/user'
})
puts result.data.rows.inspect
for version 0.9.18 of the google-api-client gem, the signing_key part should change to:
require 'google/api_client/auth/key_utils'
signing_key: Google::APIClient::KeyUtils::load_from_pkcs12(keypath, 'notasecret')

From ActiveRecord to S3

I am new to S3 and Rails so got stuck. I am wondering if I can directly pass object to S3.
My controller:
def start_upload
#forecasts = Forecast.all
Forecast.export_to_s3(#forecasts)
end
Model:
def self.export_to_s3(data)
---AWS configs ---
Aws.config = { :access_key_id => aws_access_key, :secret_access_key => aws_secret_access_key, :region => aws_region }
s3 = Aws::S3::Client.new(region:aws_region)
resp = s3.put_object(
:bucket => aws_bucket,
:key => aws_bucket_key_forcast,
:body => IO.read(data)
)
end
What are you trying to do? What is you goal?
You should try to serialize your object before you push it to your bucket.
def start_upload
#forecasts = Forecast.all
Forecast.export_to_s3(#forecasts.to_json)
end
def self.export_to_s3(data)
# ---AWS configs ---
Aws.config = { :access_key_id => aws_access_key, :secret_access_key => aws_secret_access_key, :region => aws_region }
tmpfile = Tempfile.new('forecast')
tmpfile.write(data)
tmpfile.close
s3 = Aws::S3::Client.new(region:aws_region)
resp = s3.put_object(
:bucket => aws_bucket,
:key => aws_bucket_key_forcast,
:body => IO.read(tmpfile)
)
tmpfile.unlink
end

How do i upload image from Rhomobile

I need to upload user profile image to server, i have it stored it on the app's base path.
settings = { :camera_type => #params['camera_type'],
:color_model => #params['color_model'],
:enable_editing => en_ed,
:desired_width => width,
:desired_height => height,
:flash_mode => 'auto',
:saveToDeviceGallery => 'true'}
Camera::take_picture(url_for(:action => :camera_callback), settings)
Then on callback,
Image.create({:id => generateId(), :image_uri => #params['image_uri']})
file_testname = File.join(Rho::RhoApplication::get_blob_path(#params['image_uri']))
test_content = File.binread(file_testname)
new_file_name = File.join(Rho::RhoApplication::get_base_app_path(), #params['image_uri'].to_s().split("/")[2])
f = File.new(new_file_name, "wb")
f.write(test_content)
f.close
How can i upload that image to server?
You can use the upload_file API
new_file_name = File.join(Rho::RhoApplication::get_base_app_path(), image.filename)
if File.exist?(new_file_name)
# Upload the file
result = Rho::AsyncHttp.upload_file(
:url => "#{##url}/image/upload",
:filename => new_file_name,
:headers => {},
:body => ""
)
if result["status"].upcase == "OK"
Alert.show_popup "Success"
end
end
You need to replace image.filename with your path.

rails - Model Methods, (Getting a PaperClip image URL off S3)

I have a rails 3 app with paperclip. The model looks a little like:
class Attachment < ActiveRecord::Base
has_attached_file :attachment,
:styles => {
:large => '1024x758>',
:medium => "200x150#",
:thumb => "100x75#",
:small => "50x50>"
},
:default_style => :original,
:default_url => '/images/:attachment/default_:style.png',
:path => ":instance_id/:attachment/:id/:style/:basename.:extension",
:storage => :s3,
:s3_credentials => File.join(Rails.root, 'config', 's3.yml'),
:s3_protocol => 'https',
:s3_permissions => :private,
:use_timestamp => false
def authenticated_url(style = nil, expires_in = 90.minutes)
AWS::S3::S3Object.url_for(attachment.path(style || attachment.default_style), attachment.bucket_name, :expires_in => expires_in, :use_ssl => attachment.s3_protocol == 'https')
end
This is called from a user_mailer which is called via delayed_job:
In user_mailer.rb mailer it looks a little something like this:
#comment.attachments.each do |a|
attachments[a.attachment_file_name] = open(a.authenticated_url()) {|f| f.read }
end
The issue here is that delayed mailer is erroring with:
{uninitialized constant Attachment::AWS
/Users/bhellman/Sites/cline/app/models/attachment.rb:53:in `authenticated_url'\n/Users/bhellman/Sites/cline/app/mailers/user_mailer.rb:41:in
`conversation_notification'\n/Library/Ruby/Gems/1.8/gems/activerecord-3.0.0/lib/active_record/associations/association_collection.rb:430:in
`method_missing'\n/Library/Ruby/Gems/1.8/gems/activerecord-3.0.0/lib/active_record/associations/association_proxy.rb:216:in `method_missing'\n/
Library/Ruby/Gems/1.8/gems/activerecord-3.0.0/lib/active_record/associations/association_proxy.rb:216:in
`each'\n/Library/Ruby/Gems/1.8/gems/activerecord-3.0.0/lib/active_record/associations/association_proxy.rb:216:in
`send'\n/Library/Ruby/Gems/1.8/gems/activerecord-3.0.0/lib/active_record/associations/association_proxy.rb:216:in `method_missing'\n/Library/Ruby/Gems/1.8/gems/activerecord-3.0.0/lib/active_record/associations/association_collection.rb:430:in `method_missing'\n/Users/bhellman/Sites/cline/app/mailers/user_mailer.rb:39:in `conversation_notification'\n/Library/Ruby/Gems/1.8/gems/actionpack-3.0.0/lib/abstract_controller/base.rb:150:in `send_action'\n/Library/Ruby/Gems/1.8/gems/actionpack-3.0.0/lib/abstract_controller/base.rb:150:in `process_action'\n/Library/Ruby/Gems/1.8/gems/actionpack-3.0.0/lib/abstract_controller/base.rb:119:in `process'\n/Library/Ruby/Gems/1.8/gems/actionpack-3.0.0/lib/abstract_controller/rendering.rb:40:in `process'\n/Library/Ruby/Gems/1.8/gems/actionmailer-3.0.0/lib/action_mailer/old_api.rb:75:in `process'\n/Library/Ruby/Gems/1.8/gems/actionmailer-3.0.0/lib/action_mailer/base.rb:446:in `process'\n/Library/Ruby/Gems/1.8/gems/actionmailer-3.0.0/lib/action_mailer/base.rb:441:in `initialize'\n/Library/Ruby/Gems/1.8/gems/actionmailer-3.0.0/lib/action_mailer/base.rb:425:in `new'\n/Library/Ruby/Gems/1.8/gems/actionmailer-3.0.0/lib/action_mailer/base.rb:425:in `method_missing'\n/Library/Ruby/Gems/1.8/bundler/gems/delayed_job-411719b38c51/lib/delayed/performable_mailer.rb:6:in `send'\n/Library/Ruby/Gems/1.8/bundler/gems/delayed_job-411719b38c51/lib/delayed/performable_mailer.rb:6:in `perform'\n/Library/Ruby/Gems/1.8/bundler/gems/delayed_job-411719b38c51/lib/delayed/backend/base.rb:83:in `invoke_job'\n/Library/Ruby/Gems/1.8/bundler/gems/delayed_job-411719b38c51/lib/delayed/worker.rb:119:in `run'\n/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/timeout.rb:62:in `timeout'\n/Library/Ruby/Gems/1.8/bundler/gems/delayed_job-411719b38c51/lib/delayed/worker.rb:119:in `run'\n/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/benchmark.rb:308:in `realtime'\n/Library/Ruby/Gems/1.8/bundler/gems/delayed_job-411719b38c51/lib/delayed/worker.rb:118:in `run'\n/Library/Ruby/Gems/1.8/bundler/gems/delayed_job-411719b38c51/lib/delayed/worker.rb:176:in `reserve_and_run_one_job'\n/Library/Ruby/Gems/1.8/bundler/gems/delayed_job-411719b38c51/lib/delayed/worker.rb:103:in `work_off'\n/Library/Ruby/Gems/1.8/bundler/gems/delayed_job-411719b38c51/lib/delayed/worker.rb:102:in `times'\n/Library/Ruby/Gems/1.8/bundler/gems/delayed_job-411719b38c51/lib/delayed/worker.rb:102:in `work_off'\n/Library/Ruby/Gems/1.8/bundler/gems/delayed_job-411719b38c51/lib/delayed/worker.rb:77:in `start'\n/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/benchmark.rb:308:in `realtime'\n/Library/Ruby/Gems/1.8/bundler/gems/delayed_job-411719b38c51/lib/delayed/worker.rb:76:in `start'\n/Library/Ruby/Gems/1.8/bundler/gems/delayed_job-411719b38c51/lib/delayed/worker.rb:73:in `loop'\n/Library/Ruby/Gems/1.8/bundler/gems/delayed_job-411719b38c51/lib/delayed/worker.rb:73:in `start'\n/Library/Ruby/Gems/1.8/bundler/gems/delayed_job-411719b38c51/lib/delayed/tasks.rb:9\n/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:636:in `call'\n/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:636:in `execute'\n/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:631:in `each'\n/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:631:in `execute'\n/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:597:in `invoke_with_call_chain'\n/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/monitor.rb:242:in `synchronize'\n/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:590:in `invoke_with_call_chain'\n/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:583:in `invoke'\n/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:2051:in `invoke_task'\n/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:2029:in `top_level'\n/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:2029:in `each'\n/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:2029:in `top_level'\n/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:2068:in `standard_exception_handling'\n/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:2023:in `top_level'\n/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:2001:in `run'\n/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:2068:in `standard_exception_handling'\n/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:1998:in `run'\n/Library/Ruby/Gems/1.8/gems/rake-0.8.7/bin/rake:31\n/usr/bin/rake:19:in `load'\n/usr/bin/rake:19
Any idea what could be going on?
You need to require "aws/s3" in your model.

Resources