From ActiveRecord to S3 - ruby-on-rails

I am new to S3 and Rails so got stuck. I am wondering if I can directly pass object to S3.
My controller:
def start_upload
#forecasts = Forecast.all
Forecast.export_to_s3(#forecasts)
end
Model:
def self.export_to_s3(data)
---AWS configs ---
Aws.config = { :access_key_id => aws_access_key, :secret_access_key => aws_secret_access_key, :region => aws_region }
s3 = Aws::S3::Client.new(region:aws_region)
resp = s3.put_object(
:bucket => aws_bucket,
:key => aws_bucket_key_forcast,
:body => IO.read(data)
)
end

What are you trying to do? What is you goal?
You should try to serialize your object before you push it to your bucket.
def start_upload
#forecasts = Forecast.all
Forecast.export_to_s3(#forecasts.to_json)
end
def self.export_to_s3(data)
# ---AWS configs ---
Aws.config = { :access_key_id => aws_access_key, :secret_access_key => aws_secret_access_key, :region => aws_region }
tmpfile = Tempfile.new('forecast')
tmpfile.write(data)
tmpfile.close
s3 = Aws::S3::Client.new(region:aws_region)
resp = s3.put_object(
:bucket => aws_bucket,
:key => aws_bucket_key_forcast,
:body => IO.read(tmpfile)
)
tmpfile.unlink
end

Related

Rails google api client load P12/JSON file method

Following a sample which is here: https://gist.github.com/joost/5344705
The latest version of the google-api-client gem throws an error on:
key = Google::APIClient::PKCS12.load_key(key_file, 'notasecret')
#=> Uninitialized constant Google::APIClient(name error)
What is the new method for loading the P12/JSON file?
FYI, refactored error-prone code:
# you need to set this according to your situation/needs
SERVICE_ACCOUNT_EMAIL_ADDRESS = 'xxx#developer.gserviceaccount.com' # looks like 12345#developer.gserviceaccount.com
PATH_TO_KEY_FILE = '/home/arjun/rails/learn/xxx.p12' # the path to the downloaded .p12 key file
PROFILE = 'ga:xxx' # your GA profile id, looks like 'ga:12345'
require 'google/apis/analytics_v3'
require 'googleauth/stores/file_token_store'
Analytics = Google::Apis::AnalyticsV3
# set up a client instance
client = Analytics::AnalyticsService.new
client.authorization = Signet::OAuth2::Client.new(
:token_credential_uri => 'https://accounts.google.com/o/oauth2/token',
:audience => 'https://accounts.google.com/o/oauth2/token',
:scope => 'https://www.googleapis.com/auth/analytics.readonly',
:issuer => SERVICE_ACCOUNT_EMAIL_ADDRESS,
# :signing_key => OpenSSL::PKey.load_key(PATH_TO_KEY_FILE, 'notasecret')
# Not working = throws []': no implicit conversion of Symbol into Integer (TypeError)
:signing_key => Google::Auth::Stores::FileTokenStore.new(PATH_TO_KEY_FILE.to_i)
).tap { |auth| auth.fetch_access_token! }
api_method = client.discovered_api('analytics','v3').data.ga.get
# make queries
result = client.execute(:api_method => api_method, :parameters => {
'ids' => PROILE,
'start-date' => Date.new(1970,1,1).to_s,
'end-date' => Date.today.to_s,
'dimensions' => 'ga:pagePath',
'metrics' => 'ga:pageviews',
'filters' => 'ga:pagePath==/url/to/user'
})
puts result.data.rows.inspect
for version 0.9.18 of the google-api-client gem, the signing_key part should change to:
require 'google/api_client/auth/key_utils'
signing_key: Google::APIClient::KeyUtils::load_from_pkcs12(keypath, 'notasecret')

Ruby Google Custom Seach API get results from next page

the code below is only getting the results from the first page of the google pagination... how to get more results?
require 'google/api_client'
# autenticação
client = Google::APIClient.new(:application_name => 'Ruby Drive sample',
:application_version => '1.0.0',
:client_id => 'xxxxxxxxxxxxxxxx',
:client_secret => 'xxxxxxxxxxxxxx',
:authorization => nil)
search = client.discovered_api('customsearch')
# chama a API
response = client.execute(
:api_method => search.cse.list,
:parameters => {
'q' => 'cafe',
'key' => 'xxxxxxxxxxxx',
'cx' => 'xxxxxxxxxxxxxxxx'
}
)
# recebe a resposta em json
body = JSON.parse(response.body)
items = body['items']
# printa algumas informações...
items.each do |item|
puts "#{item['formattedUrl']}"
end
Simply use the nextPageToken as a parameter in the next request until you get back a result without a nextPageToken key
next_page_token = body['nextPageToken']
response_page2 = client.execute(
:api_method => search.cse.list,
:parameters => {
'nextPageToken' => next_page_token,
'key' => 'xxxxxxxxxxxx',
'cx' => 'xxxxxxxxxxxxxxxx'
}
)

How do i upload image from Rhomobile

I need to upload user profile image to server, i have it stored it on the app's base path.
settings = { :camera_type => #params['camera_type'],
:color_model => #params['color_model'],
:enable_editing => en_ed,
:desired_width => width,
:desired_height => height,
:flash_mode => 'auto',
:saveToDeviceGallery => 'true'}
Camera::take_picture(url_for(:action => :camera_callback), settings)
Then on callback,
Image.create({:id => generateId(), :image_uri => #params['image_uri']})
file_testname = File.join(Rho::RhoApplication::get_blob_path(#params['image_uri']))
test_content = File.binread(file_testname)
new_file_name = File.join(Rho::RhoApplication::get_base_app_path(), #params['image_uri'].to_s().split("/")[2])
f = File.new(new_file_name, "wb")
f.write(test_content)
f.close
How can i upload that image to server?
You can use the upload_file API
new_file_name = File.join(Rho::RhoApplication::get_base_app_path(), image.filename)
if File.exist?(new_file_name)
# Upload the file
result = Rho::AsyncHttp.upload_file(
:url => "#{##url}/image/upload",
:filename => new_file_name,
:headers => {},
:body => ""
)
if result["status"].upcase == "OK"
Alert.show_popup "Success"
end
end
You need to replace image.filename with your path.

Paperclip processor timeout on S3

I'm inserting a digital signature into a pdf file inside my processor, but keep getting an AWS::S3::ERRORS::Requestimeout error. What is this timeout? Is there any way i could keep the connection open until the file gets uploaded ?
Your socket connection to the server was not read from or written to
within the timeout period. Idle connections will be closed.
Here is my code :
model:
...
has_attached_file :receipt_file,
:storage => :s3,
:s3_credentials => "#{Rails.root}/config/s3.yml",
:path => "/:style/:id/:filename",
:s3_protocol => "https",
:styles => {dummy:""},
processors: [:SignPdf]
#process_in_background :receipt_file
...
Processor
module Paperclip
class SignPdf < Processor
attr_accessor :receipt_id,:style
S3_CONFIG = YAML.load_file("#{::Rails.root}/config/s3.yml")[Rails.env]
ORIGAMIDIR = "/ruby/1.9.1/gems/origami-1.2.4/lib"
def initialize(file, options = {}, attachment = nil)
#file = file
#current_format = File.extname(#file.path)
#basename = File.basename(#file.path, #current_format)
#attachment = attachment
end
def make
signPdf(#file)
#file
end
end
end
begin
require 'origami'
rescue LoadError
$: << ORIGAMIDIR
require 'origami'
end
include Origami
def signPdf(file)
certFile = "#{::Rails.root}/lib/assets/Cert.pem"
rsakeyFile = "#{::Rails.root}/lib/assets/pk.pem"
passphrase = "o2Receipts"
key4pem=File.read rsakeyFile
key = OpenSSL::PKey::RSA.new key4pem, passphrase
cert = OpenSSL::X509::Certificate.new(File.read certFile)
pdf = PDF.read(file)
page = pdf.get_page(1)
# Add signature annotation (so it becomes visibles in pdf document)
sigannot = Annotation::Widget::Signature.new
sigannot.Rect = Rectangle[:llx => 89.0, :lly => 386.0, :urx => 190.0, :ury => 353.0]
page.add_annot(sigannot)
# Sign the PDF with the specified keys
pdf.sign(cert, key,
:method => 'adbe.pkcs7.sha1',
:annotation => sigannot,
:location => "Portugal",
:contact => "email#email.pt",
:reason => "Proof of Concept"
)
# Save the resulting file
pdf.save(file.path)
file
end
I have worked around this, by using the after save. See my answer related to this subject here
What you're looking for isn't in the documentation as of today. You'll need to create an AWS::S3::Client
I'm referencing: https://github.com/thoughtbot/paperclip/blob/master/lib/paperclip/storage/s3.rb#L263
config.paperclip_defaults = {
storage: :s3,
s3_credentials: "#{Rails.root}/config/s3.yml",
s3_region: ENV['AWS_REGION'],
s3_protocol: :https,
s3_options: {
client: Aws::S3::Client.new(
access_key_id: ENV['S3_KEY'],
secret_access_key: ENV['S3_SECRET'],
http_open_timeout: 10,
http_read_timeout: 5,
http_idle_timeout: 20
)
}
}

rails - Model Methods, (Getting a PaperClip image URL off S3)

I have a rails 3 app with paperclip. The model looks a little like:
class Attachment < ActiveRecord::Base
has_attached_file :attachment,
:styles => {
:large => '1024x758>',
:medium => "200x150#",
:thumb => "100x75#",
:small => "50x50>"
},
:default_style => :original,
:default_url => '/images/:attachment/default_:style.png',
:path => ":instance_id/:attachment/:id/:style/:basename.:extension",
:storage => :s3,
:s3_credentials => File.join(Rails.root, 'config', 's3.yml'),
:s3_protocol => 'https',
:s3_permissions => :private,
:use_timestamp => false
def authenticated_url(style = nil, expires_in = 90.minutes)
AWS::S3::S3Object.url_for(attachment.path(style || attachment.default_style), attachment.bucket_name, :expires_in => expires_in, :use_ssl => attachment.s3_protocol == 'https')
end
This is called from a user_mailer which is called via delayed_job:
In user_mailer.rb mailer it looks a little something like this:
#comment.attachments.each do |a|
attachments[a.attachment_file_name] = open(a.authenticated_url()) {|f| f.read }
end
The issue here is that delayed mailer is erroring with:
{uninitialized constant Attachment::AWS
/Users/bhellman/Sites/cline/app/models/attachment.rb:53:in `authenticated_url'\n/Users/bhellman/Sites/cline/app/mailers/user_mailer.rb:41:in
`conversation_notification'\n/Library/Ruby/Gems/1.8/gems/activerecord-3.0.0/lib/active_record/associations/association_collection.rb:430:in
`method_missing'\n/Library/Ruby/Gems/1.8/gems/activerecord-3.0.0/lib/active_record/associations/association_proxy.rb:216:in `method_missing'\n/
Library/Ruby/Gems/1.8/gems/activerecord-3.0.0/lib/active_record/associations/association_proxy.rb:216:in
`each'\n/Library/Ruby/Gems/1.8/gems/activerecord-3.0.0/lib/active_record/associations/association_proxy.rb:216:in
`send'\n/Library/Ruby/Gems/1.8/gems/activerecord-3.0.0/lib/active_record/associations/association_proxy.rb:216:in `method_missing'\n/Library/Ruby/Gems/1.8/gems/activerecord-3.0.0/lib/active_record/associations/association_collection.rb:430:in `method_missing'\n/Users/bhellman/Sites/cline/app/mailers/user_mailer.rb:39:in `conversation_notification'\n/Library/Ruby/Gems/1.8/gems/actionpack-3.0.0/lib/abstract_controller/base.rb:150:in `send_action'\n/Library/Ruby/Gems/1.8/gems/actionpack-3.0.0/lib/abstract_controller/base.rb:150:in `process_action'\n/Library/Ruby/Gems/1.8/gems/actionpack-3.0.0/lib/abstract_controller/base.rb:119:in `process'\n/Library/Ruby/Gems/1.8/gems/actionpack-3.0.0/lib/abstract_controller/rendering.rb:40:in `process'\n/Library/Ruby/Gems/1.8/gems/actionmailer-3.0.0/lib/action_mailer/old_api.rb:75:in `process'\n/Library/Ruby/Gems/1.8/gems/actionmailer-3.0.0/lib/action_mailer/base.rb:446:in `process'\n/Library/Ruby/Gems/1.8/gems/actionmailer-3.0.0/lib/action_mailer/base.rb:441:in `initialize'\n/Library/Ruby/Gems/1.8/gems/actionmailer-3.0.0/lib/action_mailer/base.rb:425:in `new'\n/Library/Ruby/Gems/1.8/gems/actionmailer-3.0.0/lib/action_mailer/base.rb:425:in `method_missing'\n/Library/Ruby/Gems/1.8/bundler/gems/delayed_job-411719b38c51/lib/delayed/performable_mailer.rb:6:in `send'\n/Library/Ruby/Gems/1.8/bundler/gems/delayed_job-411719b38c51/lib/delayed/performable_mailer.rb:6:in `perform'\n/Library/Ruby/Gems/1.8/bundler/gems/delayed_job-411719b38c51/lib/delayed/backend/base.rb:83:in `invoke_job'\n/Library/Ruby/Gems/1.8/bundler/gems/delayed_job-411719b38c51/lib/delayed/worker.rb:119:in `run'\n/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/timeout.rb:62:in `timeout'\n/Library/Ruby/Gems/1.8/bundler/gems/delayed_job-411719b38c51/lib/delayed/worker.rb:119:in `run'\n/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/benchmark.rb:308:in `realtime'\n/Library/Ruby/Gems/1.8/bundler/gems/delayed_job-411719b38c51/lib/delayed/worker.rb:118:in `run'\n/Library/Ruby/Gems/1.8/bundler/gems/delayed_job-411719b38c51/lib/delayed/worker.rb:176:in `reserve_and_run_one_job'\n/Library/Ruby/Gems/1.8/bundler/gems/delayed_job-411719b38c51/lib/delayed/worker.rb:103:in `work_off'\n/Library/Ruby/Gems/1.8/bundler/gems/delayed_job-411719b38c51/lib/delayed/worker.rb:102:in `times'\n/Library/Ruby/Gems/1.8/bundler/gems/delayed_job-411719b38c51/lib/delayed/worker.rb:102:in `work_off'\n/Library/Ruby/Gems/1.8/bundler/gems/delayed_job-411719b38c51/lib/delayed/worker.rb:77:in `start'\n/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/benchmark.rb:308:in `realtime'\n/Library/Ruby/Gems/1.8/bundler/gems/delayed_job-411719b38c51/lib/delayed/worker.rb:76:in `start'\n/Library/Ruby/Gems/1.8/bundler/gems/delayed_job-411719b38c51/lib/delayed/worker.rb:73:in `loop'\n/Library/Ruby/Gems/1.8/bundler/gems/delayed_job-411719b38c51/lib/delayed/worker.rb:73:in `start'\n/Library/Ruby/Gems/1.8/bundler/gems/delayed_job-411719b38c51/lib/delayed/tasks.rb:9\n/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:636:in `call'\n/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:636:in `execute'\n/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:631:in `each'\n/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:631:in `execute'\n/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:597:in `invoke_with_call_chain'\n/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/monitor.rb:242:in `synchronize'\n/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:590:in `invoke_with_call_chain'\n/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:583:in `invoke'\n/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:2051:in `invoke_task'\n/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:2029:in `top_level'\n/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:2029:in `each'\n/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:2029:in `top_level'\n/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:2068:in `standard_exception_handling'\n/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:2023:in `top_level'\n/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:2001:in `run'\n/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:2068:in `standard_exception_handling'\n/Library/Ruby/Gems/1.8/gems/rake-0.8.7/lib/rake.rb:1998:in `run'\n/Library/Ruby/Gems/1.8/gems/rake-0.8.7/bin/rake:31\n/usr/bin/rake:19:in `load'\n/usr/bin/rake:19
Any idea what could be going on?
You need to require "aws/s3" in your model.

Resources