Associate object to pre-existing file on S3, using Paperclip - ruby-on-rails

I have a file already on S3 that I'd like to associate to a pre-existing instance of the Asset model.
Here's the model:
class Asset < ActiveRecord::Base
attr_accessible(:attachment_content_type, :attachment_file_name,
:attachment_file_size, :attachment_updated_at, :attachment)
has_attached_file :attachment, {
storage: :s3,
s3_credentials: {
access_key_id: ENV['AWS_ACCESS_KEY_ID'],
secret_access_key: ENV['AWS_SECRET_ACCESS_KEY'],
},
convert_options: { all: '-auto-orient' },
url: ':s3_alias_url',
s3_host_alias: ENV['S3_HOST_ALIAS'],
path: ":class/:attachment/:id_partition/:style/:filename",
bucket: ENV['S3_BUCKET_NAME'],
s3_protocol: 'https'
}
end
Let's say the path is assets/attachments/000/111/file.png, and the Asset instance I want to associate with the file is asset. Referring at the source, I've tried:
options = {
storage: :s3,
s3_credentials: {
access_key_id: ENV['AWS_ACCESS_KEY_ID'],
secret_access_key: ENV['AWS_SECRET_ACCESS_KEY'],
},
convert_options: { all: '-auto-orient' },
url: ':s3_alias_url',
s3_host_alias: ENV['S3_HOST_ALIAS'],
path: "assets/attachments/000/111/file.png",
bucket: ENV['S3_BUCKET_NAME'],
s3_protocol: 'https'
}
# The above is identical to the options given in the model, except for the path
Paperclip::Attachment.new("file.png", asset, options).save
As far as I can tell, this did not affect asset in any way. I cannot set asset.attachment.path manually.
Other questions on SO do not seem to address this specifically.
"paperclip images not saving in the path i've set up", "Paperclip and Amazon S3 how to do paths?", and so on involve setting up the model, which is already working fine.
Anyone have any insight to offer?

As far as I can tell, I do need to turn the S3 object into a File, as suggested by #oregontrail256. I used the Fog gem to do this.
s3 = Fog::Storage.new(
:provider => 'AWS',
:aws_access_key_id => ENV['AWS_ACCESS_KEY_ID'],
:aws_secret_access_key => ENV['AWS_SECRET_ACCESS_KEY']
)
directory = s3.directories.get(ENV['S3_BUCKET_NAME'])
fog_file = directory.files.get(path)
file = File.open("temp", "wb")
file.write(fog_file.body)
asset.attachment = file
asset.save
file.close

Paperclip attachments have a copy_to_local_file() method that allows you to make a local copy of the attachment. So what about:
file_name = "temp_file"
asset1.attachment.copy_to_local_file(:style, file_name)
file = File.open(file_name)
asset2.attachment = file
file.close
asset2.save!
Even if you destroy asset1, you now have a copy of the attachment saved by asset2 separately. You probably want to do this in a background job if you're doing many of them.
Credit to this answer too: How to set a file upload programmatically using Paperclip

Related

Paperclip: Choose between saving files local or S3 at runtime

I'm using Paperclip for saving files. I have configured successfully for Paperclip saving files directly to Amazon S3. But in some situations, I need files only be saved locally. My question is: How can I do this.
Here is my example Paperclip configuration:
Paperclip.interpolates(:upload_url) { |attachment, style| "#{ENV.fetch('UPLOAD_PROTOCOL', 'http')}://#{ENV.fetch('UPLOAD_DOMAIN', 'localhost:3000')}/uploads/:class/:attachment/:id_:style.:extension" }
Paperclip::Attachment.default_options.merge!(
storage: :s3,
s3_region: ENV['CEPH_REGION'],
s3_protocol: 'http',
s3_host_name: ENV['CEPH_HOST_NAME'],
s3_credentials: {
access_key_id: ENV['CEPH_ACCESS_KEY_ID'],
secret_access_key: ENV['CEPH_SECRET_KEY'],
},
s3_options: {
endpoint: ENV['CEPH_END_POINT'],
force_path_style: true
},
s3_permissions: 'public-read',
bucket: ENV['CEPH_BUCKET'],
url: ':s3_path_url',
path: ':class/:id/:basename.:extension',
use_timestamp: false
)
module Paperclip
def self.string_to_io(options)
data = StringIO.new(options[:data])
data.class.class_eval{ attr_accessor :original_filename }
data.original_filename = options[:original_file_name]
data
end
end
You could use a lambda. As shown here. https://github.com/thoughtbot/paperclip#dynamic-configuration
it would look something like this.
class YOURMODEL < ActiveRecord::Base
has_attached_file :FILE, storage: lambda { |attachment| (attachment.instance.use_s3? ? :s3 : :filesystem) }
end
Then you would have to add a method use_s3? to your model to check if you wanted to store the file locally or with s3.

Ruby on Rails - AWS-SDK configuration file

I'm using AWS-SDK gem in my Rails project, and I want a kind-of initializer file to connect directly to my repo and make changes directly in the Rails console, something like this:
# At config/initializers/aws.rb
Aws::S3::Client.new(
:access_key_id => 'ACCESS_KEY_ID',
:secret_access_key => 'SECRET_ACCESS_KEY'
)
I've looked for documentation or tutorials but it's not clear for me. How do I do it? Thank you!
i think you can try like this
put this in aws.rb
AWS.config(
:access_key_id => ENV['ACCESS_KEY_ID'],
:secret_access_key => ENV['SECRET_ACCESS_KEY']
)
and when you initialize the object wherever you need, will call the configuration
s3 = AWS::S3.new
To share configuration between AWS service client in a Rails application, configure the AWS SDK for Ruby from a config initializer.
# config/initializers/aws-sdk.rb
Aws.config.update(
credentials: Aws::Credentials.new('access-key-id', 'secret-access-key'),
region: 'us-east-1',
)
Now you can construct a client object from any service without any options:
s3 = Aws::S3::Client.new
ec2 = Aws::EC2::Client.new
Please note, you should avoid hard-coding credentials into your application. This can be a security risk if your source code is accessed and it makes it difficult to rotate credentials.
I recommend using hands-off configuration via ENV['AWS_ACCESS_KEY_ID'] and ENV['AWS_SECRET_ACCESS_KEY'], or an EC2 instance profile.
Finally, I've found the solution:
Create the file aws.rb in your /config/initializers folder.
In aws.rb write:
S3Client = Aws::S3::Client.new(
access_key_id: 'ACCESS_KEY_ID',
secret_access_key: 'SECRET_ACCESS_KEY',
region: 'REGION'
)
That's it. Thank you all for your answers!
Also with aws-sdk-rails (1.0.0)
# config/initializers/aws.rb
Aws.config[:credentials] = Aws::Credentials.new(ENV['AWS_ACCESS_KEY_ID'], ENV['AWS_SECRET_ACCESS_KEY'])
TRY THIS CONFIGURATION:-
in config/intitalizers/s3.rb
Paperclip.interpolates(:s3_eu_url) { |attachment, style|
"#{attachment.s3_protocol}://s3-eu-west-1.amazonaws.com/#{attachment.bucket_name}/#{attachment.path(style).gsub(%r{^/}, "")}"
}
config/initializers/paperclip.rb
require 'paperclip/media_type_spoof_detector'
module Paperclip
class MediaTypeSpoofDetector
def spoofed?
false
end
end
end
Paperclip::Attachment.default_options[:url] = ':s3_domain_url'
Paperclip::Attachment.default_options[:path] = '/:class/:id/:style/:filename'
S3_CREDENTIALS = Rails.root.join("config/s3.yml")
/config/s3.yml
development:
bucket: development_bucket
access_key_id: AKIA-----API KEYS---------MCLXQ
secret_access_key: qTNF1-------API KEYS--------DTy+rPubaaG
production:
bucket: production_bucket
access_key_id: AKI-----API KEYS--------LXQ
secret_access_key: qTNF1dW---API KEYS---+rPubaaG
hope you have gem "aws-sdk"in the Gemfile
add you asset in model
has_attached_file :avatar,
:styles => {:view => "187x260#"},
:storage => :s3,
:s3_permissions => :private,
:s3_credentials => S3_CREDENTIALS
verify using rails console with static image in public
Image.create(avatar: File.new("#{Rails.root}/public/images/colorful_blue.jpg"))

Paperclip uploads on Heroku using S3

I'm sorry to rehash an old gripe but I'm at my wits end and not sure where to go next. I am using Paperclip on Heroku and have S3 uploads configured. I was able to get things working in my local development environment but once it's running on Heroku I run into this error:
AWS::S3::Errors::PermanentRedirect (The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.
I've googled this error and read through the Heroku documentation and I believe I have everything set up correctly. I initially thought that my problems stemmed from having my bucket in the s3-us-west-1.amazonaws.com region, but I'm not convinced anymore.
Here are the relevant parts of my Heroku config:
AWS_REGION: us-west-1
S3_BUCKET_NAME: my-super-awesomely-amazing-bucket
From my config/environments/production.rb file:
config.paperclip_defaults = {
:storage => :s3,
:s3_credentials => {
:bucket => ENV['S3_BUCKET_NAME'],
:access_key_id => ENV['AWS_ACCESS_KEY_ID'],
:secret_access_key => ENV['AWS_SECRET_ACCESS_KEY']
}
}
My paperclip.rb initialize file:
if Rails.env.production?
Paperclip::Attachment.default_options[:url] = ':s3_domain_url'
Paperclip::Attachment.default_options[:path] = '/:class/:attachment/:id_partition/:style/:filename'
Paperclip::Attachment.default_options[:s3_host_name] = 's3-us-west-1.amazonaws.com'
end
And my paperclip config from the relevant model:
has_attached_file :document,
:styles => { },
:default_url => "/image_styles/:style/missing.png"
So...what am I doing wrong here? At this point I'm sure I've missed something obvious but I'm stumped on where to go from here. I feel like I've assiduously configured everything and yet that PermanentRedirect error keeps coming up.
Bucket
This might not be the direct solution, but we've found that you have to include the bucket option outside of your s3_credentials block:
#config/environments/production.rb
config.paperclip_defaults = {
storage: :s3,
s3_host_name: 's3-eu-west-1.amazonaws.com',
s3_credentials: {
access_key_id: ENV['AWS_ACCESS_KEY_ID'],
secret_access_key: ENV['AWS_SECRET_ACCESS_KEY']
},
bucket: ENV['S3_BUCKET_NAME']
}
This is working 100% for us on Heroku, but whether it wil work for you (as your bucket is in a different region) is a different matter
If you need more help, ask a comment and I'll gladly give you some ideas
Try this link http://adamthedeveloper.blogspot.in/2014/02/awss3permanentredirect-bucket-you-are.html . Previously this happened to Europe region buckets only and get resolved by setting AWS::S3::DEFAULT_HOST. Hope this solve your issue.

Rails 4, Paperclip, Amazon S3 Config Amazon Path

I'm trying to configure the endpoint which is returned from paperclip when my object is successfully uploaded to Amazon's S3 service. The upload and everything is working correctly, but the URL that is being returned is incorrect for displaying the upload.
Right now, the url that is being returned is http://s3.amazonaws.com/path/to/my/items (as seen in the picture below).
Instead of s3.amazonaws.com, I would like the root to be specific to the bucket's location (e.g. s3-us-west-1.amazonaws.com/path/to/my/items)
Where should I try and configure a different url path (from s3.amazonaws.com to something else)? I've tried to add a url with the above path into my configuration file like:
#Paperclip Amazon S3
config.paperclip_defaults = {
:storage => :s3,
:url => "https://s3-us-west-1.amazonaws.com/",
:s3_credentials => {
:bucket => ENV['S3_BUCKET_NAME'],
:access_key_id => ENV['AWS_ACCESS_KEY_ID'],
:secret_access_key => ENV['AWS_SECRET_ACCESS_KEY']
}
}
Which did not appear to have any effect. Please advise on where I should be setting this option!
Thanks in advance!
If you're going to use S3, we've found that you have to include the S3 credentials in your actual model (not just the config files). Here's what we do:
Model
#Image Upload
Paperclip.options[:command_path] = 'C:\RailsInstaller\ImageMagick'
has_attached_file :image,
:styles => { :medium => "x300", :thumb => "x100" },
:default_url => "****",
:storage => :s3,
:bucket => '****',
:s3_credentials => S3_CREDENTIALS,
:url => "/:image/:id/:style/:basename.:extension",
:path => ":image/:id/:style/:basename.:extension"
config/application.rb
# Paperclip (for Amazon) (we use EU servers)
config.paperclip_defaults = {
:storage => :s3,
:s3_host_name => 's3-eu-west-1.amazonaws.com'
}
config/s3.yml
#Amazon AWS Config
development:
access_key_id: **********
secret_access_key: **************
bucket: ****
production:
access_key_id: ***********
secret_access_key: ***********
bucket: ****
Hope this helps?
I also had the same problem when migrating to Spree 2.2 and am still not sure how to solve it the correct way. It seems like Paperclip should have been updating the path from the configuration, but it isn't.
Lacking a better solution, I've overridden the Spree::Image class like this:
1 Spree::Image.class_eval do
2 has_attached_file :attachment,
3 styles: { mini: '48x48>', small: '100x100>', product: '240x240>', large: '600x600>' },
4 default_style: :product,
5 url: '/spree/products/:id/:style/:basename.:extension',
6 path: 'products/:id/:style/:basename.:extension',
7 convert_options: { all: '-strip -auto-orient -colorspace sRGB' }ยท
8 end
After some experimentation I have found that setting :s3_host_name globally suffices. I ended up with the same problem because I was setting :s3_region, which was being used by Paperclip (post-4.3.1, with aws-sdk 2) for storing attachments, but not when generating the URLs.
This may also be of interest to readers who end up on this problem: https://github.com/thoughtbot/paperclip/wiki/Restricting-Access-to-Objects-Stored-on-Amazon-S3

Rails Paperclip S3 ArgumentError (missing required :bucket option):

I've been stuck on this for ages now and can't figure out what's wrong. There are a lot of people that seem to have this same problem, but I can't actually find any answers that actually work.
production.rb
config.paperclip_defaults = {
:storage => :s3,
:s3_credentials => {
:bucket => ENV['my bucket name is here'],
:access_key_id => ENV['my key is here'],
:secret_access_key => ENV['my secret key is here']
}
}
game.rb
require 'aws/s3'
class Game < ActiveRecord::Base
attr_accessible :swf, :swf_file_name, :name, :description, :category, :age_group, :dimension_x, :dimension_y, :image, :image_file_name, :feature_image, :feature_image_file_name, :developer, :instructions, :date_to_go_live, :date_to_show_countdown, :plays
has_attached_file :swf
has_attached_file :image
has_attached_file :feature_image
def swfupload_file=(data)
data.content_type =
MIME::Types.type_for(data.original_filename).first.content_type
logger.warn("Data content type is: #{data.content_type}")
self.file = data
end
end
paperclip.rb
Paperclip::Attachment.default_options[:url] = ':s3_domain_url'
Paperclip::Attachment.default_options[:path] = '/:class/:attachment/:id_partition/:style/:filename'
Here is my paperclip initialization stuff:
Paperclip::Attachment.default_options.merge!({
storage: :s3,
s3_credentials: {
access_key_id: ENV['S3_KEY'],
secret_access_key: ENV['S3_SECRET'],
bucket: "#{ENV['S3_BUCKET']}-#{Rails.env}"
},
url: ":s3_domain_url",
path: "/:class/:attachment/:id_partition/:style/:filename"
})
This assumes that we have three environment variables setup called, you guessed it... S3_KEY, S3_SECRET, and S3_BUCKET. I did a little trick so that I could have a different bucket in each environment by adding Rails.env to the bucket variable.
You seem to indicate in your question that you're putting the actual name of the bucket in the reference to ENV, which would not work. You should put the name of the bucket in the environment variable and use the name of the environment variable as the key.
I hope this helps.

Resources