Carrierwave uploading with s3 and fog - ruby-on-rails

been trying to search the reason for this error for a long time and can't seem to find any...
So I have a rails app, and I utilize carrierwave for pictures uploading. I also want to utilize Amazon S3 for file upload storage in my app.
Initially as I am developing the app I allowed file uploads to be on the on :file, i.e.
image_uploader.rb
# Choose what kind of storage to use for this uploader:
storage :file
# storage :fog
Now upon finishing up development and placing it live (I use heroku), I decided to change the carrierwave storage to S3 to test it locally.
image_uploader.rb
# Choose what kind of storage to use for this uploader:
# storage :file
storage :fog
However, now when I try to upload a picture (be it for user avatar, etc) I get this error:
Excon::Errors::Forbidden in UsersController#update
Expected(200) <=> Actual(403 Forbidden)
request => {:connect_timeout=>60, :headers=>{"Content-Length"=>74577, "x-amz- acl"=>"private", "Content-Type"=>"image/png", "Date"=>"Sun, 26 Feb 2012 10:00:43 +0000", "Authorization"=>"AWS AKIAJOCDPFOU7UTT4HOQ:8ZnOy7X71nQAM87yraSI24Y5bSw=", "Host"=>"s3.amazonaws.com:443"}, :instrumentor_name=>"excon", :mock=>false, :read_timeout=>60, :retry_limit=>4, :ssl_verify_peer=>true, :write_timeout=>60, :host=>"s3.amazonaws.com", :path=>"/uploads//uploads%2Fuser%2Favatar%2F1%2Fjeffportraitmedium.png", :port=>"443", :query=>nil, :scheme=>"https", :body=>"\x89PNG\r\n\x1A\n\x00\x00\x00\rIHDR\x00\x00\x00\xC2\x00\x00\x00\xC3\b\x06\x00\x00\x00\xD0\xBD\xCE\x94\x00\x00\nCiCCPICC Profile\x00\x00x\x01\x9D\x96wTSY\x13\xC0\xEF{/\xBD\xD0\x12B\x91\x12z\rMJ\x00\x91\x12z\x91^E%$\
...
# The code you see above to the far right repeats itself a LOT
...
1#\x85\xB5\t\xFC_y~\xA6=:\xB2\xD0^\xBB~i\xBB\x82\x8F\x9B\xAF\xE7\x04m\xB2i\xFF\x17O\x94S\xF7l\x87\xA8&\x00\x00\x00\x00IEND\xAEB`\x82", :expects=>200, :idempotent=>true, :method=>"PUT"}
response => #<Excon::Response:0x007fc88ca9f3d8 #body="<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>8EFA56C0DDDC8878</RequestId><HostId>1OxWXppSSUq1MFjQwvnFptuCM3gKOuKdlQQyVSEgvzzv4Aj+r2hSFM2UUw2NYyrR</HostId></Error>", #headers={"x-amz-request-id"=>"8EFA56C0DDDC8878", "x-amz-id-2"=>"1OxWXppSSUq1MFjQwvnFptuCM3gKOuKdlQQyVSEgvzzv4Aj+r2hSFM2UUw2NYyrR", "Content-Type"=>"application/xml", "Transfer-Encoding"=>"chunked", "Date"=>"Sun, 26 Feb 2012 10:00:47 GMT", "Connection"=>"close", "Server"=>"AmazonS3"}, #status=403>
And then it says this as well for my application trace:
app/controllers/users_controller.rb:39:in `update'
And my REQUEST parameters:
{"utf8"=>"✓",
"_method"=>"put",
"authenticity_token"=>"DvADD1vYpCLcghq+EIOwVSjsfmAWCHhtA3VI5VGD/q8=",
"user"=>{"avatar"=>#<ActionDispatch::Http::UploadedFile:0x007fc88cde76f8
#original_filename="JeffPortraitMedium.png",
#content_type="image/png",
#headers="Content-Disposition: form-data; name=\"user[avatar]\";
filename=\"JeffPortraitMedium.png\"\r\nContent-Type: image/png\r\n",
#tempfile=#<File:/var/folders/vg/98nv58ss4v7gcbf8px_8dyqc0000gq/T/RackMultipart20120226- 19096-1ppu2sr>>,
"remote_avatar_url"=>"",
"name"=>"Jeff Lam ",
"email"=>"email#gmail.com",
"user_bio"=>"Tester Hello",
"shop"=>"1"},
"commit"=>"Update Changes",
"id"=>"1"}
Here's my users_controller.rb partial code:
def update
#user = User.find(params[:id])
if #user.update_attributes(params[:user])
redirect_back_or root_path
flash[:success] = "Your have updated your settings successfully."
else
flash.now[:error] = "Sorry! We are unable to update your settings. Please check your fields and try again."
render 'edit'
end
end
My image_uploader.rb code
# encoding: utf-8
class ImageUploader < CarrierWave::Uploader::Base
# Include RMagick or MiniMagick support:
# include CarrierWave::RMagick
include CarrierWave::MiniMagick
# Choose what kind of storage to use for this uploader:
# storage :file
storage :fog
# Override the directory where uploaded files will be stored.
# This is a sensible default for uploaders that are meant to be mounted:
def store_dir
"uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
end
# Provide a default URL as a default if there hasn't been a file uploaded:
# def default_url
# "/images/fallback/" + [version_name, "default.png"].compact.join('_')
# end
# Process files as they are uploaded:
# process :scale => [200, 300]
#
# def scale(width, height)
# # do something
# end
# Create different versions of your uploaded files:
version :thumb do
process resize_to_fill: [360, 250]
end
version :cover_photo_thumb do
process resize_to_fill: [1170, 400]
end
version :event do
process resize_to_fill: [550, 382]
end
version :product do
process resize_to_fit: [226, 316]
end
# Add a white list of extensions which are allowed to be uploaded.
# For images you might use something like this:
def extension_white_list
%w(jpg jpeg gif png)
end
# Override the filename of the uploaded files:
# Avoid using model.id or version_name here, see uploader/store.rb for details.
# def filename
# "something.jpg" if original_filename
# end
# fix for Heroku, unfortunately, it disables caching,
# see: https://github.com/jnicklas/carrierwave/wiki/How-to%3A-Make-Carrierwave-work-on-Heroku
def cache_dir
"#{Rails.root}/tmp/uploads"
end
end
Finally, my fog.rb file in the config/initializers
CarrierWave.configure do |config|
config.fog_credentials = {
:provider => 'AWS', # required
:aws_access_key_id => 'ACCESS_KEY', # required
:aws_secret_access_key => 'SECRET_ACCESS_KEY/ZN5SkOUtOEHd61/Cglq9', # required
:region => 'Singapore' # optional, defaults to 'us-east-1'
}
config.fog_directory = 'ruuva/' # required
config.fog_public = false # optional, defaults to true
end
I'm actually quite confused on some of the things in my fog.rb. Firstly, should I change my region to Singapore if I created a bucket called "ruuva", with region "Singapore" on my amazon s3 account?
Thank you to anyone that can help in advance!

First make sure you use the right credentials by not setting custom region and custom directory (create a fake bucket for free in the default region)
Then I think you are not using the right name for the region. Try setting your region like this:
:region => 'ap-southeast-1'

We were facing the same problem and fixed that changing the user's permission associated to your access key, changing it to "Power User". Check if you need your user to be power user before put it into productions.

Related

CKEDITOR + CARRIERWAVE + S3 no image

I can't figure out why the heck CKEDITOR is not displaying my images...I am using Carrierwave as my uploader to S3, which obviously have been set to all have their own 'uploader.rb' If I upload directly through Carrierwave I have no problems retrieving said image back from S3 to display. BUT if I upload said image through CKEDITOR or attach it inside CKEDITOR it will only display it's source link. This happens in both production and dev localhost.
uploaders/ckeditor_attachement_file_uploader.rb
# encoding: utf-8
require 'carrierwave'
class CkeditorAttachmentFileUploader < CarrierWave::Uploader::Base
include Ckeditor::Backend::CarrierWave
# Include RMagick or ImageScience support:
# include CarrierWave::RMagick
# include CarrierWave::MiniMagick
# include CarrierWave::ImageScience
# Choose what kind of storage to use for this uploader:
if Rails.env.production?
storage :fog
else
storage :file
end
# Override the directory where uploaded files will be stored.
# This is a sensible default for uploaders that are meant to be mounted:
def store_dir
"uploads/ckeditor/attachments/#{model.id}"
end
# Provide a default URL as a default if there hasn't been a file uploaded:
# def default_url
# "/images/fallback/" + [version_name, "default.png"].compact.join('_')
# end
# Process files as they are uploaded:
# process :scale => [200, 300]
#
# def scale(width, height)
# # do something
# end
# Add a white list of extensions which are allowed to be uploaded.
# For images you might use something like this:
def extension_white_list
Ckeditor.attachment_file_types
end
end
uploaders/ckeditor_picture_uploader.rb
# encoding: utf-8
class CkeditorPictureUploader < CarrierWave::Uploader::Base
include Ckeditor::Backend::CarrierWave
# Include RMagick or ImageScience support:
# include CarrierWave::RMagick
include CarrierWave::MiniMagick
# include CarrierWave::ImageScience
# Choose what kind of storage to use for this uploader:
if Rails.env.production?
storage :fog
else
storage :file
end
# Override the directory where uploaded files will be stored.
# This is a sensible default for uploaders that are meant to be mounted:
def store_dir
"uploads/ckeditor/pictures/#{model.id}"
end
# Provide a default URL as a default if there hasn't been a file uploaded:
# def default_url
# "/images/fallback/" + [version_name, "default.png"].compact.join('_')
# end
# Process files as they are uploaded:
# process scale: [200, 300]
#
# def scale(width, height)
# # do something
# end
process :extract_dimensions
# Create different versions of your uploaded files:
version :thumb do
process resize_to_fill: [118, 100]
end
version :content do
process resize_to_limit: [800, 800]
end
# Add a white list of extensions which are allowed to be uploaded.
# For images you might use something like this:
def extension_white_list
Ckeditor.image_file_types
end
end
models/picture.rb
class Ckeditor::Picture < Ckeditor::Asset
mount_uploader :data, CkeditorPictureUploader, mount_on: :data_file_name
def url_content
url(:content)
end
end
models/attachement_file.rb
class Ckeditor::AttachmentFile < Ckeditor::Asset
mount_uploader :data, CkeditorAttachmentFileUploader, mount_on: :data_file_name
def url_thumb
#url_thumb ||= Ckeditor::Utils.filethumb(filename)
end
end
I know I am answering my own question but hopefully this will help you. If you are reading this then you obviously have all your storage set up correctly such as S3 (in my case). Your views must require the html_safe method when attempting to display your contents via ckeditor ex.
<div class="col-xs-12">
<h1>Webur Blog</h1>
<h2><%= #blogpost.title %></h2>
<span><p><%= image_tag #blogpost.picture.url if #blogpost.picture? %></p></span>
<p><%= #blogpost.content.html_safe %></p>
<% if is_an_admin %>
<%= link_to "Return to blog", blogposts_path, class: 'btn btn-primary' %>
<%= link_to "Edit Post", edit_blogpost_path, class: 'btn btn-primary' %> |
<%= link_to "Delete Post", blogpost_path(#blogpost),
method: :delete, data: {confirm: "Are you sure?"}, class: 'btn btn-danger' %>
<% else %>
<%= link_to "Return to blog", blogposts_path, class: 'btn btn primary' %>
<% end %>
</div>
Hopefully this helps others. CKeditor does not mention this in their setup guide, and if you are a beginner such as myself you may not catch subtleties like this

Undefined method `<<` in image upload

def create
#product = Product.find(1)
#product.images << params[:image]
#product.save
end
The schema is
t.string "images", default: [], array: true
I want the assign the image coming in params to the images array, if the image is already there. The new image should append, otherwise it should be the first one
Here is the Product model
class Product < ActiveRecord::Base
mount_uploader :images, ProductUploader
validates :name, presence: true
validates :price, presence: true
validates :availability, presence: true
validates :about, presence: true
validates :ref, presence: true
validates :texture, presence: true
validates :shipping, presence: true
validates :category, presence: true
validates :notes, presence: true
end
I generated the ProductUploader through carrierwave uploader command
class ProductUploader < CarrierWave::Uploader::Base
# Include RMagick or MiniMagick support:
# include CarrierWave::RMagick
# include CarrierWave::MiniMagick
# Choose what kind of storage to use for this uploader:
storage :file
# storage :fog
# Override the directory where uploaded files will be stored.
# This is a sensible default for uploaders that are meant to be mounted:
def store_dir
"uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
end
# Provide a default URL as a default if there hasn't been a file uploaded:
# def default_url
# # For Rails 3.1+ asset pipeline compatibility:
# # ActionController::Base.helpers.asset_path("fallback/" + [version_name, "default.png"].compact.join('_'))
#
# "/images/fallback/" + [version_name, "default.png"].compact.join('_')
# end
# Process files as they are uploaded:
# process scale: [200, 300]
#
# def scale(width, height)
# # do something
# end
# Create different versions of your uploaded files:
# version :thumb do
# process resize_to_fit: [50, 50]
# end
# Add a white list of extensions which are allowed to be uploaded.
# For images you might use something like this:
# def extension_whitelist
# %w(jpg jpeg gif png)
# end
# Override the filename of the uploaded files:
# Avoid using model.id or version_name here, see uploader/store.rb for details.
# def filename
# "something.jpg" if original_filename
# end
end
After uploading the image, the image shows up in params. But does not get saved in the table. Here is the screenshot of byebug
Edit:
I have made some progress
I updated the string [] to json for images like this
And instead of files coming like string, it's coming as file. Now the only problem is, I am unable to access the name properly name=\"image[avatar_1]\" in the params
"utf8"=>"✓", "authenticity_token"=>"JbtcVdUvPJIzSHycX0wdr349zcATDN51vSewI33nvHIKQdhxxgxWUEJz9ZsubNrgvq2ftO+DosHXWM8Tipce7w==", "image"=>{"avatar_1"=>#<ActionDispatch::Http::UploadedFile:0x007f2ba8377f78 #tempfile=#<Tempfile:/tmp/RackMultipart20160704-20497-1czlw3s.png>, #original_filename="no-save.png", #content_type="image/png", #headers="Content-Disposition: form-data; name=\"image[avatar_1]\"; filename=\"no-save.png\"\r\nContent-Type: image/png\r\n">}, "commit"=>"submit", "controller"=>"upload", "action"=>"create"}
Here is the view file
<%= simple_form_for :image, method: :post, url: save_image_path do |f| %>
<%= f.file_field :avatar_1 %>
<%= f.submit :submit %>
<% end %>
here it is avatar_1. I need to change it in create code, but I am not accessing params itself properly
Let's divide your question into subproblems:
Database & controller
Your database and controller are working properly. No need to change anything!
Uploaded image
You uploaded image (in params[:image]) is a String, instead of a ActionDispatch::Http::UploadedFile. You should probably add the multipart form data to your form.
Change filename
You can access the uploaded filename with params[:image].original_filename. If you want to change the filename, then you can redefine the filename method for your CarrierWave handler (the last one, which is commented).

Rails 3 + Carrierwave: File Failed to manipulate with MiniMagick, maybe it is not an image? Original Error: MiniMagick::Invalid

Hi I'm trying to set up carrierwave for my Rails project. I've looked at other answers but nothing seems to work. When I try to create an image it gives me this:
1.9.3p392 :003 > bi = BookCoverImage.new
=> #<BookCoverImage id: nil, file: nil, created_at: nil, updated_at: nil, book_id: nil, category_id: nil, collection_id: nil, creator_id: nil, group: "BookCoverImage", homepage_slide_id: nil>
1.9.3p392 :004 > bi.file = File.open("#{Rails.root}/spec/support/sample_img.jpg")
=> #<File:/Users/emai/Documents/mysite/spec/support/sample_img.jpg>
1.9.3p392 :005 > bi.save
(0.1ms) SAVEPOINT active_record_1
(0.1ms) ROLLBACK TO SAVEPOINT active_record_1
=> false
1.9.3p392 :006 > bi.valid?
=> false
1.9.3p392 :007 > bi.errors.full_messages
=> ["File Failed to manipulate with MiniMagick, maybe it is not an image? Original Error: MiniMagick::Invalid"]
I'm using Ruby 1.9.3, Mac Mavericks, and homebrew. Can anyone help?
BookCoverImage model:
class BookCoverImage < Image
default_scope { where(group: self.name) }
belongs_to :book
mount_uploader :file, BookCoverImageUploader
end
It inherits from Image:
class Image < ActiveRecord::Base
attr_accessible :file
after_initialize :set_group
before_save :set_group
private
def set_group
self.group = self.class.name
end
end
My BookCoverImageUploader:
# encoding: utf-8
class BookCoverImageUploader < CarrierWave::Uploader::Base
# Include RMagick or MiniMagick support:
# include CarrierWave::RMagick
include CarrierWave::MiniMagick
# Choose what kind of storage to use for this uploader:
storage :file
# storage :fog
# Override the directory where uploaded files will be stored.
# This is a sensible default for uploaders that are meant to be mounted:
def store_dir
if Rails.env.test? || Rails.env.cucumber?
"#{Rails.root}/public/test/file_uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
else
"uploads/#{model.class.to_s.underscore}/#{model.id}"
end
end
# Provide a default URL as a default if there hasn't been a file uploaded:
# def default_url
# # For Rails 3.1+ asset pipeline compatibility:
# # ActionController::Base.helpers.asset_path("fallback/" + [version_name, "default.png"].compact.join('_'))
#
# "/images/fallback/" + [version_name, "default.png"].compact.join('_')
# end
# Process files as they are uploaded:
# process :scale => [200, 300]
#
# def scale(width, height)
# # do something
# end
# Create different versions of your uploaded files:
version :thumb do
process :resize_to_fit => [80, 10000]
end
version :medium do
process :resize_to_fit => [159, 10000]
end
version :main do
process :resize_to_fit => [238, 10000]
end
# Add a white list of extensions which are allowed to be uploaded.
# For images you might use something like this:
def extension_white_list
%w(jpg jpeg gif png)
end
# Override the filename of the uploaded files:
# Avoid using model.id or version_name here, see uploader/store.rb for details.
# def filename
# "something.jpg" if original_filename
# end
end
The error File Failed to manipulate with MiniMagick, maybe it is not an image? Original Error: MiniMagick::Invalid is thrown if ImageMagick fails to identify the given file as an image.
Try this on the command prompt: (Replace with your filename)
> identify spec/support/facebook.jpg
### If image identified correctly, outputs as below
spec/support/facebook.jpg JPEG 400x267 400x267+0+0 8-bit sRGB 13.4KB 0.000u 0:00.000
> identify spec/support/test.png
### If image is invalid, outputs as below
identify: improper image header `spec/support/test.png' # error/png.c/ReadPNGImage/4003.

Carrierwave Reset after Reboot

Both the app and db (mongodb) servers were rebooted last night. All carrierwave mounted uploaders are returning the default images for avatars, even though the files still exist.
I am using fog storage on Rackspace CDN. Each user model contains a field of avatar_filename. I tried running user.avatar.recreate_versions! however that errors out due to nil.
Is there any way to restore my images (they still exist!) and prevent this from happening again? I have searched around but it doesn't look like this is a common prom.
In my user model:
# Avatar
mount_uploader :avatar, AvatarUploader
AvatarUploader:
class AvatarUploader < CarrierWave::Uploader::Base
include CarrierWave::RMagick
storage :fog
def default_url
"/assets/users/profile-default_#{version_name}.png"
end
# Large
version :large do
resize_to_limit(600, 600)
end
# Small
version :small do
process :crop
resize_to_fill(140, 140)
end
# Thumbnail
version :thumb, :from_version => :small do
resize_to_fill(35, 35)
end
def extension_white_list
%w(jpg jpeg png)
end
def filename
if #filename_created
#filename_created
elsif original_filename
#name ||= Digest::MD5.hexdigest(File.dirname(current_path))
#filename_created = "a_#{timestamp}_#{#name}.#{file.extension}"
#filename_created
end
end
def timestamp
var = :"##{mounted_as}_timestamp"
model.instance_variable_get(var) or model.instance_variable_set(var, Time.now.to_i)
end
def crop
if model.crop_x.present?
resize_to_limit(600, 600)
manipulate! do |img|
x = model.crop_x.to_i
y = model.crop_y.to_i
w = model.crop_w.to_i
h = model.crop_h.to_i
img.crop!(x, y, w, h)
end
end
end
end
Given that the images are there, you could reupload them as remote files with user.remote_avatar_url = "the url for this avatar"
To avoid this in the future you have to keep in mind how you are processing the file name. That process is reapplied each time you do recreate_versions!. Put this code in your uploader to get around this:
class AvatarUploader < CarrierWave::Uploader::Base
def filename
if original_filename
if model && model.read_attribute(:avatar).present?
model.read_attribute(:avatar)
else
# new filename
end
end
end
end
You can find more information about this in the following wiki article: https://github.com/jnicklas/carrierwave/wiki/How-to%3A-Create-random-and-unique-filenames-for-all-versioned-files

Amazon access key showing in URL for Carrierwave and Fog

I just switched from storing my images uploaded via Carrierwave locally to using Amazon s3 via the fog gem in my Rails 3.1 app. While images are being added, when I click on an image in my application, the URL is providing my access key and a signature. Here is a sample URL (XXX replaced the string with the info):
https://s3.amazonaws.com/bucketname/uploads/photo/image/2/IMG_4842.jpg?AWSAccessKeyId=XXX&Signature=XXX%3D&Expires=1332093418
This is happening in development (localhost:3000) and when I am using heroku for production. Here is my uploader:
class ImageUploader < CarrierWave::Uploader::Base
include CarrierWave::RMagick
storage :fog
def store_dir
"uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
end
process :convert => :jpg
process :resize_to_limit => [640, 640]
version :thumb do
process :convert => :jpg
process :resize_to_fill => [280, 205]
end
version :avatar do
process :convert => :jpg
process :resize_to_fill => [120, 120]
end
end
And my config/initializers/fog.rb :
CarrierWave.configure do |config|
config.fog_credentials = {
:provider => 'AWS',
:aws_access_key_id => 'XXX',
:aws_secret_access_key => 'XXX',
}
config.fog_directory = 'bucketname'
config.fog_public = false
end
Anyone know how to make sure this information isn't available?
UPDATE: Adding view and controller code:
from a partial in users/show.html.erb:
<% if #user.photos.any? %>
<% for photo in #user.photos %>
<li class="span4 hidey">
<div class="thumb_box">
<%=link_to(image_tag(photo.image_url(:thumb).to_s), photo.image_url.to_s,
:class=>"lb_test") %>
...
</div>
</li>
<% end %>
<% end %>
users_controller.rb:
def show
#user = User.find(params[:id])
end
UPDATE: Adding an error page I get when removing the access key information from the url:
This XML file does not appear to have any style information associated with it. The document tree is shown below.
<Error>
<Code>AccessDenied</Code>
<Message>Access Denied</Message>
<RequestId>47077D6EC13AD1D8</RequestId>
<HostId>+HTeODcWTqv3gbRIAwf+lI6sPzfNTegDXjT9SnMdqrYr7gLD1TD0qN+OgMLwA1JO
</HostId>
</Error>
Remove
config.fog_public = false
That's a non-default value :)
What you are seeing is a signed-url. Without the full url (including key,signature,expires), you'll get an access denied. It is working exactly as it should. And I am guessing the key is just a public key, that is useless without your private key (which AWS has).
Try photo.image.url instead of photo.image_url. That's what I'm using.

Resources