We've recently moved to a new webhost that offers limited storage space so we're trying to move all of our user uploads (Avatars, Files, etc.) uploaded via the Paperclip gem to Amazon S3. I have several hundred files all corresponding to different models that I'm now trying to migrate en masse.
I found this document that introduces a nifty paperclip rake task:
rake paperclip:refresh:missing_styles
This command does some of the work for me, however, I noticed it's only setting up the files structure without sending any data - in addition it's not setting up any thumbnails defined using the :styles hash in the has_attached_file call. I.e., I have the following paperclip setup on one of my models:
class User < ActiveRecord::Base
has_attached_file :avatar,
:styles => {
:thumb => "100x100#",
:small => "150x150>",
:medium => "200x200" }
end
Here's some sample output after running the command:
$ rake paperclip:refresh:missing_styles
Regenerating User -> avatar -> [thumb, :small, :medium]
Regenerating Mercury::Image -> image -> [:medium, :thumb]
Regenerating Profile -> image -> [:home_feature, :large, :medium, :thumb]
Regenerating Page -> preview -> [:portfolio]
Regenerating Category -> default_image -> [:home_feature, :large, :medium, :thumb]
Navigating to my S3 Bucket I can see all the directories are correctly setup and for each attachment, but only for original image files and they're all 0 bytes. Am I misunderstanding the usage of this command? I couldn't find any other tool for uploading entire directories of files in bulk to S3, if there's a safe tool out there that already covers this without requiring payment then I'm open ears. I've tried building a ruby script to plug into their SDK and upload these files manually, but their Ruby documentation isn't great.
Looks like I needed to go beyond ruby on this one, s3cmd seemed to be the most appropriate tool for this sort of job. In my case, the sync command did the trick:
s3cmd sync my-app/public/system/ s3://mybucket
Related
I have a Rails 4 application that uses paperclip to attach photos. My db/seeds.rb file adds some photos for my Person model with lines like this:
Person.create(:first_name => 'Jon', :last_name => 'Snow',
:photo => File.open("#{Rails.root}/app/assets/images/jon-snow.png))
In my app/models/person.rb file I have the :photo as a paperclip attachment where it is cropped and resized:
class Person < ActiveRecord::Base
has_attached_file :photo,
:styles => { :medium => "256x256#", :small => "64x64#", :tiny => "24x24#" },
:default_url => :set_default_avatar,
:url => "/assets/photos/:id/:style/:basename.:extension",
:path => ":rails_root/public/assets/photos/:id/:style/:basename.:extension"
validates_attachment_size :photo, :less_than => 5.megabytes
validates_attachment_content_type :photo, :content_type => ['image/jpeg', 'image/png']
My problem is that when I try to deploy this to heroku (with the Cedar stack) and then seed the database, the images all come up as broken. However, on my local computer everything comes up completely fine.
The broken image tag that is generated on heroku might look something like:
<img alt="Jon Snow" src="/assets/photos/21/small/jon-snow.png?1386825683">
Does anyone know why this link would break on heroku but not on my computer? I know that heroku is generating the resized photos because the heroku console outputs things similar to what is in this post: Seed images in heroku with paperclip.
However, it is putting them into public/assets instead of in public/assets/photos/:id/:style/:basename.:extension as the controller specifies.
I've tried doing what that linked post mentions, as well as running:
heroku run rake assets:precompile
and a number of other things, but nothing works. Any help would be greatly appreciated.
The issue is the same I described in this answer How to use paperclip with rails and how does it work in deployment?
The legacy Bamboo stack had a read-only file system so you were unable to write on the file-system.
In the new Cedar stack, the file system is no longer read-only, so the upload will not fail. However, you should keep using AWS or any other external storage because Heroku distributes your compiled application across several machines, and it's not guaranteed that the image will be visible from another request. They call it ephemeral filesystem.
In other words, you should keep using AWS or any other storage outside Heroku file-system.
From heroku-dev
The filesystem for the slug is read-only, which means you cannot
dynamically write to the filesystem for semi-permanent storage. The
following types of behaviors are not supported: Caching pages in the
public directory Saving uploaded assets to local disk (e.g. with
attachment_fu or paperclip)
Use AWS S3
For Ruby on Rails I want to use
<img src =../../../photo/1.PNG >
to show some images stored outside my app folder. I use this and use .. to point out where image it it , But it does not work。 I use Win7, Is there anyone could help me ?
I have a MVC named "showcase" generated by scaffold, and I wanna to have the same showcase in two apps.
In app1: showcase (where I wanna to share the images)model: house is the image name.
class Showcase < ActiveRecord::Base
attr_accessible :house,:sequence
has_attached_file :house, :whiny => false,
:styles => { :large => "350x300>",
:medium => "120x100>",
:thumb => "50x50>"}
end
then the images will be uploaded in the folder: public/systems/showcases/houses/000/000/0001/1.PNG
in app2. I create a showcase model with migration. and use ActiveResources share the database between app1 and app2. but app2 can not configure paperclip type.
class Showcase < ActiveResource::Base
self.site = "http://localhost:3000"
end
how can I in app2, call images uploaded from app1.
If your trying to really share images between applications, and they are uploads and not static assets then create a systems folder somewhere else, then make the system folders in each app a symlink to the share folder. That way your not having crazy url paths, and they both can asses it, and you wont end up with dupes or mismatches.
You should really use the rails assets pipeline if you are using rails 3 or above. Put your images in app/assets/images and call the rails functions that gives you url to the assets e.g. <%= image_tag "rails.png" %>.
For more detail read the following guideline:
http://guides.rubyonrails.org/asset_pipeline.html
I just upgraded to Rails 3.2 and I'm using Amazon S3 with Paperclip to upload photos to my app.
Before my Image Urls would be:
http://s3.amazonaws.com/dealphotos.website.com/photos/428/large/Sandisk120Drive?1334754504
Now my Image Urls on Localhost are:
http://s3.amazonaws.com/dealphotos.website.com/deals/photos/000/000/428/large/Sandisk120Drive?1334754504
Notice the additional 000/000's - even if I take them out and visit the link it says:
<Error>
<Code>AccessDenied</Code>
<Message>Access Denied</Message>
<RequestId>"Long String of Numbers"</RequestId>
<HostId>
"Gives me a really long string of letters and numbers"
</HostId>
My Deal Model:
has_attached_file :photo, :styles => { :small =>"268x160>", :large =>"350x250>" },
:storage => :s3,
:bucket => 'dealphotos.website.com',
:s3_credentials => {
:access_key_id => ENV['S3_KEY_SPICY'],
:secret_access_key => ENV['S3_SECRET_SPICY']
}
And the images dont show up!
Before all I had to do was pull from Heroku and all my images and files would go to my development. What's going on?
You are most likely using the latest version of Paperclip (version >= 3).
Since this is a major version of paperclip there are documented incompatibilities with older versions of paperclip.
One MAJOR (and in my books good) change is that the default storage location of assets has changed.
Earlier it used to be /system/:attachment/:id/:style/:filename.:extension (from memory). This would be disastrous if you had 2 attachments with the same name (Company logo and Product logo) for example.
Now the new path used by paperclip is /system/:class/:attachment/:id_partition/:style/:filename.:extension
This means all your assets will be stored in the directory specified by above. :id_partitions are used so that the number of nodes (files/directories) in one directory doesnt exceed 1000.
I hope i've been able to explain the reason WHY you are seeing the problem. The default path has changed.
You have 2 options now -
Move old assets into the correct place in the new directory scheme (painful but recommended).
Move new assets into the old directory structure and add a :url + :path option in your attachment definition in the model to continue using the older scheme. (See https://github.com/thoughtbot/paperclip/blob/master/UPGRADING ).
Since it appears that you have images in BOTH the new structure and the old one. So no matter what you decide files will need to be moved from one structure to another. Unless the assets are bookmarked in some way, I suggest you move the older assets into the new directory structure.
Check out CarrierWave gem to file uploads. It works with 3.2 without any problems ;)
http://railscasts.com/episodes/253-carrierwave-file-uploads
I'm working on an app that allows users to upload videos, pictures, audio clips, and documents. The files are being uploaded with the Paperclip gem and being stored on s3.
Right now I am running some processing on the file sizes after they are uploaded, but paperclip continues to save the file to the database with a file size of '0' on any file that is not .jpg, .gif, .png. I've tried inspecting the file size myself, the upload's length, and several other hair brained ideas.
Is there anyway to monkeypatch, or hijack the request from paperclip to find out what the file size is? Has anyone run into this issue before?
By default, Paperclip is built to process images. If you need to be able to handle other file types, you'll have to look into writing your own Processor: https://github.com/thoughtbot/paperclip/blob/master/lib/paperclip/processor.rb. This will allow you to process files in whatever way you need when declared in your model:
has_attached_file :scan, :styles => { :text => { :quality => :better } },
:processors => [:ocr]
Here's an important point from the README docs:
NOTE: Because processors operate by turning the original attachment
into the styles, no processors will be run if there are no styles
defined.
This means you'll need to specify some sort of styles in order for your processor or any other processor to get run. For a basic custom processor something as simple as :styles => { :default => true } should be enough to get you going.
I've been trying to get paperclip to upload images to amazon s3, but all I get is the original file being uploaded. No thumbnails are generated. My model has this in it:
has_attached_file :screenshot, :styles => {
:thumb => "100x80>",
:medium => "195x150>",
:large => "390x300>"
},
:storage => :s3,
:s3_credentials => "#{RAILS_ROOT}/config/s3.yml",
:path => ":attachment/:id/:style.:extension"
The original file is in fact uploaded, but none of thumbnails appear. If I copy the src of a thumb format image, for instance, I get
<Error>
<Code>NoSuchKey</Code>
<Message>The specified key does not exist.</Message>
<Key>screenshots/8/thumb.png</Key>
<RequestId>B8A408560070E836</RequestId>
−
<HostId>
HAHUCuNxGKVDvqE3GnhGT1RcBgFGUxa35kqXmyRE+ii60DJS6r22ChDr22cOeCea
</HostId>
</Error>
This is running on heroku, which forces me to use a service like amazon. Not a problem but all the documentation I can find makes this look straightforward. I can't find any dependencies (bundler is happy), and I've been using AWS for 2 years now, without these problems, on another site, with a different bucket on the same account. Is it a problem of connecting the bucket to the account? I'm using the same credentials as in the other site. Only the bucket changes.
A couple more things. The directories that are being created in the path of the image that is being uploaded are permissions 000 or whatever the aws equivalent is. Nobody is permitted to do anything.
using Rails 3.
I had the same problem and the answer lies in:
:path => ":attachment/:id/:style.:extension"
Instead of storing the images in the same folder_id with different names, use the same name for the image and use the style for folders. The above would translate into:
:path => ":attachment/:id/:style/:filename"
So instead of trying to generate something like:
avatars/13/thumb.png
avatars/13/medium.png
avatars/13/large.png
You get this:
avatars/13/thumb/my_pic.png
avatars/13/medium/my_pic.png
avatars/13/large/my_pic.png
I installed paperclip as a plugin, and everything works.
Don't forget to remove the gem from your Gemfile.