I have the code below to save my local file to AWS S3. It uploads my file to S3 correctly, but then my local file is not deleted. Now I've filled disk space completely. How can I ask paperclip to remove local file after it has uploaded it?
class JobArtifact < ActiveRecord::Base
attr_reader :remote_url
has_attached_file :file, path: 'tmp/:id/:fingerprint.:extension'
do_not_validate_attachment_file_type :file
def remote_url=(url)
self.file = URI.parse(url_value)
#remote_url = url
end
end
It is called this way:
#filename = "#{Rails.root}/tmp/values.csv"
JobArtifact.create(file: File.open(#filename))
Since you know the path of the file you can just do something like this after it's uploaded to s3 -
def remove_file
File.delete(#filename) if File.exist?(#filename)
end
Related
Carrierwave's enter link description here gives both the directory where Carrierwave uploads files and the directory where Carrierwave looks for files. However, I'd like to know if there's a way to define something like a read_dir, which would be the path where Carrierwave looked for files, and leave the store_dir only for storing files.
I know this probably doesn't make much practical sense, but I'd just like to know.
You can calculate store_dir based on some condition of the object. You can also set this condition when you need a different directory. For example:
class MyModel < ActiveRecord::Base
attr_accessible :use_directory_for_storing
mount_uploader :file, MyFileUploader
end
class MyFileUploader < CarrierWave::Uploader::Base
# ...
def store_dir
if model.use_directory_for_storing
"some/directory/for/storing"
else
"some/directory/for/reading"
end
end
end
# Usage
object = MyModel.new(params)
object.use_directory_for_storing = true
object.save # the file will be stored in ".../some/directory/for/storing/" directory
object = MyModel.last
# will look for the file in the ".../some/directory/for/reading/" directory
object.file.path # => ".../some/directory/for/reading/..."
I'm using Carrierwave and Fog gems to store a file to my Amazon S3 bucket (to /files/file_id.txt). I need to store a slightly different version of the file to a different location in the bucket (/files/file_id_processed.txt) at the same time (right after the original is stored). I don't want to create a separate uploader attribute for it on the model - is there any other way?
This my current method that stores the file:
def store_file(document)
file_name = "tmp/#{document.id}.txt"
File.open(file_name, 'w') do |f|
document_content = document.content
f.puts document_content
document.raw_export.store!(f)
document.save
end
# I need to store the document.processed_content
File.delete(file_name) if File.exist?(file_name)
end
This is the Document model:
class Document < ActiveRecord::Base
mount_uploader :raw_export, DocumentUploader
# here I want to avoid adding something like:
# mount_uploader :processed_export, DocumentUploader
end
This is my Uploader class:
class DocumentUploader < CarrierWave::Uploader::Base
storage :fog
def store_dir
"files/"
end
def extension_white_list
%w(txt)
end
end
This is how my final solution looks like (kinda) - based on Nitin Verma's answer:
I had to add a custom processor method for the version to the Uploader class:
# in document_uploader.rb
...
version :processed do
process :do_the_replacements
end
def do_the_replacements
original_content = #file.read
File.open(current_path, 'w') do |f|
f.puts original_content.gsub('Apples','Pears')
end
end
considering that you need similar file but with different name.
for this you need to create a version for file in uploader.
version :processed do
process
end
and now second file name will be processed_{origional_file}.extension. if you want to change file name of second file you can use this link https://github.com/carrierwaveuploader/carrierwave/wiki/How-to:-Customize-your-version-file-names
I have an external service that creates files and stores them into S3 (that my Rails app has access too).
I want to be able to use Paperclip in order to store that S3 object, as well as a thumbnail of it. I haven't been able to find documentation on how to use Paperclip with files that are already on S3. So, my code should look like:
page = Page.find(3)
page.s3_url = s3_url # I have the s3 route. And I would need to generate a thumbnail too.
page.save
So, in other words:
How can I tell PaperClip that my attachment is already in S3?
EDIT:
Here is what I have so far:
I know the file_name, the content_length and the content_type of the file that is already uploaded in S3. So, since the association is a Page has_attached_file :screenshot
This is what I do:
#page = Page.find(3)
#page.update_attributes(screenshot_content_type: screenshot_content_type, screenshot_file_size: screenshot_file_size, screenshot_update_at: screenshot_updated_at, screenshot_file_name: screenshot_file_name)
So, now I can do:
#page.screenshot and I see the paperclip object. However, when I do:
#page.screenshot.url => The url is not the one that I originally stored the image.
I've managed to solve this problem with paperclip interpolations:
Define in your model a field, witch would store path to the uploaded files
Load the paperclip attachment info to DB through property update
With interpolations specify a paperclip attachment :path, so it first looks for uploaded file, and then for default
Profit!
This will do the trick, because of how S3 storage composes urls:
path: This is the key under the bucket in which the file will be stored. The URL will be constructed from the bucket and the path. This is what you will want to interpolate. Keys should be unique, like filenames, and despite the fact that S3 (strictly speaking) does not support directories, you can still use a / to separate parts of your file name.
Here are more details with code:
Model
class MyModel
has_attached_file :file, path: ":existent_file_or_default", storage: :s3
end
Paperclip interpolations
Put this under config/initializers
Paperclip.interpolates :existent_file_or_default do |attachment, style|
attachment.instance.existent_file_path ||
attachment.interpolator.interpolate(":class/:attachment/:id_partition/:style/:filename", attachment, style)
end
Attach existent items
MyModel.create({
existent_file_path: "http://your-aws-region.amazonaws.com/your-bucket/path/to/existent/file.jpeg",
file_file_name: "some_pretty_file_name.jpeg",
file_content_type: "image/jpeg",
file_file_size: 123456
})
S3
Paperclip S3 integration is actually relatively simple - you'll be best looking at this Railscast on how to use Paperclip, and the afore-linked documentation to give you an idea on how Paperclip works; then you can just connect it to S3
--
In short, Paperclip basically takes a file object & saves the relevant data to your db. The storage of the file is dependent on the service you associate with Paperclip
What I'm trying to say is the two elements (data allocation & file storage) are two different elements of the gem, and if you can get Paperclip to handle the inbound files, you're half-way there:
Paperclip
Default implementation:
#app/models/page.rb
Class Page < ActiveRecord::Base
has_attached_file :photo
end
This will allow you to save an "attachment" to your Page database:
#app/controllers/pages_controller.rb
Class PagesController < ApplicationController
def new
#page = Page.new
end
def create
#page = Page.new(page_params)
end
private
def page_params
params.require(:page).permit(:title, :other, :information, :photo)
end
end
#app/views/pages/new.html.erb
<%= form_for #page do |f| %>
<%= f.file_field :photo %>
<%= f.submit %>
<% end %>
This will handle Paperclip uploads directly to your own app (storing the files in the public/system directory)
In order to get it to work with S3, you just have to add the S3 credentials to the model (taken from the Paperclip documentation):
#app/models/page.rb
Class Page < ActiveRecord::Base
has_attached_file :photo,
:storage => :s3,
:s3_credentials => Proc.new{|a| a.instance.s3_credentials }
def s3_credentials
{:bucket => "xxx", :access_key_id => "xxx", :secret_access_key => "xxx"}
end
end
Path
If you'd like to call the "S3" filepath for your file - you'll have to do this:
#app/controllers/pages_controller.rb
Class PagesController < ApplicationController
def show
#page = Page.find 3
#page.photo.url #-> yields S3 path ;)
end
end
I have model:
class RealtyPhoto < ActiveRecord::Base
attr_accessible :name, :photo
mount_uploader :photo, RealtyPhotoUploader
belongs_to :realty_object
end
And uploader:
class RealtyPhotoUploader < CarrierWave::Uploader::Base
include CarrierWave::RMagick
storage :file
def store_dir
"uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
end
def default_url
[version_name, "/assets/default-flat.jpg"].compact.join('_')
end
def extension_white_list
%w(jpg jpeg gif png)
end
end
I write in my code something like this:
require 'open-uri'
object = RealtyObject.last
photo = RealtyPhoto.new
photo.photo = open('http://milushov.ru/bg.jpg').read
object.realty_photos << photo
And i get an error on that line:
photo.photo = open('http://milushov.ru/bg.jpg').read
ArgumentError: string contains null byte
from /home/roma/.rvm/gems/ruby-1.9.3-p327#miel/gems/carrierwave-0.7.1/lib/carrierwave/sanitized_file.rb:113:in `expand_path'
Does anybody know how to download and save file from outside? (when i save file from form on the page - all ok). Maybe using CarrierWave Uploader?
Carrierwave upload image from remote url using remote_column_name_url in your case
photo.remote_photo_url = "url"
You seemed to have forgotten to close the file handle. Closing the file will likely make your application work correctly.
you could also do it in another way..
more efficient is
model[:name] = filename
model.save
surl = model.photo.url
FileUtils.copy(filesrc,surl,derefrence=true)
this approach basically saves you from copying the file to one location and ten copying it to another location when saved
I am uploading image via jquery_file_upload (Railscast episode #383)
The upload work perfectly and i get the url in the callback function
The problem is: I want to create some thumbnails after uploading directly to s3
for that i assume:
The server has to read the image from the s3
The server has to create the thumbnails
The server has to store the thumbnails directly in s3
The server has to get callback and display the thumbnails accordingly.
The server has to store a field the the thumbnails created
Now, after the image finish to upload the image controller callback called:
def create
#get the url from the callbcack
#e.g: image[image_source] = https://<BUCKET>.s3.amazonaws.com/uploads/myimage.png
#image = Image.new(params[:image])
#image.user_id = current_user.id
#image.save
end
And the model:
class Image < ActiveRecord::Base
attr_accessible :image_thumb, :user_id, :width ,:album_type_id
after_save :enqueue_image
mount_uploader :image_thumb, ImageUploader
def enqueue_image
#checking if i have the original image
if self.image_source present?
#what to do here?
#how can i call the carrierwave function to create the thumbnails?
#how to send the thumbnails directly to s3 with callback name?
end
end
end
Do you necessarily need to create the thumbnails after uploading? If not, I think that you can create a thumbnail (or as many sizes as you want) during the initial upload by using version. Example:
class ImageUploader < CarrierWave::Uploader::Base
version :thumb do
process :resize_to_fill => [50, 50]
end
end