Importing old data with Rails and Paperclip - ruby-on-rails

I'm using paperclip for attachments in my application. I'm writing an import script for a bunch of old data, but I don't know how to create paperclip objects from files on disk. My first guess is to create mock CGI multipart objects, but that seems like a bit of a crude solution, and my initial attempt failed, I think because I didn't get the to_tempfile method right.
Is there a Right Way to do this? It seems like something that should be fairly easy.

I know that I've done the same thing, and I believe that I just created a File object from the path to each file, and assigned it to the image attribute. Paperclip will run on that file:
thing.image = File.new("/path/to/file.png")
thing.save

This works great for local files but it doesn't work as well for remote files. I have an app that uses paperclip for uploading images. Those images are getting stored on amazon s3. Anyway, I had some old data that I needed to import so I tried the following:
thing.image = open('http://www.someurl.com/path/to/image.jpg')
thing.save
If the file is small (say, less than 10K) then openuri returns a stringio object and my file would get stored on s3 as stringio.txt
If the file is larger than around 10K, then openuri returns a TempFile object. But the filename on s3 ends up being unique, but not really relating to the original filename of image.jpg
I was able to fix the problem by doing the following:
remote_photo = open('http://www.someurl.com/path/to/image.jpg')
def remote_photo.original_filename;base_uri.path.split('/').last; end
thing.image = remote_photo
thing.save

Related

Carrierwave + Fog + S3 remove file without going through a model

I am building an application that has a chat component to it. The application allows users to upload files to the chat. The chat is all javascript but i wanted to use Carrierwave for the uploads because i am using it elsewhere in the application. I am doing the handling of the uploads through AJAX so that i can get into Rails land and let Carrierwave take over.
I have been able to get the chat to successfully upload the files to the correct location in my S3 bucket. The thing i can't figure out is how to delete the files. Here is my code the uploads the files - this is the method that is called from the route that the AJAX call hits.
def upload
file = File.open(params[:file_0].tempfile)
uploader = ChatUploader.new
uploader.store!(file)
end
There is little to no documentation with Carrierwave on how to upload files without going through a model and basically NO documentation on how to remove files without going through a model. I assume it is possible though - i just need to know what to call. So i guess my question is how do i delete files?
UPDATE (11/23)
I got the code to save and delete files from S3 using these methods:
# code to save the file
def upload
file = File.open(params[:file_0].tempfile)
uploader = ChatUploader.new
uploader.store!(file)
uploader.store_path()
end
# code to remove files
def remove_file
file = params[:file]
uploader = ChatUploader.new
uploader.retrieve_from_store!(file)
uploader.remove!
end
My only issue now is that the filename for the uploaded file is not correct. It saves all files with a "RackMultipart" and then some numbers which look like a date, time, and identifier? (example: RackMultipart20141123-17740-1tq4j1g) Need to try and use the original filename plus maybe a timestamp for uniqueness.
I believe it has something to do with these two lines:
file = File.open(params[:file_0].tempfile)
and
uploader.store!(file)

carrierwave: point to existing image

In my rails app, I'm using Carrierwave to upload images on Amazon S3. I'd like to point to existing Amazon S3 images without having to re-upload the image. For example, if I have an existing Amazon S3 image at http://test.s3.amazonaws.com/image/path/0001/image.jpg, can I update an image's path to point to this image? I don't want to use the remote upload option because I really just want to use the same exact image that's already there (but save it in my record's "path" attribute).
In the console, I've tried:
image.update_attributes(:path=> "http://test.s3.amazonaws.com/image/path/0001/image.jpg")
but this fails to override the image's path.
Chiming in, better late than never! Caveat: This is for rails 4, and I am testing on rails 4.1 only at the moment.
This is harder than it should be, methinks! The reason this was absolutely crucial to me was that I am attaching 100MB+ MP3 files, which I cannot receive on my host, due to CloudFlare SSL limitations (and common sense). Fortunately, AWS supports preauthorized uploads, and I got carrierwave to do the right thing for me:
Step 1: get carrierwave to tell me where it would store a file if it could:
m.raw_write_attribute('file','file.mp3');
url = m.file.url
signed = aws_presigned_url(url)
raw_write_attribute does not save anything, it just bypasses carrierwave when setting the value. This makes the object act as if it read 'file.mp3' out of the database. Then you can ask Carrierwave "where the file lives". I then upload the file directly from the client to S3. When that's done, I make another API call to Rails, which performs the following code:
m.raw_write_attribute('file','file.mp3');
m.update_attribute('file','file.mp3');
These two paired get around Carrierwave. The first makes carrierwave think that the 'file' column is set to 'file.mp3', the second explicitly tells rails to persist 'file.mp3' to the DB. Because of the raw_write_attribute call, Carrierwave allows the second through un-changed.
In my case update_column and update_columns worked great:
model.update_columns file_1: 'filename.txt'
Update column is with comma:
model.update_column :file_1, 'filename.txt'
This will not run any callback and set column to filename.txt.
When I do model.file_1.url I get the right S3 URL.
I am a bit late to the party, but you can use Active Record's raw_write_attribute method something like:
#image.raw_write_attribute(:path, "http://test.s3.amazonaws.com/image/path/0001/image.jpg")
I found that you can actually do this, for example if your mount_uploader is :path, then:
image.remote_path_url = "http://test.s3.amazonaws.com/image/path/0001/image.jpg"
image.save

Carrierware: store file in directories accordint to created_at date

I am using carrierwave to handle my uploads. I have specified the store_dir following way:
def store_dir
"uploads/#{Time.now.year}/#{Time.now.month}/#{Time.now.day}"
end
Uploading files work like a charm - each time I upload a file it ends up in directory where it should end; i.e. "today's directory".
When I try to download the file, carrierwave is constructing the download path dynamically based on store_dir options. So lets say a file which was uploaded on 1.12.2012 is available on the following path on fliesystem:
/uploads/2012/12/01/file.ext
will be retrieved by carrierwave as:
/uploads/2012/12/12/file.ext
Which obviously leads to "Cannot read file" error.
I came with 2 different possible solutions:
Create a separate filed where I will be storing the actual filepath to the file upon it's creation and then will use this value to retrieve file.
Overload retrieve_from_store! method (which is part of carrierwave gem) and make it construct path based on created_at field from the file record than rather from store_dir.
I am inclining to the second possibility since it feels not that dirty. Yet both feel "not-rails-way". Which one will be better to use and why? Or maybe carrierwave provides a way to solve this issue?
Totally guessing here but by looking at the docs I think something like this should work:
def store_dir
"uploads/#{model.created_at.year}/#{model.created_at.month}/#{model.created_at.day}"
end

How to assign a remote file to Carrierwave?

I have video model with the following definition:
class Video
require 'carrierwave/orm/activerecord'
mount_uploader :attachment, VideoUploader
mount_uploader :attachment_thumbnail, VideoThumbnailUploader
...
end
When I upload a video file. It also sends the file to our encoding service Zencoder, which encodes the video file and creates a thumbnail for it.
Normally, I could do something like #video.attachment.url, which will return the path of the video file. I'd like to do the same thing with the thumbnail. i.e. #video.attachment_thumbnail.url
However, since the attachment is created by our encoding service, which also uploads it to a specified S3 bucket. How do I assign the attachment to the attachment_thumbnail column for the record?
Can I simply do something like:
#video.update_attributes(
:attachment_thumbnail => 'https://bucket_name.s3.amazonaws.com/uploads/users/1/video/1/thumb.png'
)
Is it possible to assign files like this to Carrierwave?
You can do the following:
#video.remote_attachment_thumbnail_url = 'https://bucket_name.s3.amazonaws.com/uploads/users/1/video/1/thumb.png'
But that will cause Carrierwave to download + reprocess the file rather than just make it the thumbnail. If you're not going to use Carrierwave's processing, then it might make more sense to just store the URL to the thumbnail on the model rather than even using Carrierwave.
This worked for me, with CarrierWave 0.5.8
model.update_attributes(:remote_uploader_url => "http://path/to/image.jpg")
Of course, you need to set remote_uploader_url to be attr_accessible for this.
I was looking for this as well.
The blocking point in the zencoder case would be that Carrierwave doesn't track different different file type versions for the original file. It only references the original file.
So having the original file as an .mp4 a a thumbnail version as a .png doesn't work.
While you can have an 'image.png' and also track 'thumb_png_image.png', you can't also create a 'thumb_jpg_image.jpg' for the same file.
Otherwise you could create a dummy version and using conditional versioning tell CW not to process it.
Since CW would create the dummy version anyway but not upload it, you could have it reference a path matching the file returned by Zencoder. But oh well...
At the end of this episode (7:35), Ryan Bates adds a remote_image_url in a file form upload:
http://railscasts.com/episodes/253-carrierwave-file-uploads

can't write uploaded files in rails

I'm still fairly new to rails but I've done file uploading before. I'm trying to implement very basic file uploading - nothing fancy, just upload the file, save it, and make a record of it. So here's my method for saving the file:
def self.save(upload,name)
directory='public/uploads'
ext=File.extname(upload.original_filename)
path=File.join(directory, "#{name}#{ext}")
File.open(path,'wb') { |f| f.write(upload.read) }
end
The file is apparently valid, as extname() gets the correct extension. The new file is created at the correct place. But somehow the writing fails, as the file is always empty. Doesn't matter what type of file I use. What could be going wrong? I'm using Rails 3.0 if it matters.
Try doing File.open(path,'wb') { |f| f.write(upload.read); f.close }
The IO buffer probably isn't flushing, but closing the file should flush it.
Also, I'd strongly recommend using a plugin such as paperclip for file uploads, simply because file uploads can be annoying to manage, and paperclip provides a very nice way to abstract most of that into conventions so you can just add a couple of columns and do model.upload = params[:file].

Resources