Use existing image on S3 if it exists from Rails seeds - ruby-on-rails

I am trying to improve a set of Rails seeds which currently upload all their file attachments to S3 regardless of whether they currently exist. This results in very slow seeds and an obvious wastage of S3 resources. It looks like this:
factory :logo_attachment do
file do
filename = Dir.glob(Rails.root.join('test', 'fixtures', 'files','logos','*.{jpg,png,gif,svg}')).sample
extname = File.extname(filename)[1..-1]
mime_type = Mime::Type.lookup_by_extension(extname)
content_type = mime_type.to_s
fixture_file_upload(filename, content_type)
end
end
Attached with:
after(:create) do |organisation|
organisation.logo = create(:logo_attachment, attacher: organisation)
end
How can I improve this state of affairs so it uses pre-existing images on S3? I did wish to use local images only for this process but the application is heavily tied to S3.

Related

Download file in service object with ActiveStorage

A user uploads a document and this gets stored in Azure with ActiveStorage. The next step is that the backend processes this and therefore I have a service object to do this. So I need to download the file from Azure to the tmp folder within the Rails app. How do I download the file? I cannot use rails_blob_url because it is not available in a service object, only in controllers and views.
When I still used Paperclip I did something like this:
require 'open-uri'
file = Rails.root.join('tmp', user.attachment_file_name)
name = user.attachment_file_name
download = open(user.attachment.url)
download_result = IO.copy_stream(download, file)
How can I do something similar with ActiveStorage?
You can use ActiveStorage::Blob#open:
Downloads the blob to a tempfile on disk. Yields the tempfile.
Given this example from the guides:
class User < ApplicationRecord
has_one_attached :avatar
end
You can do this with:
user.avatar.open do |tempfile|
# do something with the file
end
If its has_many_attached you of course need to loop through the attachments.
See:
Active Storage Overview

Upload files to multiple buckets

I create a rails application for uploading files through carrierwave to S3 bucket,
I uploaded them to one bucket and I want to upload them to two buckets and regions at the same time .
How can I do that?
You can create an upload method and send your bucket name as an argument. A quick and dirty version would look something like:
def upload_file(specific_bucket = nil)
unless specific_bucket
BUCKET_LIST.each do |bucket|
# send file to bucket
end
else
# upload to specific_bucket
end
end
Store your bucket list in an appropriate location
BUCKET_LIST = [bucket_name_one, bucket_name_two]

Saving image created with RMagick using Rails and CarrierWave

In my Rails app I have a module that takes several images, and using RMagick "stitches" them together into a new single image. I'm able to successfully create the final image, but I'm having trouble saving this new image as an attachment to a model (using CarrierWave). The method that does the stitching looks like this:
def generate_collage(params)
final_image = ImageList.new
# ... code that puts together the composite image ...
return final_image.append(true).to_blob { |attrs| attrs.format = 'JPEG' }
end
I've got my User model with an uploader mounted:
class User < ActiveRecord::Base
mount_uploader :image, UserImageUploader
end
In the CarrierWave documentation under the ActiveRecord section, they show how to assign a new image, but they assume the file already exists somewhere. In my case it doesn't yet exist on the filesystem, and I'm outputting a blob... is there any way to go from that blob to generating an image upload for CarrierWave?
I suppose I'm trying to avoid saving this image temporarily into "#{Rails.root}/tmp/" and then reading it from there... it seems like I could cut out this step and send directly to CarrierWave somehow, but I don't know how! Is it possible?
I'm working on something similar right now. This should be possible, but an easy workaround is to save it to a temp file:
temp_file = Tempfile.new([ 'temp', '.png' ])
image.write(temp_file.path)
user = User.new
user.avatar = temp_file
user.save
temp_file.close
temp_file.unlink
I'm hoping to try to improve it to remove the file system dependency completely, by following the advice in one of these answers: How to handle a file_as_string (generated by Prawn) so that it is accepted by Carrierwave?

Carrierwave + Fog + S3 remove file without going through a model

I am building an application that has a chat component to it. The application allows users to upload files to the chat. The chat is all javascript but i wanted to use Carrierwave for the uploads because i am using it elsewhere in the application. I am doing the handling of the uploads through AJAX so that i can get into Rails land and let Carrierwave take over.
I have been able to get the chat to successfully upload the files to the correct location in my S3 bucket. The thing i can't figure out is how to delete the files. Here is my code the uploads the files - this is the method that is called from the route that the AJAX call hits.
def upload
file = File.open(params[:file_0].tempfile)
uploader = ChatUploader.new
uploader.store!(file)
end
There is little to no documentation with Carrierwave on how to upload files without going through a model and basically NO documentation on how to remove files without going through a model. I assume it is possible though - i just need to know what to call. So i guess my question is how do i delete files?
UPDATE (11/23)
I got the code to save and delete files from S3 using these methods:
# code to save the file
def upload
file = File.open(params[:file_0].tempfile)
uploader = ChatUploader.new
uploader.store!(file)
uploader.store_path()
end
# code to remove files
def remove_file
file = params[:file]
uploader = ChatUploader.new
uploader.retrieve_from_store!(file)
uploader.remove!
end
My only issue now is that the filename for the uploaded file is not correct. It saves all files with a "RackMultipart" and then some numbers which look like a date, time, and identifier? (example: RackMultipart20141123-17740-1tq4j1g) Need to try and use the original filename plus maybe a timestamp for uniqueness.
I believe it has something to do with these two lines:
file = File.open(params[:file_0].tempfile)
and
uploader.store!(file)

What's the proper way to copy a carrierwave file from one record to another?

I need to copy a file from one carrier wave object to another. They are different tables and different types of uploaders.
I started with:
user.avatar = image.content
(where user and image are model instances, avatar and content are the carrierwave mounted uploaders) which worked sometimes. It seems to work all the time locally, with a file storage, but intermittent when using fog and s3.
In a mailing list post I found this code:
user.avatar = image.content.file
that again worked sometimes.
My working solution so far is:
require "open-uri"
begin
user.avatar = open(image.url)
rescue Errno::ENOENT => e
begin
user.avatar = open(image.path)
rescue Errno::ENOENT => e
# Ok, whatever.
end
end
which is not only ugly, but fails to pass the extension validation because the opening of a remote file doesn't maintain the extension (jpg, png, etc.).
Perhaps one way you can do it is to set a remote image URL as per the Carrierwave gem documentation?
user.remote_avatar_url = image.url
From solutions discussed here I created simple CopyCarrierwaveFile gem to do this
usage is something like this:
original_resource = User.last
new_resource = User.new
CopyCarrierwaveFile::CopyFileService.new(original_resource, new_resource, :avatar).set_file
new_resource.save
nev_resource.avatar.url # https://...image.jpg
Here's a (albeit hacky) solution to that doesn't require an HTTP request to fetch the image:
module UploadCopier
def self.copy(old, new)
new.instance_variable_set('#_mounters', nil)
old.class.uploaders.each do |column, uploader|
new.send("#{column}=", old.send(column))
end
end
end
old_user = User.last
new_user = User.new
UploadCopier.copy(old_user, new_user)
new_user.save
I needed to copy a reference from one model to another model and I was successfully able to do so by doing the following:
my_new_model.update_column('attachment', my_other_model.attributes["attachment"]);
In this scenario, I did not care to actually make a copy of the file, nor did I care that 2 records were now linked to the same file (my system never deletes or modifies files after uploaded).
This may be useful to anyone who wants to just copy the reference to a file from one model to another model using the same uploader.
You can do this by copying files.
store_path is a carrierwave method from Uploader class. It returns uploaded file's folder relative path.
this clone file method should be called after model record is saved.
If record not saved, store_path may return wrong path if you specify store_dir with model id in uploader.
def clone_carrierwave_file(column_name)
origin_files = Dir[File.join(Rails.root, 'public', original_record.send(column_name).store_path, '*')]
return if origin_files.blank?
new_file_folder = File.join(Rails.root, 'public', send(column_name).store_path)
FileUtils.mkdir new_file_folder if !Dir.exist? new_file_folder
FileUtils.cp(origin_files, new_file_folder)
end
Hope it works.
I just wanted to copy an avatar reference from one object to another, and what worked for me was:
objectB.avatar.retrieve_from_store!(objectA.avatar.identifier)
objectB.save

Resources