Carrierwave files not deleting from S3 - ruby-on-rails

I'm using the carrierwave and carrierwave-aws gems to upload to S3 from a Rails application. I can upload files without any issue, but I can't remove them from S3.
My ActiveRecord model is called Episode and its video property is set by my CarrierWave::Uploader::Base uploader (which includes CarrierWave::Video).
Here are the steps I've followed:
Create an episode
Upload a video, verify it exists on S3 (this works!)
Call episode.remove_video!; episode.save! from the console
At this point, the video still exists on S3, despite receiving no error message
I've also tried:
episode.video.remove!
episode.save!
Which produces this output, but doesn't delete the file from S3:
[180] pry(main)> episode.video.remove!
=> [:remove_versions!]
I've also tried:
episode.destroy!
which should call the ActiveRecord callback added by Carrierwave, but doesn't. I added my own before_destroy method (destroy_assets below), but this also doesn't work.
Here is my setup:
carrierwave.rb
CarrierWave.configure do |config|
config.storage = :aws
config.aws_bucket = 'BUCKET_NAME'
config.aws_acl = 'public-read'
config.aws_authenticated_url_expiration = 60 * 60 * 24 * 7
config.aws_attributes = {
expires: 2.week.from_now.httpdate,
cache_control: 'max-age=604800'
}
config.aws_credentials = {
access_key_id: ENV['aws_access_key'],
secret_access_key: ENV['aws_secret'],
region: 'us-east-1'
}
config.remove_previously_stored_files_after_update = true
end
video_uploader.rb
class VideoUploader < CarrierWave::Uploader::Base
include CarrierWave::Video
storage :was
def store_dir
"uploads/videos/#{model.id}"
end
end
episode.rb
class Episode < ActiveRecord::Base
mount_uploader :video, VideoUploader
before_destroy :destroy_assets
def destroy_assets
self.video.remove! if self.video
self.save!
end
end
The AWS credentials I'm using are for an IAM user with the AmazonS3FullAccess policy, if that makes any difference here.

I would update your store_dir to be:
"uploads/videos/#{mounted_as}/#{model.id}"
So that carrierwave, while utilizing the mounting system, removes the file, it knows where to find it when it calls the full name method internally!

Related

How to migrate from Refile to ActiveStorage?

Any idea how to migrate a running project using Refile to the new rails's Active Storage?
Anyone knows any tutorial/guide about how to do that?
Thanks,
Patrick
I wrote a short post about it here which explains the process in detail:
https://dev.to/mtrolle/migrating-from-refile-to-activestorage-2dfp
Historically I hosted my Refile attached files in AWS S3, so what I did was refactoring all my code to use ActiveStorage instead. This primarily involved updating my model and views to use ActiveStorage syntax.
Then I removed the Refile gem and replaced it with ActiveStorage required gems like the image_processing gem and the aws-sdk-s3 gem.
Finally I created a Rails DB migration file to handle the actual migration of existing files. Here I looped through all records in my model with a Refile attachment to find their respective file in AWS S3, download it and then attach it to the model again using the ActiveStorage attachment.
Once the files were moved I could remove the legacy Refile database fields:
require 'mini_magick' # included by the image_processing gem
require 'aws-sdk-s3' # included by the aws-sdk-s3 gem
class User < ActiveRecord::Base
has_one_attached :avatar
end
class MovingFromRefileToActiveStorage < ActiveRecord::Migration[6.0]
def up
puts 'Connecting to AWS S3'
s3_client = Aws::S3::Client.new(
access_key_id: ENV['AWS_S3_ACCESS_KEY'],
secret_access_key: ENV['AWS_S3_SECRET'],
region: ENV['AWS_S3_REGION']
)
puts 'Migrating user avatar images from Refile to ActiveStorage'
User.where.not(avatar_id: nil).find_each do |user|
tmp_file = Tempfile.new
# Read S3 object to our tmp_file
s3_client.get_object(
response_target: tmp_file.path,
bucket: ENV['AWS_S3_BUCKET'],
key: "store/#{user.avatar_id}"
)
# Find content_type of S3 file using ImageMagick
# If you've been smart enough to save :avatar_content_type with Refile, you can use this value instead
content_type = MiniMagick::Image.new(tmp_file.path).mime_type
# Attach tmp file to our User as an ActiveStorage attachment
user.avatar.attach(
io: tmp_file,
filename: "avatar.#{content_type.split('/').last}",
content_type: content_type
)
if user.avatar.attached?
user.save # Save our changes to the user
puts "- migrated #{user.try(:name)}'s avatar image."
else
puts "- \e[31mFailed to migrate the avatar image for user ##{user.id} with Refile id #{user.avatar_id}\e[0m"
end
tmp_file.close
end
# Now remove the actual Refile column
remove_column :users, :avatar_id, :string
# If you've created other Refile fields like *_content_type, you can safely remove those as well
# remove_column :users, :avatar_content_type, :string
end
def down
raise ActiveRecord::IrreversibleMigration
end
end

Using carrierwave to upload images to google cloud storage, the file name ends up being saved and not the public link to the image in the bucket

I'm trying to implement image upload to google cloud storage from my rails 4.2 app using the carrierwave gem. Whenever I go to upload the image it is uploaded to the bucket fine but its saved in the db as the original image name e.g. image.png, not as the google cloud storage public link for the image e.g. https://storage.googleapis.com/project/bucket/image.png
Not too sure what is needed to be done from here to get it saving the public link from the bucket and not just the file name.
carrierwave.rb file
CarrierWave.configure do |config|
config.fog_credentials = {
provider: 'Google',
google_storage_access_key_id: 'key',
google_storage_secret_access_key: 'secret key'
}
config.fog_directory = 'bucket-name'
end
uploaders/check_item_value_image_uploader.rb
class CheckItemValueImageUploader < CarrierWave::Uploader::Base
# Include RMagick or MiniMagick support:
# include CarrierWave::RMagick
# include CarrierWave::MiniMagick
# Choose what kind of storage to use for this uploader:
#storage :file
storage :fog
# Override the directory where uploaded files will be stored.
# This is a sensible default for uploaders that are meant to be mounted:
def store_dir
"check-item-value-images/#{model.id}"
end
# Add a white list of extensions which are allowed to be uploaded.
# For images you might use something like this:
def extension_white_list
%w(jpg jpeg gif png)
end
end
related gems
gem 'gcloud'
gem "fog"
gem 'google-api-client', '~> 0.8.6'
gem "mime-types"
check_category_item_value model
mount_uploader :value, CheckItemValueImageUploader
check_category_item_value update method
if #check_category_item_value.save
flash[:success] = "Successfully updated"
redirect_to category_items_edit_path(#guide, #category, #category_item)
else
render 'category_items/edit'
end
edit form
<%= form_for(#check_category_item_value) do |f| %>
<%= f.file_field :value, :value => item_key.value, accept: "image/jpeg, image/jpg, image/gif, image/png" %>
<%= f.submit "Submit" %><hr>
<% end %>
The forms works fine but the value saved is the original image name not the google cloud storage public link for the image.
I used the carrierwave docs, this post, and this video by google cloud platform to get what I have now. What am I missing?
update
adding config.fog_public = true does nothing
CarrierWave.configure do |config|
config.fog_credentials = {
provider: 'Google',
google_storage_access_key_id: 'key',
google_storage_secret_access_key: 'secret key'
}
config.fog_public = true
config.fog_directory = 'bucket-name'
end
To set the link public, please check this config this in your config file:
# You may set it false now
config.fog_public = true
For filename you may overwrite in your CheckItemValueImageUploader, here is an example:
class CheckItemValueImageUploader < CarrierWave::Uploader::Base
def filename
"#{model.id}-#{original_filename}.#{file.extension}" if original_filename.present?
end
end

Carrierwave: `eval': no implicit conversion of nil into String (TypeError)

When trying to configure carrierwave for uploads, I'm getting this error, it says configuration.rb line 73 and the file doesn't even exist? or so i can't find it.
/Users/spencerlong/.rvm/gems/ruby-2.0.0-p451/gems/carrierwave-0.10.0/lib/carrierwave/uploader/configuration.rb:73:in `eval': no implicit conversion of nil into String (TypeError)
carrierwave.rb
CarrierWave.configure do |config|
config.storage = :aws
config.aws_bucket = 'larfs'
config.aws_acl = :public_read
config.asset_host = ''
config.aws_authenticated_url_expiration = 60 * 6 * 24 * 365
config.aws_credentials = {
access_key_id: "ABCABC",
secret_access_key: "ABCABC"
}
end
image_uploader.rb
class ImageUploader < CarrierWave::Uploader::Base
include CarrierWave::MiniMagick
storage :aws
def store_dir
"uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
end
version :thumb do
process :resize_to_fit => [50, 50]
end
def extension_white_list
%w(jpg jpeg gif png)
end
end
Not sure where you got the information on the configuring carrierwave this way. Carrierwave uses fog to communicate to AWS and any other cloud. You need to set the storage to :fog not aws.
Follow the documentation here and it should be fine: Using Amazin S3
If you want to use this config format.
You need "carrierwave-aws" gem
Please confirm whether this line is included in Gemfile
gem 'carrierwave-aws'

How can I use Carrierwave with cloudinary in seeds file

I want to upload multiple images to cloudinary by association of a carrierwave active record. How can I do this in a seeds file, using an array of remote urls?
Ive read countless articles how to use carrierwave/cloudinary helper tags to target an image upload within an html form, but nothing on doing this directly within code, any ideas?
That's not so hard to do at all.
So what I did:
# Gemfile
gem 'cloudinary'
gem 'carrierwave'
#config/cloudinary.yml
development:
cloud_name: "carrierwave-example"
api_key: "YOUR CLOUDINARY CREDENTIALS"
api_secret: "YOUR CLOUDINARY CREDENTIALS"
# in your uploader
class ImageUploader < CarrierWave::Uploader::Base
include Cloudinary::CarrierWave #include cloudinary lib
# storage :file - comment this out
# def store_dir - comment this too
# "uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
# end
end
# You model, that uses uploader
class Picture < ActiveRecord::Base
mount_uploader :image, ImageUploader
end
After writting this simple staff you can create Pictures, that will store images in clodinary like this:
Picture.create(image: "/path/to/image")
or if you have remote links of images, U just itterate through them
["http://image1.jpg","http://image2.jpg","http://image3.jpg"].each do |link|
Picture.create(remote_image_url: link)
end
Just remember to use remote_#{your_column_name}_url if you have remote link

Public URL with Fog and Amazon S3

Versions of all RubyGems. I am using Ruby on Rails 3.1.3, Ruby 1.9.2, CarrierWave 0.5.8, and Fog 1.1.2.
I am using the CarrierWave RubyGem too for image uploading and the Fog RubyGem for Amazon S3 file upload.
In my CarrierWave initializer file I have:
CarrierWave.configure do |config|
config.fog_credentials = {
provider: 'AWS',
aws_access_key_id: 'xxx',
aws_secret_access_key: 'xxx'
}
if Rails.env.production?
config.fog_directory = 'bucket1'
elsif Rails.env.development?
config.fog_directory = 'bucket2'
else
config.fog_directory = 'bucket3'
end
config.fog_public = false
config.fog_authenticated_url_expiration = 60
end
I have an uploader file:
class PageAttachmentUploader < CarrierWave::Uploader::Base
CarrierWave.configure do |config|
if Rails.env.development? || Rails.env.development? || Rails.env.production?
config.fog_public = true
end
end
storage :fog
end
I am having two uploader files. I want one to be set to private and one to public.
I am trying to overwrite CarrierWave configuarations when PageAttachmentUploader is invoked and set the URL to public. This works like charm in the local machine, but it does not work in staging, sandbox and production.
I changed config.fog_public = true in the CarrierWave intializer. Even that does not work in sandbox. How do I fix this problem?
No, you should not use CarrierWave.configure directly in your uploaders as it will change the default configuration for all uploaders and not only each uploader.
I don't know if that's the best solution but you can change the default fog configuration directly by setting class methods in your uploaders like this :
class ImageUploader < CarrierWave::Uploader::Base
storage :fog
def self.fog_public
true # or false
end
end
Actually, the best way (I've found) is to do the following:
class ImageUploader < CarrierWave::Uploader::Base
storage :fog
configure do |c|
c.fog_public = true # or false
end
end
It feels more in line with CarrierWave's style to do it this way.

Resources