The guide says that I can save an attachment to disc to run a process on it like this:
message.video.open do |file|
system '/path/to/virus/scanner', file.path
# ...
end
My model has an attachment defined as:
has_one_attached :zip
And then in the model I have defined:
def process_zip
zip.open do |file|
# process the zip file
end
end
However I am getting an error :
private method `open' called
on the zip.open call.
How can I save the zip locally for processing?
As an alternative in Rails 5.2 you can do this:
def process_zip
# Download the zip file in temp dir
zip_path = "#{Dir.tmpdir}/#{zip.filename}"
File.open(zip_path, 'wb') do |file|
file.write(zip.download)
end
Zip::File.open(zip_path) do |zip_file|
# process the zip file
# ...
puts "processing file #{zip_file}"
end
end
That’s an edge guide (note edgeguides.rubyonrails.org in the URL); it applies to the master branch of the rails/rails repository on GitHub. The latest changes in master haven’t been included in a released version of Rails yet.
You’re likely using Rails 5.2. Use edge Rails to take advantage of ActiveStorage::Blob#open:
gem "rails", github: "rails/rails"
Related
Any idea how to migrate a running project using Refile to the new rails's Active Storage?
Anyone knows any tutorial/guide about how to do that?
Thanks,
Patrick
I wrote a short post about it here which explains the process in detail:
https://dev.to/mtrolle/migrating-from-refile-to-activestorage-2dfp
Historically I hosted my Refile attached files in AWS S3, so what I did was refactoring all my code to use ActiveStorage instead. This primarily involved updating my model and views to use ActiveStorage syntax.
Then I removed the Refile gem and replaced it with ActiveStorage required gems like the image_processing gem and the aws-sdk-s3 gem.
Finally I created a Rails DB migration file to handle the actual migration of existing files. Here I looped through all records in my model with a Refile attachment to find their respective file in AWS S3, download it and then attach it to the model again using the ActiveStorage attachment.
Once the files were moved I could remove the legacy Refile database fields:
require 'mini_magick' # included by the image_processing gem
require 'aws-sdk-s3' # included by the aws-sdk-s3 gem
class User < ActiveRecord::Base
has_one_attached :avatar
end
class MovingFromRefileToActiveStorage < ActiveRecord::Migration[6.0]
def up
puts 'Connecting to AWS S3'
s3_client = Aws::S3::Client.new(
access_key_id: ENV['AWS_S3_ACCESS_KEY'],
secret_access_key: ENV['AWS_S3_SECRET'],
region: ENV['AWS_S3_REGION']
)
puts 'Migrating user avatar images from Refile to ActiveStorage'
User.where.not(avatar_id: nil).find_each do |user|
tmp_file = Tempfile.new
# Read S3 object to our tmp_file
s3_client.get_object(
response_target: tmp_file.path,
bucket: ENV['AWS_S3_BUCKET'],
key: "store/#{user.avatar_id}"
)
# Find content_type of S3 file using ImageMagick
# If you've been smart enough to save :avatar_content_type with Refile, you can use this value instead
content_type = MiniMagick::Image.new(tmp_file.path).mime_type
# Attach tmp file to our User as an ActiveStorage attachment
user.avatar.attach(
io: tmp_file,
filename: "avatar.#{content_type.split('/').last}",
content_type: content_type
)
if user.avatar.attached?
user.save # Save our changes to the user
puts "- migrated #{user.try(:name)}'s avatar image."
else
puts "- \e[31mFailed to migrate the avatar image for user ##{user.id} with Refile id #{user.avatar_id}\e[0m"
end
tmp_file.close
end
# Now remove the actual Refile column
remove_column :users, :avatar_id, :string
# If you've created other Refile fields like *_content_type, you can safely remove those as well
# remove_column :users, :avatar_content_type, :string
end
def down
raise ActiveRecord::IrreversibleMigration
end
end
I'm working on a project that is migrating data from a customers old_busted DB into rails objects to be worked on later. Similarly, I need to convert these objects into a CSV and upload it to a neutral FTP (this is to allow a coworker to build the example pages through Sugar CRM). I've created rake files to do all of this, and it was successful. Now, I'm going to continue this process for each object that I create in rails (relative to the previous DB) and, best case, wanted these generated when I run rake generate scaffold <object>.
Here is my import rake:
desc "Import Clients from db"
task :get_busted_clients => [:environment] do
#old_clients = Busted::Client.all
#old_clients.each do |row|
#client = Client.new();
#client.client_id = row.NUMBER
#client.save
end
end
Here is my CSV convert/FTP upload rake:
desc "Exports db's to local CSV and uploads them to FTP"
task :export_clients_CSV => [:environment] do
# Required libraries for CSV read/write and NET/FTP IO #
require 'csv'
require 'net/ftp'
# Pull all Editor objects into clients for reading #
clients = Client.all
puts "Creating CSV file for <Clients> and updating column names..."
# Open a new CSV file that uses the column headers from Client #
CSV.open("clients.csv", "wb",
:write_headers => true, :headers => Client.column_names) do |csv|
puts "--Loading each entry..."
# Load all entries from Client into the CSV file row by row #
clients.each do |client|
# This line specifically puts the attributes in the rows WITH RESPECT TO#
# THE COLUMNS
csv << client.attributes.values_at(*Client.column_names)
end
puts "--Done loading each entry..."
end
puts "...Data populated. Finished bulding CSV. Closing File."
puts "------------------------"
# Upload CSV File to FTP server by requesting new FTP connection, assigning credentials
# and informing the client what file to look for and what to name it
puts "Uploading <Clients>..."
ftp = Net::FTP.new('192.168.xxx.xxx')
ftp.login(user = "user", passwd = "passwd")
ftp.puttextfile("clients.csv", "clients.csv")
ftp.quit()
puts "...Finished."
end
I ran rake generate g get_busted and put this in my get_busted_generator.rb:
class GetBustedGenerator < Rails::Generators::NamedBase
source_root File.expand_path('../templates', __FILE__)
def generate_get_busted
copy_file "getbusted.rake", "lib/tasks/#{file_name}.rake"
end
end
After that, I got lost. I can't find anything on templating a rake file or the syntax included to do so.
Rails has been a recent endeavor and I may be overlooking something in terms of design of the solution to my problem.
TL;DR: Is templating a rake file a bad thing? Solution alternatives? If not, whats the syntax for generating either script custom to the object (or point me in the direction, please).
I've been knocking my head around with Heroku, while trying to download a zip file with all my receipt files data.
The files are stored on amazon s3 and it all works fine on my development machine..
I thought it had to do with Tempfile, and abandoned that previous solution, since heroku has some strict policies with their filesystem, so i used the tmp folder, but the problem doesn't seem to be there. I already tried to load directly from s3 (using openUri) to the zip file, but it doesn't seem to work either on Heroku.
What might be wrong with my code for Heroku not loading the files to the zip?
Here is my model method :
def zip_receipts(search_hash=nil)
require 'zip/zip'
require 'zip/zipfilesystem'
t=File.open("#{Rails.root}/tmp/#{Digest::MD5.hexdigest(rand(12).to_s)}_#{Process.pid}",'w')
# t = Tempfile.new(Digest::MD5.hexdigest(rand(12).to_s))
# Give the path of the temp file to the zip outputstream, it won't try to open it as an archive.
Zip::ZipOutputStream.open(t.path) do |zos|
logger.debug("search hash Zip: #{search_hash.inspect}")
self.feed(search_hash).each do |receipt|
begin
require 'open-uri'
require 'tempfile'
#configures filename
filen = File.basename(receipt.receipt_file_file_name)
ext= File.extname(filen)
filen_noext = File.basename(receipt.receipt_file_file_name, '.*')
filen=filen_noext+SecureRandom.hex(10)+ext
logger.info("Info Zip - Filename: #{filen}")
# Create a new entry on the zip file
zos.put_next_entry(filen)
# logger.info("Info Zip - Added entry: #{zos.inspect}")
# Add the contents of the file, reading directly from amazon
tfilepath= "#{Rails.root}/tmp/#{File.basename(filen,ext)}_#{Process.pid}"
open(tfilepath,"wb") do |file|
file << open(receipt.authenticated_url(:original),:ssl_verify_mode => OpenSSL::SSL::VERIFY_NONE).read
end
zos.print IO.binread tfilepath
# logger.info("Info Zip - Extracted from amazon: #{zos.inspect}")
rescue Exception => e
logger.info("exception #{e}")
end # closes the exception begin
end #closes receipts cycle
end #closes zip file stream cycle
# The temp file will be deleted some time...
t.close
#returns the path for send file controller to act
t.path
end
My controller:
def download_all
#user = User.find_by_id(params[:user_id])
filepath = #user.zip_receipts
# Send it using the right mime type, with a download window and some nice file name.
send_file(filepath,type: 'application/zip', disposition: 'attachment',filename:"MyReceipts.zip")
end
And I write also my view and routes, so that it might serve anyone else trying to implement a download all feature
routes.rb
resources :users do
post 'download_all'
end
my view
<%= link_to "Download receipts", user_download_all_path(user_id:user.id), method: :post %>
The problem seemed to be with the search hash, and the sql query, and not the code itself. For some reason, the receipts get listed, but aren't downloaded. So it is an all different issue
In the end i have this code for the model
def zip_receipts(search_hash=nil)
require 'zip/zip'
require 'zip/zipfilesystem'
t=File.open("#{Rails.root}/tmp/MyReceipts.zip_#{Process.pid}","w")
# t = Tempfile.new(Digest::MD5.hexdigest(rand(12).to_s))
#"#{Rails.root}/tmp/RecibosOnline#{SecureRandom.hex(10)}.zip"
puts "Zip- Receipts About to enter"
# Give the path of the temp file to the zip outputstream, it won't try to open it as an archive.
Zip::ZipOutputStream.open(t.path) do |zos|
self.feed(search_hash).each do |receipt|
begin
require 'open-uri'
require 'tempfile'
filen = File.basename(receipt.receipt_file_file_name)
ext= File.extname(filen)
filen_noext = File.basename(receipt.receipt_file_file_name, '.*')
filen=filen_noext+SecureRandom.hex(10)+ext
# puts "Info Zip - Filename: #{filen}"
# Create a new entry on the zip file
zos.put_next_entry(filen)
zos.print open(receipt.authenticated_url(:original),:ssl_verify_mode => OpenSSL::SSL::VERIFY_NONE).read
rescue Exception => e
puts "exception #{e}"
end # closes the exception begin
end #closes receipts cycle
end #closes zip file stream cycle
# The temp file will be deleted some time...
t.close
#returns the path for send file controller to act
t.path
end
I am using carrierwave to upload a video then have a version called thumb with a custom processor that takes the video and creates a screenshot using streamio-ffmpeg. Both the video and the file are uploaded correctly but when calling uploader.url(:thumb) I get:
ArgumentError: Version thumb doesn't exist!
VideoUploader.rb
require 'carrierwave/processing/mime_types'
require 'streamio-ffmpeg'
class VideoUploader < CarrierWave::Uploader::Base
include CarrierWave::VideoConverter
include CarrierWave::MimeTypes
process :set_content_type
storage :file
version :thumb do
process :create_thumb
#def full_filename(for_file)
# "thumb_#{File.basename(for_file, File.extname(for_file))}.png"
#end
end
def create_thumb
cached_stored_file! if !cached?
movie = FFMPEG::Movie.new(current_path)
dirname = File.dirname(current_path)
thumb_path = "#{File.join(dirname, File.basename(path, File.extname(path)))}.png"
movie.screenshot(thumb_path, :seek_time => 5)
File.rename thumb_path, current_path
end
def file_identifier
model[:video]
end
# Override the directory where uploaded files will be stored.
# This is a sensible default for uploaders that are meant to be mounted:
def store_dir
return "uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.user_id}/#{model.id}"
end
end
Then model.video_url(:thumb) returns the argument error. I am not sure what to do or why the version isn't registered any help would be great, thanks.
Fix
What contributed to the error was a mix of restarting the server but not restarting the rails console. Once i did this the Argument error went away but I was getting the wrong path. So i uncommented
def full_filename(for_file)
"thumb_#{File.basename(for_file, File.extname(for_file))}.png"
end
and used
[model].video.recreate_versions!
to correct any errors in the paths or naming schemes that could have occured
most likely some step in your create_thumb method is failing and thus the thumb is never created and has no URL. Are there any exceptions being thrown to your logs?
Perhaps you need to specify the FFMPEG binary location:
FFMPEG.ffmpeg_binary = '/usr/local/bin/ffmpeg'
What is the best way to get a temporary directory with nothing in it using Ruby on Rails? I need the API to be cross-platform compatible. The stdlib tmpdir won't work.
The Dir object has a method mktmpdir which creates a temporary directory:
require 'tmpdir' # Not needed if you are using rails.
Dir.mktmpdir do |dir|
puts "My new temp dir: #{dir}"
end
The temporary directory will be removed after execution of the block.
The Dir#tmpdir function in the Ruby core (not stdlib that you linked to) should be cross-platform.
To use this function you need to require 'tmpdir'.
A general aprox I'm using now:
def in_tmpdir
path = File.expand_path "#{Dir.tmpdir}/#{Time.now.to_i}#{rand(1000)}/"
FileUtils.mkdir_p path
yield path
ensure
FileUtils.rm_rf( path ) if File.exists?( path )
end
So in your code you can:
in_tmpdir do |tmpdir|
puts "My tmp dir: #{tmpdir}"
# work with files in the dir
end
The temporary dir will be removed automatically when your method will finish.
Ruby has Dir#mktmpdir, so just use that.
require 'tempfile'
Dir.mktmpdir('prefix_unique_to_your_program') do |dir|
### your work here ###
end
See http://www.ruby-doc.org/stdlib-1.9.3/libdoc/tmpdir/rdoc/Dir.html
Or build your own using Tempfile tempfile that is process and thread unique, so just use that to build a quick Tempdir.
require 'tempfile'
Tempfile.open('prefix_unique_to_your_program') do |tmp|
tmp_dir = tmp.path + "_dir"
begin
FileUtils.mkdir_p(tmp_dir)
### your work here ###
ensure
FileUtils.rm_rf(tmp_dir)
end
end
See http://www.ruby-doc.org/stdlib-1.9.3/libdoc/tempfile/rdoc/Tempfile.html for optional suffix/prefix options.
require 'tmpdir' # not needed if you are loading Rails
tmp_dir = File.join(Dir::tmpdir, "my_app_#{Time.now.to_i}_#{rand(100)}")
Dir.mkdir(tmp_dir)
Works for me.
You can use Dir.mktmpdir.
Using a block will get rid of the temporary directory when it closes.
Dir.mktmpdir do |dir|
File.open("#{dir}/foo", 'w') { |f| f.write('foo') }
end
Or if you need multiple temp directories to exist at the same time, for example
context 'when there are duplicate tasks' do
it 'raises an DuplicateTask error' do
begin
tmp_dir1 = Dir.mktmpdir('foo')
tmp_dir2 = Dir.mktmpdir('bar')
File.new("#{tmp_dir1}/task_name", 'w+')
File.new("#{tmp_dir2}/task_name", 'w+')
expect { subject.filepath('task_name') }.to raise_error(TaskFinder::DuplicateTask)
ensure
FileUtils.remove_entry tmp_dir1
FileUtils.remove_entry tmp_dir2
end
end
end
Dir.mktmpdir creates a temporary directory under Dir.tmpdir (you'll need to require 'tmpdir' to see what that evaluates to).
If you want to use your own path, Dir.mktmpdir takes an optional second argument tmpdir if non-nil value is given. E.g.
Dir.mktmpdir(nil, "/var/tmp") { |dir| "dir is '/var/tmp/d...'" }
I started to tackle this by hijacking Tempfile, see below.
It should clean itself up as Tempfile does, but doesn't always yet..
It's yet to delete files in the tempdir.
Anyway I share this here, might be useful as a starting point.
require 'tempfile'
class Tempdir < Tempfile
require 'tmpdir'
def initialize(basename, tmpdir = Dir::tmpdir)
super
p = self.path
File.delete(p)
Dir.mkdir(p)
end
def unlink # copied from tempfile.rb
# keep this order for thread safeness
begin
Dir.unlink(#tmpname) if File.exist?(#tmpname)
##cleanlist.delete(#tmpname)
#data = #tmpname = nil
ObjectSpace.undefine_finalizer(self)
rescue Errno::EACCES
# may not be able to unlink on Windows; just ignore
end
end
end
This can be used the same way as Tempfile, eg:
Tempdir.new('foo')
All methods on Tempfile , and in turn, File should work.
Just briefly tested it, so no guarantees.
Update: gem install files, then
require "files"
dir = Files do
file "hello.txt", "stuff"
end
See below for more examples.
Here's another solution, inspired by a few other answers. This one is suitable for inclusion in a test (e.g. rspec or spec_helper.rb). It makes a temporary dir based on the name of the including file, stores it in an instance variable so it persists for the duration of the test (but is not shared between tests), and deletes it on exit (or optionally doesn't, if you want to check its contents after the test run).
def temp_dir options = {:remove => true}
#temp_dir ||= begin
require 'tmpdir'
require 'fileutils'
called_from = File.basename caller.first.split(':').first, ".rb"
path = File.join(Dir::tmpdir, "#{called_from}_#{Time.now.to_i}_#{rand(1000)}")
Dir.mkdir(path)
at_exit {FileUtils.rm_rf(path) if File.exists?(path)} if options[:remove]
File.new path
end
end
(You could also use Dir.mktmpdir (which has been around since Ruby 1.8.7) instead of Dir.mkdir but I find the API of that method confusing, not to mention the naming algorithm.)
Usage example (and another useful test method):
def write name, contents = "contents of #{name}"
path = "#{temp_dir}/#{name}"
File.open(path, "w") do |f|
f.write contents
end
File.new path
end
describe "#write" do
before do
#hello = write "hello.txt"
#goodbye = write "goodbye.txt", "farewell"
end
it "uses temp_dir" do
File.dirname(#hello).should == temp_dir
File.dirname(#goodbye).should == temp_dir
end
it "writes a default value" do
File.read(#hello).should == "contents of hello.txt"
end
it "writes a given value" do
# since write returns a File instance, we can call read on it
#goodbye.read.should == "farewell"
end
end
Update: I've used this code to kickstart a gem I'm calling files which intends to make it super-easy to create directories and files for temporary (e.g. unit test) use. See https://github.com/alexch/files and https://rubygems.org/gems/files . For example:
require "files"
files = Files do # creates a temporary directory inside Dir.tmpdir
file "hello.txt" # creates file "hello.txt" containing "contents of hello.txt"
dir "web" do # creates directory "web"
file "snippet.html", # creates file "web/snippet.html"...
"<h1>Fix this!</h1>" # ...containing "<h1>Fix this!</h1>"
dir "img" do # creates directory "web/img"
file File.new("data/hello.png") # containing a copy of hello.png
file "hi.png", File.new("data/hello.png") # and a copy of hello.png named hi.png
end
end
end # returns a string with the path to the directory
Check out the Ruby STemp library: http://ruby-stemp.rubyforge.org/rdoc/
If you do something like this:
dirname = STemp.mkdtemp("#{Dir.tmpdir}/directory-name-template-XXXXXXXX")
dirname will be a string that points to a directory that's guaranteed not to exist previously. You get to define what you want the directory name to start with. The X's get replaced with random characters.
EDIT: someone mentioned this didn't work for them on 1.9, so YMMV.