I am uploading an audiofile and make a mp3 version, which works. Additionally I am generating a waveform out of the mp3 as a "png". Wich also works well.
The image was generated and saved but with "mp3" as suffix, which should be "png".
The view has rendered the image correctly with the "mp3" fileextension name.
Now I get a 404 error when the view tries to get the image. The filename was not assumed correctly:
https://mybucket.amazonaws.com/uploads/sound/soundfile/142/waveform_Sky_02.wav
which should be
https://mybucket.amazonaws.com/uploads/sound/soundfile/142/waveform.png
here is my :version code:
version :waveform do
def filename
"watermark.png" if original_filename.present?
end
def convert_to_waveform
cache_stored_file! if !cached?
Dir::Tmpname.create(File.basename(current_path)) do |tempname|
begin
puts system %Q{ffmpeg -y -i "#{current_path}" -f wav "#{tempname}" > /dev/null 2>&1}
FileUtils.rm current_path
Waveform.generate(tempname, current_path, method: :rms, background_color: :transparent)
ensure
FileUtils.rm tempname
end
end
end
process :convert_to_waveform
end
The database saves "waveform_Sky_02.wav"
How can I get this working?
this has been working for me:
def full_filename(for_file=file)
super.chomp('wav') + '.png'
end
or for all filetypes:
def full_filename(for_file=file)
super.chomp(File.extname(super)) + '.png'
end
Related
For a variety of reasons I am migrating my uploads from ActiveStorage (AS) to CarrierWave (CW).
I am making rake task and have the logic sorted out - I am stumped at how to feed the AS blob into the CW file.
I am trying something like ths:
#files.each.with_index(1) do | a, index |
if a.attachment.attached?
a.attachment.download do |file|
a.file = file
end
a.save!
end
end
This is based on these two links:
https://edgeguides.rubyonrails.org/active_storage_overview.html#downloading-files
message.video.open do |file|
system '/path/to/virus/scanner', file.path
# ...
end
and
https://github.com/carrierwaveuploader/carrierwave#activerecord
# like this
File.open('somewhere') do |f|
u.avatar = f
end
I tested this locally and the files are not mounted via the uploader. My question(s) would be:
am I missing something obvious here?
is my approach wrong and needs a new one?
Bonus Karma Question:
I can't seem to see a clear path to set the CW filename when I do this?
Here is my final rack task (based on the accepted answer) - open to tweaks. Does the job for me:
namespace :carrierwave do
desc "Import the old AS files into CW"
task import: :environment do
#files = Attachment.all
puts "#{#files.count} files to be processed"
puts "+" * 50
#files.each.with_index(1) do | a, index |
if a.attachment.attached?
puts "Attachment #{index}: Key: #{a.attachment.blob.key} ID: #{a.id} Filename: #{a.attachment.blob.filename}"
class FileIO < StringIO
def initialize(stream, filename)
super(stream)
#original_filename = filename
end
attr_reader :original_filename
end
a.attachment.download do |file|
a.file = FileIO.new(file, a.attachment.blob.filename.to_s)
end
a.save!
puts "-" * 50
end
end
end
desc "Purge the old AS files"
task purge: :environment do
#files = Attachment.all
puts "#{#files.count} files to be processed"
puts "+" * 50
#files.each.with_index(1) do | a, index |
if a.attachment.attached?
puts "Attachment #{index}: Key: #{a.attachment.blob.key} ID: #{a.id} Filename: #{a.attachment.blob.filename}"
a.attachment.purge
puts "-" * 50
#count = index
end
end
puts "#{#count} files purged"
end
end
Now in my case I am doing this in steps - I have branched my master with this rake task and the associated MCV updates. If my site was in true production would probably run the import rake task first then confirm all went well THEN purge the old AS files.
The file object you get from the attachment.download block is a string. More precisely, the response from .download is the file, "streamed and yielded in chunks" (see documentation). I validated this by calling file.class to make sure the class is what I expected.
So, to solve your issue, you need to provide an object on which .read can be called. Commonly that is done using the Ruby StringIO class.
However, considering Carrierwave also expects a filename, you can solve it using a helper model that inherits StringIO (from blogpost linked above):
class FileIO < StringIO
def initialize(stream, filename)
super(stream)
#original_filename = filename
end
attr_reader :original_filename
end
And then you can replace a.file = file with a.file = FileIO.new(file, 'new_filename')
I have a very strange bug i been fighting with for a day now.
I'm trying to make an automatic uploader for uploading hundreds of PDF-files and putting them into categories from my local computer to my remote Amazon s3 server using Carrierwave.
When i run it with :file in my pdf_uploader everything is working as it should be, but when i run it with fog trying to save them on the Amazon s3 server everything is duplicated. And not like 1,1,2,2,3,3 but 1,2,3,1,2,3
(i already have two image_uploaders and i tried uploading from a local directory earlier on with images, so the server and connection should be set up right)
Here are my files:
#routes.rb
get "upload_pdfs/now" => "pdf_categories#upload_pdfs", as: "upload_pdfs"
#pdf_categories_controller.rb
def upload_pdfs
path = "/Users/bruger/Desktop/DENHER"
Dir.foreach(path) do |category_path|
next if folder_or_file_excluded? category_path
pdf_category = PdfCategory.create!(title: category_path)
Dir.foreach("#{path}/#{category_path}") do |file_path|
next if folder_or_file_excluded? file_path
pdf_file_without_file_ending = remove_last_obj_in_arr file_path.split(".")
new_pdf = Pdf.new(
title: pdf_file_without_ending,
pdf_category_id: pdf_category.id,
)
#### SOMETHING IS HAPPENING HERE! ###
File.open(path + "/" + category_path + "/" + file_path) do |file_from_path|
new_pdf.file = file_from_path
new_pdf.save!
end
#### SOMETHING IS HAPPENING HERE! ###
end # file_path
end # category_path
redirect_to root_path
end
def is_integer? thing
thing.to_i.to_s == thing
end
def remove_last_obj_in_arr array
new_array = array.first array.size - 1
new_array.join(".")
end
def folder_or_file_excluded? folder_file
list = %w(Fravalgt Fravalg Valgt fra fravalg fravalgt JPEG . .git DS_Store .. .DS_Store image_upload.rb)
list.include? folder_file
end
#pdf_uploader.rb
class PdfUploader < CarrierWave::Uploader::Base
storage :fog
def store_dir
"pdf/#{mounted_as}/#{model.id}"
end
def extension_whitelist
%w(pdf PDF)
end
end
Have you experienced anything like this or do you have a clue?
Please. Any help will be appreciated!
I'm exporting some data from my server to the client.
It's an zip archive but when the amount of data is to big : TimeOut !
#On my controller
def export
filename = 'my_archive.zip'
temp_file = Tempfile.new(filename)
begin
Zip::OutputStream.open(temp_file) { |zos| }
Zip::File.open(temp_file.path, Zip::File::CREATE) do |zip|
#videos.each do |v|
video_file_name = v.title + '.mp4'
zip.add(video_file_name, v.source.file.path(:original))
end
end
zip_data = File.read(temp_file.path)
send_data(zip_data, :type => 'application/zip', :filename => filename)
ensure
temp_file.close
temp_file.unlink
end
end
I'm using PaperClip to attach my video on my app.
Is there any way to create and upload the zip (with a stream?) without a too long wait?
You could try the zipline gem. It claims to be "Hacks on Hacks on Hacks" so heads up! Looks very easy to use though, worth a shot.
I received files in mp3 (2 minutes/files) I want to concatenate together and create a bigger file. So I created my model a function to do this using ffmpeg and IO.popen
FileUtils.mkdir_p "#{Rails.root}/tmp/files"
imported_dir = "#{Rails.root}/tmp/files/#{SecureRandom.uuid}"
links.each_with_index do |link, index|
file_path = "#{imported_dir}_#{index}#{File.extname(link)}"
File.open(file_path, 'wb') do |file|
file.write open(link).read
end
concat_list << "file '#{file_path}'\n"
end
File.open("#{imported_dir}.txt", 'w'){ |f| f.write(concat_list)}
io = IO.popen("#{Rails.root}/lib/ffmpeg/ffmpeg -f concat -i #{imported_dir}.txt -c copy #{imported_dir}.mp3").readlines
if sound = Sound.create(user_id: user.id, file: File.open("#{imported_dir}.mp3"), lang: lang, title: title)
audio = FFMPEG::Movie.new("#{imported_dir}.mp3")
if !audio.valid?
puts "//_!_\\\\ Failed reading with ffmpeg (#{sound.id})#{sound.title} //_!_\\\\"
return false
end
end
the problem is that my .txt file containing the file path
file '/home/test/apps/example/releases/20150305224026/tmp/files/4dbe9707-cfef-467b-ab2c-a5e1e1165953_0.mp3'
created files as well but the final file is not created and i got the error message :
No such file or directory # rb_sysopen - /home/test/apps/example/releases/20150305224026/tmp/files/4dbe9707-cfef-467b-ab2c-a5e1e1165953.mp3
If anyone could help me
Make sure the target directory exists and is writable by the user running your app.
I have a small Rails 3.2.1 app that uses CarrierWave 0.5.8 for file uploads to S3 (using Fog)
I want users to be able to select some images that they'd like to download, then zip them up and send them a zip. Here is what I've come up with:
def generate_zip
#A collection of Photo objects. The Photo object has a PhotoUploader mounted.
photos = Photo.all
tmp_filename = "#{Rails.root}/tmp/" << Time.now.strftime('%Y-%m-%d-%H%M%S-%N').to_s << ".zip"
zip = Zip::ZipFile.open(tmp_filename, Zip::ZipFile::CREATE)
zip.close
photos.each do |photo|
file_to_add = photo.photo.file
zip = Zip::ZipFile.open(tmp_filename)
zip.add("tmp/", file_to_add.path)
zip.close
end
#do the rest.. like send zip or upload file and e-mail link
end
This doesn't work because photo.photo.file returns an instance of CarrierWave::Storage::Fog::File instead of a regular file.
EDIT: The error this leads to:
Errno::ENOENT: No such file or directory - uploads/photos/name.jpg
I also tried the following:
tmp_filename = "#{Rails.root}/tmp/" << Time.now.strftime('%Y-%m-%d-%H%M%S-%N').to_s << ".zip"
zip = Zip::ZipFile.open(tmp_filename, Zip::ZipFile::CREATE)
zip.close
photos.each do |photo|
processed_uri = URI.parse(URI.escape(URI.unescape(photo.photo.file.authenticated_url)).gsub("[", "%5B").gsub("]", "%5D"))
file_to_add = CarrierWave::Uploader::Download::RemoteFile.new(processed_uri)
zip = Zip::ZipFile.open(tmp_filename)
zip.add("tmp/", file_to_add.path)
zip.close
end
But this gives me a 403. Some help would be greatly appreciated.. It probably is not that hard I'm just Doing it Wrong™
I've managed to solve the problem with help from #ffoeg
The solution offered by #ffoeg didn't work quite so well for me since I was dealing with zip files > 500 MB which caused me problems on Heroku. I've therefor moved the zipping to a background process using resque:
app/workers/photo_zipper.rb:
require 'zip/zip'
require 'zip/zipfilesystem'
require 'open-uri'
class PhotoZipper
#queue = :photozip_queue
#I pass
def self.perform(id_of_object_with_images, id_of_user_to_be_notified)
user_mail = User.where(:id => id_of_user_to_be_notified).pluck(:email)
export = PhotoZipper.generate_zip(id_of_object_with_images, id_of_user_to_be_notified)
Notifications.zip_ready(export.archive_url, user_mail).deliver
end
# Zipfile generator
def self.generate_zip(id_of_object_with_images, id_of_user_to_be_notified)
object = ObjectWithImages.find(id_of_object_with_images)
photos = object.images
# base temp dir
temp_dir = Dir.mktmpdir
# path for zip we are about to create, I find that ruby zip needs to write to a real file
# This assumes the ObjectWithImages object has an attribute title which is a string.
zip_path = File.join(temp_dir, "#{object.title}_#{Date.today.to_s}.zip")
Zip::ZipOutputStream.open(zip_path) do |zos|
photos.each do |photo|
path = photo.photo.path
zos.put_next_entry(path)
zos.write photo.photo.file.read
end
end
#Find the user that made the request
user = User.find(id_of_user_to_be_notified)
#Create an export object associated to the user
export = user.exports.build
#Associate the created zip to the export
export.archive = File.open(zip_path)
#Upload the archive
export.save!
#return the export object
export
ensure
# clean up the tempdir now!
FileUtils.rm_rf temp_dir if temp_dir
end
end
app/controllers/photos_controller.rb:
format.zip do
#pick the last ObjectWithImages.. ofcourse you should include your own logic here
id_of_object_with_images = ObjectWithImages.last.id
#enqueue the Photozipper task
Resque.enqueue(PhotoZipper, id_of_object_with_images, current_user.id)
#don't keep the user waiting and flash a message with information about what's happening behind the scenes
redirect_to some_path, :notice => "Your zip is being created, you will receive an e-mail once this process is complete"
end
Many thanks to #ffoeg for helping me out. If your zips are smaller you could try #ffoeg's solution.
Here is my take. There could be typos but I think this is the gist of it :)
# action method, stream the zip
def download_photos_as_zip # silly name but you get the idea
generate_zip do |zipname, zip_path|
File.open(zip_path, 'rb') do |zf|
# you may need to set these to get the file to stream (if you care about that)
# self.last_modified
# self.etag
# self.response.headers['Content-Length']
self.response.headers['Content-Type'] = "application/zip"
self.response.headers['Content-Disposition'] = "attachment; filename=#{zipname}"
self.response.body = Enumerator.new do |out| # Enumerator is ruby 1.9
while !zf.eof? do
out << zf.read(4096)
end
end
end
end
end
# Zipfile generator
def generate_zip(&block)
photos = Photo.all
# base temp dir
temp_dir = Dir.mktempdir
# path for zip we are about to create, I find that ruby zip needs to write to a real file
zip_path = File.join(temp_dir, 'export.zip')
Zip::ZipFile::open(zip_path, true) do |zipfile|
photos.each do |photo|
zipfile.get_output_stream(photo.photo.identifier) do |io|
io.write photo.photo.file.read
end
end
end
# yield the zipfile to the action
block.call 'export.zip', zip_path
ensure
# clean up the tempdir now!
FileUtils.rm_rf temp_dir if temp_dir
end