I've run into an annoying problem with Paperclip. Paperclip is working fine for uploading jpg/gif files but it's choking on .targa files with the error "not recognized by identify”. Just to confirm, it's working 100% with jpg/gif/png files and I have imagemagick installed and working, this error only occurs with .tga files.
The general process for paperclip is:
User uploads a file
Tempfile is created containing the contents of that file
The identify command is run on the Tempfile to get the width/height of the image.
With jpg/png/gif files, identify can run on file without needing a valid extension (eg .jpg) to be able to recognise the file type. However, when identifying a .tga file, it requires the ".tga" extension in the file name.
The problem is this:
When Paperclip creates the temp file, it creates with a name similar to: stream.0.1. Because this tempfile lacks the .tga extension, the identify command can not parse the dimensions of the image, thus causing the "not recognized by identify”.
I'm not sure how to go about fixing this, the best idea I can come up with is to use the regular File.new command rather than Tempfile.new to create the temporary file with a random name but the correct file extension. This would require also patching in custom code to remove the Files after processing has been done rather than relying on Ruby's inbuilt ability to delete tempfiles after use.
Does anybody have some ideas on the best method I could go about fixing this?
This was a bug in Paperclip and has been fixed. More details can be found here:
http://groups.google.com/group/paperclip-plugin/browse_thread/thread/7fd7a8d7bab696a7
Related
I'm having a lot of trouble finding a solution to uploading a file to a folder in Rails.
I have a file that I need to upload to a specific folder in the app 'public/uploads' with a specific name. Each time i upload, i need to run a pre-existing background job, which will remove the file after it's done.
If it happens that a file already exists, it should just overwrite it.
I can't find a solution that covers this. All the examples are things about attaching a file to an instance of a model and storing it in my DB. I don't need that. That's overkill for my scenario.
Just upload file to a folder, simple as.
Suggestions?
You can modify the file path. The easiest strategy is to add a randomly generated or sequential subfolder/filename prefix for each upload.
So Rails.root.join('public', 'uploads', uploaded_file.original_filename) becomes Rails.root.join('public', 'uploads', "#{my_random_value}-#{uploaded_file.original_filename}").
With Rails 6, I need to replace Paperclip, but I can't find any substitutions that actually easily replicate it.
Specifically, the file structure paperclip used:
:model/:attachmant_field/000/000/000/:identifier/:style/:original_file_name
Over the last decade we have built several tools that rely on that structure (or something similar) and in addition our users expect that after uploading an image, they can reference the styles with the same file name and a permanent url (not a randomly generated name like ActiveStorage and Shrine does) and change the "style" component in the url to a different one in their html.
I've spent several days both on Shrine and ActiveStorage working to get the file structure and naming to work on and keep failing, as despite being "natural replacements" they don't actually handle things in the same way.
Our end system is on Amazon S3, though integrating with that hasn't been the issue, just the file system.
Thanks for your help, it's been really frustrating having to remove something that works great when there seems to be nothing that actually replaces it, if you want/need things done in the same way. I'd rather not have to start rewriting all of tools that we developed and resetting our customers expectations to work with a new structure.
Thanks so much.
Have you tried Carrierwave? You can specify any storage path and build it dynamically using model name (model.class.to_s.underscore), attachment field (mounted_as), model id (model.id). The original file name is also available as original_filename.
I like to generate PDF-Files which also includes images uploaded by the user. The prawn gem works so far, and can also embed images located in the file system.
My problem is that I want to include user uploaded images, and where these images are stored will probably change in the future (from file system to some cloud service).
I use the dragonfly gem for handling the images, this uses rack to access the images and sometimes process them on the fly.
Now the simpliest idea does not work
(with report being my object and spot_image my image field)
image report.spot_image
no implicit conversion of #<Class:0x007fc07ecf1110> into String
I tried also to open the file via http with open-uri. This should work, but it blocks on my development machine, I think because the development rails server is single threaded:
image_path = report.spot_image.remote_url
image_url = "#{view.request.protocol}\#view.request.host_with_port}
/#{image_path.sub(/^\//,"")}"
image open(image_url) # timeout in development-mode
It may probably work in production, but even then it does a needless http-request. Can I ask dragonfly (or rack) directly for the image? prawn uses duck-typing and needs some object that responds to read and rewind as image.
I found relevant documentation about dragonfly file handling. The following did not work:
The report.spot_image.data method returns the image data as string, but prawn recognizes the data as path instead of as image data. The dragonfly tempfile method returns a closed temporary file, but prawn does not recognize that it can open it.
I had success with the file method, which returns an open file handle. It is unclear if prawn is closing this file than. So I used the block-style method to make sure the file is closed:
report.spot_image.file do |f|
image f
end
This works so far. (not yet tested with cloud storage)
My application allows a user to upload an image and then create different versions of that image (e.g. aligned and cropped with another image). The intermediate step is that I need to copy the uploaded file to another object and process it. The easiest way to do this is to upload it locally.
From the carrierwave wiki, they suggest using a modified version of StringIO.
On the carreirwave readme, they also suggest using File.open (something like obj.image=File.open('path_to_file').
I've also found references to using fixture_file_upload from ActionDispatch::TestProcess (usually in testing, but I'm unsure why it is restricted to that environment).
Can anyone give me a good explanation on the pros and cons (if any) of using these methods?
Thanks.
I just discovered one major difference, at least in the context of carrierwave. If you use carrierwave with the move_to_cache option set to true and mount your upload column with File.open, the file given to File.open will be moved, while with StringIO, it won't.
I want to do something what I thought will be a simple task:
Have a form with these controls:
File upload for one file
Checkbox if this file should be extracted
Text input where I would specify which file should I link to (required only if the checkbox is checked) - index_file
After submitting form:
If the checkbox isn't checked, upload the file via CarrierWave to S3 to the specified store_dir
If the checkbox is checked, extract all files from the archive (I expect only ZIP archives; I need to keep the directory structure), upload extracted files to the specified store_dir and set the index_file in database (I don't need to save to database anything about other extracted files)
As I have found, it isn't an easy task because of Heroku limitations. These files will have a large size (hundreds of MiBs or a few GiBs), so I don't want to redownload this file from S3 if possible.
I think that using Delayed Job or Resque might work, but I'm not exactly sure how to do it and what is the best solution of my problem.
Does anyone have any idea how to solve it with using the lowest resources as possible? I can change CarrierWave to another uploader (Paperclip etc.) and my hosting provider too if it isn't possible on Heroku.
I was also thinking about using CloudFlare, would this still work without problems?
Thank you for answers.
Based on this heroku support email, it would seem that the /tmp directory is many gigs in size. You just need to clean up after yourself so Heroku as a platform is not the issue.
A couple of articles may help you solve the problem:
https://github.com/jnicklas/carrierwave/wiki/How-to%3A-Make-Carrierwave-work-on-Heroku - which explains how to configure your app to use the /tmp directory as the cache directory for CarrierWave. Pay attention to the following line:
use Rack::Static, :urls => ['/carrierwave'], :root => 'tmp' # adding this line
This instructs rack to serve /carrierwave/xzy from the /tmp directory (useful for storing images temporarily)
Then, using the uploader.cache! method, you can deliberately cache the inbound uploaded file. Once stored, you can do checks to determine whether to call the uploader.store! method which will promote the contents to S3 (assuming you configured S3 as the store for CarrierWave.