Uploading apkg file with paperclip on Rails application - ruby-on-rails

I am trying to create a rails app that will upload Anki decks (apkg file extension) using paperclip. The error I am getting states:
"File has an extension that does not match its contents."
I am wondering what is the correct way to file map this extension. Thanks.

Well, you're running into Paperclip's file type validations. There's apparently a bunch of different things that can trigger this message. If, for instance, you're running on Windows, you have to manually install the file utility. As of version 4.0.0, every upload model needs to have a filetype validation of some sort, and has an un-removable validation to protect against filetype spoofing.
I don't actually know which of these you're running into, but that same link suggests solutions to each.
If you don't have any kind of filetype validation, you probably need to add some:
validates_attachment_file_name :column_name, :matches => [/apkg\Z/]
or, if not feeling security-conscious:
do_not_validate_attachment_file_type :column_name
For the second, well, I'm just going to quote the docs:
[The unavoidable spoofing check] can sometimes cause false validation errors in applications that
use custom file extensions. In these cases you may wish to add your
custom extension to the list of file extensions allowed for your mime
type configured by the mime-types gem:
# Allow ".foo" as an extension for files with the mime type "text/plain".
text_plain = MIME::Types["text/plain"].first
text_plain.extensions << "foo"
MIME::Types.index_extensions text_plain

Related

Errno::ENOENT: No such file or directory in finding duration of video from S3

Using gem paperclip-ffmpeg in rails for videos.It works fine but when I find the duration of video in seconds it gives me an error
Errno::ENOENT: No such file or directory - the file 'http://getpayad-dev.s3.amazonaws.com/ads/videos/000/000/014/original/Ufone_Tarzan_commercial_%28Ufone_Network_Quality%29_most_Funny_Ad.mp4?1451555000' does not exist
from /home/des0071/.rvm/gems/ruby-2.2.1/gems/streamio-ffmpeg-1.0.0/lib/ffmpeg/movie.rb:11:in `initialize
My code is
movie = FFMPEG::Movie.new("#{self.video.url}")
Well, the FFMPEG::Movie.new definition is found here: streamio-ffmpeg/movie.rb
raise Errno::ENOENT, "the file '#{path}' does not exist" unless File.exists?(path)
#ruby 2.2.0p0 (2014-12-25 revision 49005)
File.exists?("http://getpayad-dev.s3.amazonaws.com/ads/videos/000/000/014/original/Ufone_Tarzan_commercial_%28Ufone_Network_Quality%29_most_Funny_Ad.mp4?1451555000")
=> false
The problem is with ruby's File Class. So I tried this:
File.exists?("http://www.google.com")
=> false
OK, so either google isn't online or File can't take URI as a parameter.
A File is an abstraction of any file object accessible by the program and is closely associated with class IO File includes the methods of module FileTest as class methods, allowing you to write (for example) File.exist?("foo").
Class: File Ruby 2.2.0
So, File Class is really a child of IO, what does IO say?
Many of the examples in this section use the File class, the only standard subclass of IO. The two classes are closely associated. Like the File class, the Socket library subclasses from IO (such as TCPSocket or UDPSocket).
Class:IO Ruby 2.2.0
It looks like the reason for the error is due to inheritance, or because the gem is not designed to stream the file over http.
As the previous answer suggestions, ffmpeg doesn't appear to be able to fetch the file over HTTP -- it's expecting a local file.
Depending upon how your files are encoded, they may have metadata at the beginning or end of the file which contains this information.
Subsequently a potential approach is to grab the first or last ~100KB of the file and check for the MOOV atom/metadata there.

When using paperclip on Rails3, the some characters( # and ~) get erased or altered from the file name when uploading

I'm not sure if this is a paperclip issue. Tried it on gitlab and the same thing happened.
I have a back end for an iOS app written in Rails, and when I upload an image file with the # character in the filename, it gets erased upon uploading, if I have a file named,
aaa#2x.jpg
it gets saved as
aaa2x.jpg
Also, ~ gets converted into a _.
This is a problem because iOS apps presume that retina supported images are named with the #2x prefix.
I can regex the file name post upload and change it in the database and rename the file, but that seems like an odd hack to do, anyone have any idea whats happening? How to have the file name saved properly to begin with?
According to this article: http://en.wikipedia.org/wiki/HFS_Plus, you should be able to use any character, including NUL in file names. But OS APIs may limit some characters for legacy reasons.
It can be server or client issue, try to debug your application and check file name provided in request.request_parameters it should contain valid file name.
If you going to use uploaded files in URLs you should transliterate them before upload, this also resolve your problem. To do this you can use this extension:
module TransliteratePaperclip
def transliterate_file_name(paperclip_file)
paperclip_file=[paperclip_file] unless paperclip_file.is_a?(Enumerable)
paperclip_file.each do |file|
filename=read_attribute("#{file}_file_name")
if filename.present?
extension = File.extname(filename).gsub(/^\.+/, '')
filename = filename.gsub(/\.#{extension}$/, '')
self.send(file).instance_write(:file_name, "#{filename.parameterize}.#{extension.parameterize}")
end
end
end
end
# include the extension
ActiveRecord::Base.send(:include, TransliteratePaperclip)
put this code in /config/initializers/paperclip_transliterate.rb and in your paperclip model:
before_post_process { |c| transliterate_file_name(:file) }
where :file is attribute defined by has_attached_file.

How do you access the raw content of a file uploaded with Paperclip / Ruby on Rails?

I'm using Paperclip / S3 for file uploading. I upload text-like files (not .txt, but they are essentially a .txt). In a show controller, I want to be able to get the contents of the uploaded file, but don't see contents as one of its attributes. What can I do here?
attachment_file_name: "test.md", attachment_content_type: "application/octet-stream", attachment_file_size: 58, attachment_updated_at: "2011-06-22 01:01:40"
PS - Seems like all the Paperclip tutorials are about images, not text files.
In Paperclip 3.0.1 you could just use the io_adapter which doesn't require writing an extra file to (and removing from) the local file system.
Paperclip.io_adapters.for(attachment.file).read
#jon-m answer needs to be updated to reflect the latest changes to paperclip, in order for this to work needs to change to something like:
class Document
has_attached_file :revision
def revision_contents(path = 'tmp/tmp.any')
revision.copy_to_local_file :original, path
File.open(path).read
end
end
A bit convoluted as #jwadsack mentioned using Paperclip.io_adapters.for method accomplishes the same and seems like a better, cleaner way to do this IMHO.
To access the file you can use the path method:
csv_file.path
http://rdoc.info/gems/paperclip/Paperclip/Attachment#path-instance_method
This can be used along with for example the CSV reader.
Here's how I access the raw contents of my attachment:
class Document
has_attached_file :revision
def revision_contents
revision.copy_to_local_file.read
end
end
Please note, I've omitted my paperclip configuration options and any sort of error handling.
You would need to load the contents of the file (using Rubys File.open) into a variable before you show it. This may be an expensive operation if your app gets lots of use, so it may be worthwhile reading the contents of the file and putting it into a text column in your database after uploading it.
Attachment already inherits from IOStream. http://rdoc.info/github/thoughtbot/paperclip/master/Paperclip/Attachment
So it should just be "#{attachment}" or <% RDiscount.new(attachment).to_html %> or send_data(attachment). However you wanted to display the data.
This is a method I used for upload from paperclip to active storage and should provide some guidance on temporarily working with a file in memory. Note: This should only be used for relatively small files.
Written for gem paperclip 6.1.0
Where I have a simple model
class Post
has_attached_file :image
end
Working with a temp file in ruby so we do not have to worry about closing the file
Tempfile.create do |tmp_file|
post.image.copy_to_local_file(nil, tmp_file.path)
post.image_temp.attach(
io: tmp_file,
filename: post.image_file_name,
content_type: post.image_content_type
)
end

can't write uploaded files in rails

I'm still fairly new to rails but I've done file uploading before. I'm trying to implement very basic file uploading - nothing fancy, just upload the file, save it, and make a record of it. So here's my method for saving the file:
def self.save(upload,name)
directory='public/uploads'
ext=File.extname(upload.original_filename)
path=File.join(directory, "#{name}#{ext}")
File.open(path,'wb') { |f| f.write(upload.read) }
end
The file is apparently valid, as extname() gets the correct extension. The new file is created at the correct place. But somehow the writing fails, as the file is always empty. Doesn't matter what type of file I use. What could be going wrong? I'm using Rails 3.0 if it matters.
Try doing File.open(path,'wb') { |f| f.write(upload.read); f.close }
The IO buffer probably isn't flushing, but closing the file should flush it.
Also, I'd strongly recommend using a plugin such as paperclip for file uploads, simply because file uploads can be annoying to manage, and paperclip provides a very nice way to abstract most of that into conventions so you can just add a couple of columns and do model.upload = params[:file].

Rails Binary Stream support

I'm going to be starting a project soon that requires support for large-ish binary files. I'd like to use Ruby on Rails for the webapp, but I'm concerned with the BLOB support. In my experience with other languages, frameworks, and databases, BLOBs are often overlooked and thus have poor, difficult, and/or buggy functionality.
Does RoR spport BLOBs adequately? Are there any gotchas that creep up once you're already committed to Rails?
BTW: I want to be using PostgreSQL and/or MySQL as the backend database. Obviously, BLOB support in the underlying database is important. For the moment, I want to avoid focusing on the DB's BLOB capabilities; I'm more interested in how Rails itself reacts. Ideally, Rails should be hiding the details of the database from me, and so I should be able to switch from one to the other. If this is not the case (ie: there's some problem with using Rails with a particular DB) then please do mention it.
UPDATE: Also, I'm not just talking about ActiveRecord here. I'll need to handle binary files on the HTTP side (file upload effectively). That means getting access to the appropriate HTTP headers and streams via Rails. I've updated the question title and description to reflect this.
As for streaming, you can do it all in an (at least memory-) efficient way. On the upload side, file parameters in forms are abstracted as IO objects that you can read from; on the download side, look in to the form of render :text => that takes a Proc argument:
render :content_type => 'application/octet-stream', :text => Proc.new {
|response, output|
# do something that reads data and writes it to output
}
If your stuff is in files on disk, though, the aforementioned solutions will certainly work better.
+1 for attachment_fu
I use attachment_fu in one of my apps and MUST store files in the DB (for annoying reasons which are outside the scope of this convo).
The (one?) tricky thing dealing w/BLOB's I've found is that you need a separate code path to send the data to the user -- you can't simply in-line a path on the filesystem like you would if it was a plain-Jane file.
e.g. if you're storing avatar information, you can't simply do:
<%= image_tag #youruser.avatar.path %>
you have to write some wrapper logic and use send_data, e.g. (below is JUST an example w/attachment_fu, in practice you'd need to DRY this up)
send_data(#youruser.avatar.current_data, :type => #youruser.avatar.content_type, :filename => #youruser.avatar.filename, :disposition => 'inline' )
Unfortunately, as far as I know attachment_fu (I don't have the latest version) does not do clever wrapping for you -- you've gotta write it yourself.
P.S.
Seeing your question edit - Attachment_fu handles all that annoying stuff that you mention -- about needing to know file paths and all that crap -- EXCEPT the one little issue when storing in the DB. Give it a try; it's the standard for rails apps. IF you insist on re-inventing the wheel, the source code for attachment_fu should document most of the gotchas, too!
You can use the :binary type in your ActiveRecord migration and also constrain the maximum size:
class BlobTest < ActiveRecord::Migration
def self.up
create_table :files do |t|
t.column :file_data, :binary, :limit => 1.megabyte
end
end
end
ActiveRecord exposes the BLOB (or CLOB) contents as a Ruby String.
I think your best bet is the attachment_fu plug-in:
http://github.com/technoweenie/attachment_fu/tree/master
UPDATE: Found some more info here http://groups.google.com/group/rubyonrails-talk/browse_thread/thread/a81beffb93708bb3
Look into the plugin, x_send_file too.
"The XSendFile plugin provides a simple interface for sending files via the X-Sendfile HTTP header. This enables your web server to serve the file directly from disk, instead of streaming it through your Rails process. This is faster and saves a lot of memory if you‘re using Mongrel. Not every web server supports this header. YMMV."
I'm not sure if it's usable with Blobs, it may just be for files on the file system. But you probably need something that doesn't tie up the web server streaming large chunks of data.

Resources