The flow is:
The user selects an image on the client.
Only filename, content-type and size are sent to the server. (E.g. "file.png", "image/png", "123123")
The response are fields and policies for upload directly to S3. (E.g. "key: xxx, "alc": ...)
The case is that if I change the extension of "file.pdf" to "file.png" and then uploads it, the data sent to the server before uploads to S3 are:
"file.png"
"image/png"
The servers says "ok" and return the S3 fields for upload .
But the content type sent is not a real content type. But how I can validate this on the server?
Thanks!
Example:
Testing Redactorjs server side code (https://github.com/dybskiy/redactor-js/blob/master/demo/scripts/image_upload.php) it checks the file content type. But trying upload fake image (test here: http://imperavi.com/redactor/), it not allows the fake image. Like I want!
But how it's possible? Look at the request params: (It sends as image/jpeg, that should be valid)
When I was dealing with this question at work I found a solution using Mechanize.
Say you have an image url, url = "http://my.image.com"
Then you can use img = Mechanize.new.get(url)[:body]
The way to test whether img is really an image is by issuing the following test:
img.is_a?(Mechanize::Image)
If the image is not legitimate, this will return false.
There may be a way to load the image from file instead of URL, I am not sure, but I recommend looking at the mechanize docs to check.
With older browsers there's nothing you can do, since there is no way for you to access the file contents or any metadata beyond its name.
With the HTML5 file api you can do better. For example,
document.getElementById("uploadInput").files[0].type
Returns the mime type of the first file. I don't believe that the method used to perform this identification is mandated by the standard.
If this is insufficient then you could read the file locally with the FileReader apis and do whatever tests you require. This could be as simple as checking for the magic bytes present at the start of various file formats to fully validating that the file conforms to the relevant specification. MDN has a great article that shows how to use various bits of these apis.
Ultimately none of this would stop a malicious attempt.
Related
I saw N number of tutorials on uploading a file to active storage locally, to S3 etc using rails view. But I cannot find a legit source on how to upload a file to an active storage through API Say via postman, like what are prerequisites required to pass an attachment through API via postman?
Let me detail it.
Step 1: Through React or any frontend framework, I will choose a file to upload(a .PDF file), ultimately it should be saved somewhere.
Step 2: This chosen file should be passed through as API to the backend service where the storage of the chosen file to be saved in a storage like AWS S3.
How will the API request be sent to store the file? Can someone help me out with it?
its Very Simple to Do
Open the Postman
Go to Body -> Form-data
after that take a mouse to the key filed you will find Option for File
Then you have to select the file and send the request
Or send the File Path in the raw JSON object as well like this
and then Create the Object at backend so place check on file object if it blank then send path like this
// For Temporary Use Over Back End Only NOT NEED To Send These
"file_path" : "/home/jolly/Downloads/pdf/pdf-5.pdf",
"file_name" : "pdf-5.pdf"
and then in the code use it to create the file object which u pass where u needed to save s3 or rails storage
file_path = params[:file_path]
file_name = params[:file_name]
image_path = Rails.root+file_path
image_file = File.new(image_path)
I'm building a webapp which is offline available.
Therefore I need urls to images and videos were not expiring at any time.
ActiveStorage is not build for that case.
So I build my own media controller for serving images and videos based on the url params
get '/media/:kind/:type/:size/:token'
# /media/video/mp4/middle/ABC123
# /media/image/logo/large/ABC124
There is a MediaObject with a unique token. With the params I get the right variant of the image or the converted video file, look for the path of the blob file, seasoned with a quick lookup what filetype it is and serving it.
path = ActiveStorage::Blob.service.send(:path_for, variant_by(size, type).key)
send_file path, type: content_type, disposition: 'inline'
everything is working good, very good. The images and videos got a cacheable (better looking) permanent url.
Now there is an issue with Safari. Every browser can handle the mp4/webm file the controller returns. Except of safari. The video cannot be watched in the browser.
I'm guessing, there is something missing in the headers.
But I cannot find the place in active storage where the videos served.
When I use the default way
# example
Rails.application.routes.url_helpers.url_for(video.mp4)
the video can be played inline.
I've tried to add the content-length to the response header, add range infos and status :partial_content, add the file extension to the url, but nothing works.
The mime type is correct.
Rails 6 introduced a public option in storage.yml. Set it to true to override the expiring URL stuff: https://edgeguides.rubyonrails.org/active_storage_overview.html#public-access
Also, if you're still on Rails 5.2, you should be able to use:
rails_blob_url(imageable.image, disposition: :attachment)
I am making a rails app with video uploads. In testing any medium to big sized video totally wrecks my server logs since it tries to print out the video file. It sent the response but 10 minutes later it is still trying to print out the video file. Can I prevent rails from logging the params?
There's a way of suppressing arbitrary parameters in your configuration, so just include the name you want to suppress. For example:
Rails.application.config.filter_parameters += [ :video ]
This is described in the documentation and is by default loaded in config/initializers/filter_parameter_logging.rb. You could either edit that and add your params, or to make it modular, add another initializer file of the same format.
Note this won't block logging of SQL queries, so if you're inserting large BLOB type objects your log will be noisy. This is why it's usually better to dump the video on an object-store like Amazon S3.
there exist some sample code for an Http Server in the Dart:io section.
Now I will distribute images with this server. To achieve this, I read the requested image file and send its content to the client via request.response.write().
The problem is the format of the read data:
Either I read the image file as 16bit-String or as Byte Array. Neither of them is compatible to a raw 8-bit array, which I have to send to the client.
May someone help me?
There exist several kinds of write-methods in the response class.
write
writeCharCode
add
While "write" writes the data 'as seen', "writeCharCode" transforms the data back to raw-format. However, writeCharCode prepends some "magic byte" (C2) at the beginning, so it corrupts the data.
Another function, called add( List < int > ) processes the readAsBytes-result as desired.
Best regards,
Alex
In my web page (rendered by Rails), I'd like to let the user right-click on a photo to bring up the browser's Save As dialog, to let the user save the photo to their hard drive.
However, the photos on my server have unusual filenames (long hex names) with no file extension. The filename prompt in the Save As dialog has this ugly filename. If the user hits save, they'll end up with a poorly-named file, with no file extension.
The web page is aware of the photo's real file name (the name that came off the camera, for example). Is there a way for me to programmatically override the Save As dialog's filename prompt with a filename of my choosing?
I'm aware of the Content-Dispostion header, and that via this header a filename can be specified. However, I think that in order to be able to make use of this header, I need to load/render the entire file to the browser. If the asset to be made available for download is a movie, that loading of the file could timeout the browser...like, if it's a 100meg video.
Thoughts?
-A
I think I understand the problem here because I encountered (and resolved) at least part of it myself not too long ago.
I have some large mp3's and I link to them on my website
A few problems
I needed to set my content-disposition header to attachment in order to prevent files from automatically streaming whenever a user clicked the download button
my files are on a remote server
my files are large (100MB)
large files can tie up rails controllers if not handled properly
Now, Michael Koziarsky advises in this article that the best way to keep your rails processes free when serving large files, is to create a download action in your controller, and the do something like this (note the use of x_sendfile=>true):
def download
send_file '/path/to/podcast.mp3', :type => 'application/octet-stream', :disposition => 'attachment', :filename=>'something.mp3', :x_sendfile=>true
end
:x_sendfile tells apache to let the file through without tying up a rails controller process. The rest of the code sets the filename and the content-disposition header.
Great, but I'm on heroku, like everyone else nowadays. So I can't use x_sendfile.
I found that I couldn't modify the nginx configuration file either as it's locked down by heroku so it was not possible to get x-accel-redirect (nginx equivalent of x-sendfile) working
So, I decided to add a perl script (see below) to the cgi-bin on our asset-host and this script sets the content-disposition to attachment and gives our file a name too.
Instead of doing a restful download like this:
link_to "download", download_podcast_path(#podcast.mp3)
we just link to the mp3 making sure that we go in through the cgi-bin so that the perl script gets called on every mp3 that leaves the server
# I'm using haml
%a{:href=>"http://afmpodcast.com/cgi-bin/download.cgi?ID=#{#podcast.mp3}"}
download
The result is that my rails controller is no longer called into action when someone downloads a file
I found the perl script here and chopped it up a bit to work for me:
#!/usr/local/bin/perl -wT
use CGI ':standard';
use CGI::Carp qw(fatalsToBrowser);
my $files_location;
my $ID;
my #fileholder;
$files_location = "../";
$ID = param('ID');
open(DLFILE, "<$files_location/$ID") || Error('open', 'file');
#fileholder = <DLFILE>;
close (DLFILE) || Error ('close', 'file');
print "Content-Type:application/x-download\n";
print "Content-Disposition:attachment;filename=$ID\n\n";
print #fileholder
My code, is on github but you'll likely have all sorts of problems using it on your machine as i make heavy use of ENV variables that I store in bashrc and I have no documentation or tests ^hides^
You could do some smart server side url rewrite, like for example rewriting foo.mpeg to youveryuglyfilenamewithoutextension.
Set the Content-Disposition to "attachment; filename="...that's fine. "attachment" explicitly means it's not to be rendered in the browser, file renaming works nonetheless (or possibly particularly for that case).
Based on your comments, you have a few problems.
You want to set the filename using your Rails app.
The file is on a remote host and your Rails app is acting as a middleman.
The file might be big, so you want the file to be sent out to the browser as you receive it instead of queuing the whole thing.
Streaming only with Rails is tricky for a few reasons.
You would need an HTTP client that lets you access the message body as you receive data instead of blocking until you have everything. Net::HTTP is not that client. I'm not sure what library would be better suited.
Once you have a more event-driven way to get your file in pieces, you can pass a proc to the render:
render :text => proc { |response, output| ... }
output can be used like an IO object. Some servers may buffer before sending anyway, though, so that's something to look out for.
It would be easier not handle the byte-shuffling in Rails.
If your webserver or the proxy in front of your webserver supports the X-REPROXY-URL HTTP header, your application can set that header and your webserver or proxy will stream the file.
Perlbal is the only proxy server I know of that supports that header out of the box.
An Apache2 module is also available.