How to read form-data with ruby - ruby-on-rails

In my controller the result of request.body.read is:
=============
--AJAX-----------------------1267183973160
Content-Disposition: form-data; name="1261400603_page_white_word.png"; filename="1261400603_page_white_word.png"
Content-Type: application/octet-stream
thefile
--AJAX-----------------------1267183973160
Content-Disposition: form-data; name="1261400536_page_white_excel.png"; filename="1261400536_page_white_excel.png"
Content-Type: application/octet-stream
thefile
--AJAX-----------------------1267183973160--
=============
It contains n form-data (2 in my example), my goal is to loop through the n form-data and get the data name, filename and a file uploaded, in my example I replaced the binary data by thefile.
here is the content of the params hash
{"action"=>"create", "controller"=>"photos", "1265144945_395.jpg"=>#<File:/var/folders/BT/BTpdsWBkF6myaI-sl3+1NU+++TI/-Tmp-/RackMultipart20100226-273-1un364r-0>}
Cheers

Have you considered using paperclip or attachment_fu? They are battle-tested and will do better than using request.body. In any case, you could parse request.body as follows but it's not the best solution.
inputs = request.body.read.split(/--ajax-+\d+-*/mi)
inputs.each do |input|
input.scan(/(.*)[\n\r]{2,4}(.*)/m) do |header, file|
header.scan(/name=["'](.*)["'];\s+filename=["'](.*)["']/) do |name, filename|
puts name
puts filename
end
puts file
end
end
Edit: So that params parsing is probably the job of Rack::Utils::Multipart.parse_mulitpart. One should probably reuse the regex's from the source of that to parse this a bit more robustly. Also, it looks like rack is creating a tmp file for you of some sort. Can you check the contents of that file?

Related

Content-Type for one file in multipart/form-data with HTTPie

I use HTTPie to POST a
multipart/form-data request (passing the -f option). It includes a
file field (using a # option). The corresponding part of the
multipart request has a pseudo-header Content-Disposition, but does
not have a pseudo-header Content-Type.
How is it possible to add such a Content-Type for a specific file?
For completeness, here is what I send now (as shown by -p B):
--xyz
Content-Disposition: form-data; name="file"; filename="file.txt"
Hello, world!
This is the content of the file.
--xyz
but here is what I need to send:
--xyz
Content-Disposition: form-data; name="file"; filename="file.txt"
Content-Type: text/plain
Hello, world!
This is the content of the file.
--xyz
Or to put it another way, like the CURL equivalent option: -F "file=file.txt;type=text/plain".
This worked for me:
http -f POST :8080/submit-file metadata=#metadata.json files#file1.jpeg files#file1.jpeg
This documentation helped me: https://httpie.org/doc#file-upload-forms
The =# makes httpie send it with content type text/plain.
This isn't possible with httpie. There is an issue for it, with some pull requests to fix it, but none have been merged yet.
I have just noticed that the documentation for multipart/form-data now reads like the following (at the end):
When uploading files, their content type is inferred from the file
name. You can manually override the inferred content type:
$ http -f POST httpbin.org/post name='John Smith' cv#'~/files/data.bin;type=application/pdf'
I do not know when that happened, but it seems that now:
the "part Content-Type" is inferred automatically
it is possible to override it from the command line option
So all I was looking for! :-)
Ref: https://httpie.org/docs#file-upload-forms.

rails get request body 0byte

I get a request.body from web service
fp = File.open("/home/mm/mms/video_rest/video_mp.mp4", "wb")
fp.write(request.body.readline)
fp.close
but when file are create size a 0 bytes
how to view if request body are a file size or how to best way to get a video file from request body?
UPDATE
have a this params
{"video"=>#<ActionDispatch::Http::UploadedFile:0x007febdc497da0 #tempfile=#<Tempfile:/tmp/RackMultipart20151007-3197-14dis8n.mp4>, #original_filename="VID_20151006_153121393.mp4", #content_type="application/octet-stream", #headers="Content-Disposition: form-data; name=\"video_presentacion\"; filename=\"VID_20151006_153121393.mp4\"\r\nContent-Type: application/octet-stream\r\nContent-Transfer-Encoding: binary\r\n">}
how to create a file on folder and change Content-Type application/octet-stream for 'video/mp4'?
im try with:
fp = File.open("/home/mm/aa/video_rest/video_mp.mp4", "wb")
fp.write(params[:video])
fp.close
or direct for paperclipt
usuario.update_attributes!(:video => params[:video])
result => Content type invalid
Solved
on android http using a on params add a content type("video/mp4")
and work fine !
regards!

Opening File in Ruby returning empty file

I am currently trying to store a pdf in a hash for an api call in ruby. The path to the pdf is stored as:
/Users/myUserName/Desktop/REPOSITORY/path_to_file.pdf
I am using a block to store the file in the hash as so:
File.open(pdf_location, "rb") do |file|
params = {
other irrelevant entries
:document => file
}
pdf_upload_request('post', params, headers)
end
I am receiving a 400 error from the server that my document is empty and when I do puts file.read, it is empty. However, when I visit the filepath, it's clear that the file is not empty. Am I missing something here? Any help would be greatly appreciated. Thank you!
Edit------
I recorded my http request with vcr, here it is:
request:
method: post
uri: request_uri
body:
encoding: US-ASCII
string: ''
headers:
Authorization:
- Bearer 3ZOCPnwfoN7VfdGh7k4lrBuEYs4gN1
Content-Type:
- multipart/form-data; boundary=-----------RubyMultipartPost
Content-Length:
- '246659'
So i don't think the issue is with me sending the file with multipart encoding
Update--------
The filepaths to the pdf are generated from a url, and stored in a the tmp folder of my application. They are generated through this method:
def get_temporary_pdf(chrono_detail, recording, host)
auth_token = User.find(chrono_detail.user_id).authentication_token
# pdf_location = "https://54.84.224.252/recording/5/analysis.pdf/?token=Ybp37kw7HrSt8NyyPnBZ"
pdf_location = host + '/recordings/' + recording.id.to_s + '/analysis.pdf/?format=pdf&download=true&token=' + auth_token
filename = "Will" + '_' + recording.id.to_s + '_' + Date.new.to_s + '.pdf'
Thread.new do
File.open(Rails.root.join("tmp",filename), "wb") do |file|
file.write(open(pdf_location, {ssl_verify_mode: OpenSSL::SSL::VERIFY_NONE}).read)
end
end
Rails.root.join("tmp",filename)
end
They are then called using the api call:
client.upload_document(patient_id, file_path, description)
I can see them physically in my temp folder, and can view them with preview. Everything seems to work. But as a test of uncertainty, I changed file_path to point to a different pdf:
Users/myUsername/Desktop/example.pdf.
Using this file path worked. The pdf was uploaded to the third party system correctly, I can physically see it there. Do you think this means it is an issue with the tmp folder or how i generate the temporary pdf's?
Most likely, the API is expecting a POST with Content-Type: multipart/form-data. Just sending the file handle (which document: file does) won't work, as the file handle is only relevant to your local Ruby process; and even sending the binary string as a parameter won't work, since your content-type isn't properly set to encode a file.
Since you're already using HTTParty, though, you can fix this by using HTTMultiParty:
require 'httmultiparty'
class SomeClient
include HTTMultiParty
base_uri 'http://localhost:3000'
end
SomeClient.post('/', :query => {
:foo => 'bar',
:somefile => File.new('README.md')
})
Try this:
file = File.read(pdf_location)
params = {
# other irrelevant entries
document: file
}
headers = {}
pdf_upload_request('post', params, headers)
Not sure but may be you need to close file first...
So the issue arose from the multi threading i used to avoid timeout errors. The file path would get generated and referenced in the api call before anything was actually written into the document.

Post JPEG file using fiddler with other body data

I'm trying to post a jpeg file to a locally developed web service via Fiddler. This would be simple enough, but I also need to include some data alongside the file and can’t quite nail the syntax that fiddler wants. If I click the upload file button and select a file to upload, it replaces my POST body with:
---------------------------acebdf13572468
Content-Disposition: form-data; name="fieldNameHere"; filename="PantheraLeo.jpg"
Content-Type: image/jpeg
<#INCLUDE *C:\temp\PantheraLeo.jpg*#>
---------------------------acebdf13572468—
Now I want to add some additional data:
user=1&album=2&photo=[OUTPUT FROM FILE UPLOAD]
I’ve tried putting this at the start of the body, but when my Node app receives the request, I’m getting a user parameter, an album parameter but no photo.
Any ideas on how I could format this request to get both parameters and the photo uploaded as the photo parameter?
I've also been looking to do something similar myself and stumbled on your question. I've just managed to achieve what I needed after a bit of messing about with Fiddler. Try this:
---------------------------acebdf13572468
Content-Disposition: form-data; name="model"
MyModelInfo
---------------------------acebdf13572468
Content-Disposition: form-data; model="test123"; filename="123.gif"
Content-Type: image/gif
<#INCLUDE *Z:\Downloads\123.gif*#>
---------------------------acebdf13572468--
It would seem that you link the form data being sent up in your request body to the 'acebdf13572468' boundary in the POST info. Provide the Content-Disposition with a key name (in my case 'model') and then the following line represents your actual value for this key. (In my case "MyModelInfo".
Using the above request body I was able to POST up a file as well as some accompanying POST data to my API.
The accepted answer works well. But be warned the extra line after MyModelInfo comes through into the string. Also when copying and pasting in and out of fiddler some lines were corrupted breaking the file.
Note I have named the file param "file" in the fiddler body and in the receiving API function.
This works for me:
---------------------------acebdf13572468
Content-Disposition: form-data; name="PARAM1"
Some text with a line before but not after
---------------------------acebdf13572468
Content-Disposition: form-data; name="file"; filename="filename.jpg"
Content-Type: image/jpeg
<#INCLUDE *C:\local\filename.jpg*#>
---------------------------acebdf13572468--
The data can be received in .net core 2 like this:
[HttpPost]
[Route("AddImage")]
public async System.Threading.Tasks.Task<IActionResult> AddImageAsync(IFormFile file)
{
//f is the same as file
//var f = Request.Form.Files.Count > 0 ? Request.Form.Files[0] : null;
//useful to check the keys
//var additionalProperties = Request.Form.Keys;
if (file != null)
{
try
{
if (Request.Form.TryGetValue("PARAM1", out StringValues p1))
{
var txt = p1.ToString():

How to Extend IOS SimpleURLConnections HTTP POST to Send Description Decodable by CGI Perl

I would like to extend the sample ios code called SimpleURLConnections. This sample code can upload a file using HTTP POST. I have enabled my apache web server with a cgi script written in Perl to receive the HTTP POST that addresses the cgi script and successfully uploads the file. The extension I would like to make is to add a title or description of the uploaded file. I have tried to update the multipart form data in the sample code and a corresponding change to my cgi script to transfer this description, but it is failing. The cgi script cannot read in the description using the cgi.pm method called param(). I'm hoping someone can help. Here are the details:
The multipart form data BEFORE my modification looks like the following (NOTE: the SimpleURLConnections routine inserts the image file between the Content-Type and the 2nd boundary on the fly):
--Boundary-D4AFFBA9-AD92-4697-9184-7BDA128C3B97
Content-Disposition: form-data; name="fileContents"; filename="1234567/TestImage1.png"
Content-Type: image/png
--Boundary-D4AFFBA9-AD92-4697-9184-7BDA128C3B97
Content-Disposition: form-data; name="uploadButton"
Upload File
--Boundary-D4AFFBA9-AD92-4697-9184-7BDA128C3B97--
The multipart form data AFTER my modification looks like the following:
--Boundary-D4AFFBA9-AD92-4697-9184-7BDA128C3B97
Content-Disposition: form-data; name="Title"; title="Hello World"
Content-Type: text/plain
--Boundary-D4AFFBA9-AD92-4697-9184-7BDA128C3B97
Content-Disposition: form-data; name="fileContents"; filename="1234567/TestImage1.png"
Content-Type: image/png
--Boundary-D4AFFBA9-AD92-4697-9184-7BDA128C3B97
Content-Disposition: form-data; name="titleButton"
my Title
--Boundary-D4AFFBA9-AD92-4697-9184-7BDA128C3B97
Content-Disposition: form-data; name="uploadButton"
Upload File
--Boundary-D4AFFBA9-AD92-4697-9184-7BDA128C3B97--
My cgi script looks like the following:
#!/usr/bin/perl -wT
use strict;use CGI;
use CGI::Carp qw ( fatalsToBrowser );
use File::Basename;
my $upload_dir = "/Library/WebServer/Documents/upload";
my $query = new CGI;
my $title = $query->param("Title");
#if (!$title) {
# die "the title is $title. $!";
#}
if ( !$filename ) {
print $query->header ( );
die "There was a problem uploading your photo (try a smaller file). : $!";
exit;
}
...
open ( UPLOADFILE, ">$upload_dir/$filename" ) or die "$!";
while ( <$upload_filehandle> ) {
print UPLOADFILE;
}
close UPLOADFILE;
This code works fine and the uploaded file gets written to the upload_dir.
But, if I uncomment that little block of code:
if (!$title) {
die "the title is $title. $!";
}
the script dies with an error that indicates that $title has no value:
[Tue Sep 27 17:09:17 2011] [error] [client 10.0.1.2] [Tue Sep 27 17:09:17 2011] upload.cgi: Use of uninitialized value $title in print at /Library/WebServer/CGI-Executables/upload.cgi line 79.
So, is the problem with my multipart form data, the cgi script, or is it not possible to pass this data in the multipart form data? Is there an alternate way?
Thanks
Ok, after getting help from #AFresh1, I found the problem. The problem was in the details of how the multipart form data was created. First I will show the wrong way.
NSString *title = #"Hello World";
bodySuffixStr = [NSString stringWithFormat: // create the footer for the POST message
#
"\r\n"
"--%#\r\n"
"Content-Disposition: form-data; name=\"titleButton\"\r\n"
"%#\r\n\r\n"
"--%#\r\n"
"Content-Disposition: form-data; name=\"uploadButton\"\r\n"
"\r\n"
"Upload File\r\n"
"--%#--\r\n"
"\r\n",
boundaryStr,
title,
boundaryStr,
boundaryStr
];
Now the correct way (a very subtle change in the 7th line - moved the CR/LF):
NSString *title = #"Hello World";
bodySuffixStr = [NSString stringWithFormat: // create the footer for the POST message
#
"\r\n"
"--%#\r\n"
"Content-Disposition: form-data; name=\"titleButton\"\r\n"
"\r\n%#\r\n"
"--%#\r\n"
"Content-Disposition: form-data; name=\"uploadButton\"\r\n"
"\r\n"
"Upload File\r\n"
"--%#--\r\n"
"\r\n",
boundaryStr,
title,
boundaryStr,
boundaryStr
];
I believe the content goes in the space after the header
and before the next boundary not in a title attribute. The part header being everything after the boundary and before the blank line.
If you are want to get "Hello World" into $title using $query->param('Title') I think you will need something more like this (note that Content-type defaults to text/plain):
--Boundary-D4AFFBA9-AD92-4697-9184-7BDA128C3B97
Content-Disposition: form-data; name="Title"
Hello World
--Boundary-D4AFFBA9-AD92-4697-9184-7BDA128C3B97

Resources