I am able to 'POST' to a Rails application (with Paperclip) using XML instead of the standard web form (trying to do it from another Ruby script). However, I would like to include a binary file.
Is there any way to include the binary data within an XML tag? Or can I do something like B64 encode the data on the client and then decode it before it hits the Paperclip plugin?
UPDATE:
The browser sends a POST with this data (among others):
Content-Disposition: form-data; name="upload[upload]"; filename="foo.jpg"
Content-Type: image/jpeg
ÿØÿà�JFIF��`�`��ÿþ�Created by AccuSoft Corp.ÿÛ�C�...
I'd like to replicate that, but within XML
The short version is: use type="file", base64-encode the file, and put it inside a CDATA block. I originally found an explanation at this link:
http://techblog.floorplanner.com/2010/02/15/restful-uploading-of-files-using-xml/
That link appears to have died, so I recommend checking out the Internet Archive copy of the blog post:
http://web.archive.org/web/20100825030057/http://techblog.floorplanner.com/2010/02/15/restful-uploading-of-files-using-xml/
Also linked from that post is a gem that implements an encoder for files posted to Rails as XML: https://github.com/nragaz/encoded_attachment
Related
I have setup a rest API inside a ruby on rails application, I now have a requirement to generate a PDF and return this PDF from a get request. I am looking for some advice on how to implement this feature.
Some of the requirements that I have are as follows: I can't save the file and give the end user a link to the file because the data in the file can be updated at any time. I am using the application as microservice so there isn't a front end that I can use to display the file.
So here is my thinking I would love some advice on how to implement this feature.
I would like to make a get request to a specific endpoint in the application. I expect a PDF file to be returned which I can then display to the end user.
I am currently using WickedPdf gem to generate a temporary PDF file, but I am really struggling with how the response should look.
Any advice would be much appreciated.
One way is to create a PDF file in memory and stream it to the client. I prefer this way, maybe later you will have to send PDF files via email, or just save them to some backup disk etc...
def get_pdf
pdf = WickedPdf.new.pdf_from_string('<h1>Hello There!</h1>')
send_data pdf, filename: 'file_name.pdf'
end
You can put the PDF generation to a different service and just call it in the controller. This provides isolation and you can test it separately.
Also you can debug the endpoint response with HTTPie http get http://localhost:3000/invoices/1/get_pdf
Rails will set all the necessary HTTP response headers:
Content-Disposition: attachment; filename="file_name.pdf"
Content-Length: 5995
Content-Transfer-Encoding: binary
Content-Type: application/pdf
So when the user clicks on a link that points to the endpoint, most probably the download dialog will pop up because of the Content-Disposition: attachment; header
Other solution is to render the get_pdf.html as PDF and send back to the client:
def get_pdf
render pdf: "file_name"
end
But in this case the Content-Disposition header will be inline, which means the browser will open the pdf (if it can read PDF format) instead of offering to download it.
Upload pdf to Amazon s3 and generate link then get pdf link in apis.
I don't know if you still need this, but for anyone in the future I found a nice solution:
pdf = WickedPdf.new.pdf_from_string(render_to_string "entradas/entradaspdf.pdf.erb")
send_data pdf, filename: "bergha.pdf", disposition: "inline"
I'm loading my pdf-html-view based template through "render_to_string" ruby method which returns the view contents in string. Then WickedPdf converts it to a pdf binary, and finally save that to "pdf" var.
Finally instead of "render" I use the "send_data" method, where first parameter is the output data (my pdf var), second is the filename of the output data, and third (optional) is to change Content-Disposition header to tell browser whether to load the file (inline) or just download it (attachment).
Hope it works, it does just fine for me
I am making a POST request from my Ruby on Rails controller to an API. The API needs to receive a multipart form where one part is JSON and the other part is a file.
Here's my best stab at it so far:
Api::Document.post("document_versions",
user_id: params[:user_id].to_i,
client_token: params[:client_token],
file_name: params[:file_name],
doc_name: params[:doc_name],
doc_type_id: params[:doc_type_id].to_i,
file: params[:upload].tempfile,
"Content-type" => "multipart/form-data; boundary= SomethingToSeparateParts")
On the API side, it receives
{"method":"POST","auth_token":"12345","timestamp":"2015-05-26T20:18:16Z","status_code":500,"request_url":"/v1/document_versions","duration":"3.902005ms","user_agent":"Faraday v0.9.0","request_body":"{\"user_id\":138,\"client_token\":\"12345\",\"file_name\":\"benTest.png\",\"doc_name\":\"BENTEST\",\"doc_type_id\":1,\"file\":[\"�PNG\\r\\n\",\"\\u001A\\n\",\"\\u00
...
0000\\u0000IEND�B`�\"],\"Content-type\":\"multipart/form-data; boundary= SomethingToSeparateParts\"}"}
So I think I have it coming as multipart/form-data, but it only has one part -- the JSON containing all attributes, including the file. I would like the JSON to contain everything BUT the file, and the file to be separated to another part of the multipart, separated by the boundary. How can I do this?
In our rails application we have a many actions that do regular webapp actions. But, we have a single action that accepts a large XML file. I would like to keep rails from parsing the XML into params. Instead, I would like to be able to get the URL params ( /documents/{id}/action ) and then write out the xml file to a specific directory. How do I keep Rails from processing it?
How would I define the action to handle this?
def handle_xml
# what to put here
end
The upload is done using Content-Type: application/xml It is a single file, and not part of a multipart form. The sample curl statement would be:
curl-H 'Accept: application/xml' -H 'Content-Type: application/xml' -X POST -d '<?xml version="1.0" encoding="UTF-8"?><test></test>' http://0.0.0.0:3000/controller/handle_xml
If you want to prevent rails from automatically parsing the XML data into a hash of parameters, you'll have to replace the ParamsParser middleware with your own custom version.
When a file is posted to rails, the ParamsParser middleware modifies the request parameters and turns it into a Hash if the data format is xml. You can find the details in the params_parser.rb file in rails.
Here's a RoR mailing list message similar to the question that you've asked
Unfortunately, as a new user I can't post any more links, but you should search google with "Sanitizing POST params with custom Rack middleware" for some more details on writing custom rack middleware.
I too have come across this problem recently. However mine is in an internal application where I have full control over both the Rails app and the clients connecting to it.
In my app the client POSTs a large XML data set to the Rails app. I wanted to process the XML document in a delayed job (resque). My workaround was to make the client use an alternate content-type. I used application/octet-stream. This prevents Rails from parsing the POST data. The data is available in request.raw_post.
The action should receive it as a file (through way of multipart form upload) and then store it as a temporary file for you.
Have you tried sending the xml file has one variable in the http uri request? So something like
#xml_file = xml..xml...xml...
parameters => {
query => {
xml_file => #xml_file
}
}
Httparty.post("url", parameters)
Then in your method:
def handle_xml
#xml_file = params[:xml_file]
#xml_file.save (or whatever you want here..)
end
I'm using restclient for a multipart form to send data to a restful web service (it's Panda video encoding service).
The trick though, is that the file I am passing into restclient (Technoweenie branch) is coming from my own form that a user submits.
So, lets walk through this. A user posts a file to my rails app. In my controller, it receives the file from params[:file]. I then want to pass params[:file] down to Panda using RestClient.
The error I'm getting is on the Panda server follows. I noticed that the file param in the stack trace is in a string as well (which I assume is Panda turning into a string for a nicer stacktrace).
~ Started request handling: Wed Aug 12 18:05:15 +0000 2009
~ Params: {"format"=>"html", "multipart"=>"true", "account_key"=>"SECURE_KEY", "action"=>"upload", "id"=>"SECURE_ID", "controller"=>"videos", "file"=>"#<File:0xcf02ca4>"}
~ 9bfb1750-6998-012c-4509-12313900b0f6: (500 returned to client) InternalServerErrorcan't convert nil into String
/var/local/www/panda/app/models/video.rb:246:in `extname'
/var/local/www/panda/app/models/video.rb:246:in `initial_processing'
/var/local/www/panda/app/controllers/videos.rb:79:in `upload'
I doubt you can really pass a CGI-style upload param from Rails into restclient and expect it to work.
A regular upload in Rails would have quite some extra attributes which do not belong in a posted resource (like the original filename and so on), and a Rails upload contains an IO with the actual file data. Also a file upload object in Rails might be a Tempfile handle and might be a StringIO - depending on the size of the upload.
What you effectively need to do is "repackage" your upload for rest-client to handle it properly, and pass the repackaged and rewound Tempfile object to restclient. Maybe you can get away with just picking the upload object itself instead of the whole params[:file]
Confirm that your restclient action can save locally first. If the action cannot save locally, then you will have a better idea where to look while trouble shooting.
Looks like the problem is with rest-client's posting of the file, check out an alternative method for posting like curb.
Lots of examples for posting multipart form data on this question: Ruby: How to post a file via HTTP as multipart/form-data?
I'm trying to load-test a Rails application using JMeter. A critical part of the application involves a form that includes both text inputs and file uploads. It works fine in a browser, but when I try to post that page in JMeter, Rails is saving all of the parts of the multipart form as temp files, which causes things to break when it's looking for a string and gets a tempfile instead.
It appears that the difference is that, from a browser, the piece of the multipart request that contains a text input looks like this:
-----------------------------7d93b4186074c
Content-Disposition: form-data; name="field_name"
test
-----------------------------7d93b4186074c
while from JMeter it looks like this:
-----------------------------7d159c1302d0y0
Content-Disposition: form-data; name="field_name"
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
test
-----------------------------7d159c1302d0y0
So apparently Rails sees the former and interprets it as a plain text value and treats it as a string, but sees the latter and saves it to a temp file.
I have not been able to find a setting to convince JMeter not to send the additional headers in the multipart form for non-file fields.
Is there a way to convince Rails to ignore those headers and treat the text/plain text as strings instead of text files? Or a quick way to put a filter in front of my controller that will strip the extra headers?
Alternately, is there a better tool to load-test a Rails application that includes file upload?
Turns out these days you can just tick "use browser compatible headers" in JMeter. Could've saved myself a hell of a lot of time there :-)
So, I have customized JMeter's multipart request posting part in the source code to put out the request that rails understand. The change is easy as shown below but to create compiling Java/JMeter environment took time. :(
Anyways, now I can successfully upload a file by multipart post via JMeter.
in src/protocol/http/org/apache/jmeter/protocol/http/sampler/PostWriter.java
writeStartFileMultipart()
//writeln(out, "Content-Transfer-Encoding: binary"); // $NON-NLS-1$
writeFormMultipart()
/*****
writeln(out, "Content-Type: text/plain; charset=" + charSet); // $NON-NLS-1$
writeln(out, "Content-Transfer-Encoding: 8bit"); // $NON-NLS-1$
*****/
P.S.
A tip tip to create the build environment for 2.4 is
to comment out the 3rd party libraries check in build.xml file.
copy lib/xstream-1.3.1.jar from binary archive into lib/ directory
There may be a better way, but I ended up adding a quick filter to turn the text/plain tempfiles into strings within the parameter hash:
def change_text_files_to_strings
params.each_pair do |key, value|
params[key] = value.read if (value.class.to_s=='Tempfile' && value.content_type.start_with?('text/plain') )
end
end
By the way, it turns out that jmeter is correct here, and rails incorrect: according to RFC 2388, each item in a multipart request should have a content type (not just files), so Rails really shouldn't be using the presence of a content-type header to determine whether it's a file. Ah well.
I also used the solution above as ColdFusion was sending similar headers (minus the Content-Transfer-Encoding) with each piece of form data. I wonder if there's a better way.
EDIT: Anyone know if this has been fixed in Rails 3?
What kind of error do you get? Something like
NoMethodError (undefined method `rewind' for "1":String):
There is an issue with Rack that could explain your problem. See https://github.com/rack/rack/issuesearch?state=open&q=rewind#issue/116
We were also having a similar issue, In addition to the above answers we also correlate the X-CSRF-Token of HTTP Header Manager in that request and were
successfully able to upload the required media as many as times we wanted.