I'm been playing with calling my rails controller using an HTTP POST. I can get it to work with a curl command such as this, given a model named item and an attribute in that item called name:
curl -X POST -d "<item><name>myname</name></item>" -H "Content-Type: text/xml" http://localhost:3000/items.xml
What I'm curious about is how to make the same call using text instead of xml as my content type.... I tried:
curl -X POST -d "name=myname" http://localhost:3000/items.xml
but that seems to pass the 'name' parameter as the top level scope; so it doesn't end up in my params in the controller....
I'm a noob at this; just want to understand how to do it both ways....
Thanks!
Short answer, you need to
(a) send it as -H "Content-type: application/x-www-form-urlencoded".
(b) specify the parameters as Object[field]=value - for example User[name] would refer to the name field in some user object.
(c) [Not Required because you use curl] encode the parameters and POST/PUT them.
Encoding the parameters
The encoding is simple enough and although curl will do it for you, it's useful to understand it. I've quoted an extract from the W3C spec...
"
application/x-www-form-urlencoded
This is the default content type. Forms submitted with this content type must be encoded as follows:
Control names and values are escaped. Space characters are replaced by +', and then reserved characters are escaped as described in [RFC1738], section 2.2: Non-alphanumeric characters are replaced by%HH', a percent sign and two hexadecimal digits representing the ASCII code of the character. Line breaks are represented as "CR LF" pairs (i.e., `%0D%0A').
The control names/values are listed in the order they appear in the document. The name is separated from the value by =' and name/value pairs are separated from each other by&'.
"
Simple Example (New User Form)
The example below is how to send a simple "New User" form.
If I have fields in the user object for name, password, email etc, I specify them like this ...
user[firstname]=chris&user[login]=cmccauley&user[company_id]=8&user[email]=&user[surname]=mccauley
then curl will escape them to give ...
user%5Bfirstname%5D=chris&user%5Blogin%5D=cmccauley&user%5Bcompany_id%5D=8&user%5Bemail%5D=&user%5Bsurname%5D=mccauley
... before posting them like this extract from a wire dump ...
Wireshark dump
POST /users/19 HTTP/1.1
Host: localhost:3000
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.10) Gecko/2009042523 Ubuntu/9.04 (jaunty) Firefox/3.0.10
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: http://localhost:3000/users/19/edit
Cookie: _my_session=048d330143de668e027c8cd52654e8c5
Content-Type: application/x-www-form-urlencoded
Content-Length: 259
user%5Bfirstname%5D=chris&user%5Blogin%5D=cmccauley&user%5Bcompany_id%5D=8&user%5Bemail%5D=&user%5Bsurname%5D=mccauley&user_password=********&user%5Bjob_id%5D=14&user%5Bpassword%5D=dd793a64b74e108fcdc5d809040e24afcc21ad2c&authenticity_token=&id=19&_method=PUT
"Content-type: application/x-www-form-urlencoded" is already default with curl -d, no need for anything extra
-X POST is superfluous, as -d implies POST
-d does not URL encode the data. To get curl to do that for you, you need to use --data-urlencode instead
Related
I'm trying to send a vCard to my iOS using curl, but the vCard is not properly rendered in my iPhone.
EXCLAMATION_MARK='!'
curl -X POST https://api.twilio.com/2010-04-01/Accounts/<acc>/Messages.json \
--data-urlencode "Body=Test6$EXCLAMATION_MARK" \
--data-urlencode "MediaUrl=https://mighty-health-assets.s3.amazonaws.com/vcf/James.vcf" \
--data-urlencode "From=+14155926669" \
--data-urlencode "To=+14159108243" \
-u <acc>:<token>
I tried different Content-Type and Content-Disposition with my file.
For Content-Disposition:
inline; filename="James.vcf"
attachment; filename="James.vcf"
inline; name="James"
attachment; name="James"
For Content-Type:
text/vcard
text/x-vcard
text/vcard
text/vcard; charset=utf-8; name="fileName.vcf"
The result is always the same:
I'd like to have ideas on what to try next or if you already experienced the same issue before.
Troubleshooting problem
Turns out one of the configuration was correct, and the problem I was facing was a different one.
When Twilio receives a file for the first time, it caches
If you adjust the headers accordingly it won't impact the delivery of the VCF file, because it will use the cached version
There is a way to overcome that
Twilio Caching
To remove the caching on your files or to set the appropriate cache policy, read: https://support.twilio.com/hc/en-us/articles/360024716314-How-Can-I-Change-the-Cache-Behavior-or-Message-Media-Files-
Troubleshooting I figured out that Twilio don't change cache based only on filename for VCF files, but by the file content itself, so you have to modify your file in order for it to clear the cache, not just the name.
Correct configuration
In order to get the file properly parsed by an iOS phone, you can use the same file format as attached https://mighty-health-assets.s3.amazonaws.com/vcf/James+Li.vcf
Headers
The contact card on iOS can only display the same text as of your file, so name your file with the same filename header property on Content-Type and it should work accordingly
Content-Disposition: inline; filename="<You file name>.vcf"
Content-Type: text/x-vcard
Cache-Control: no-cache
I use HTTPie to POST a
multipart/form-data request (passing the -f option). It includes a
file field (using a # option). The corresponding part of the
multipart request has a pseudo-header Content-Disposition, but does
not have a pseudo-header Content-Type.
How is it possible to add such a Content-Type for a specific file?
For completeness, here is what I send now (as shown by -p B):
--xyz
Content-Disposition: form-data; name="file"; filename="file.txt"
Hello, world!
This is the content of the file.
--xyz
but here is what I need to send:
--xyz
Content-Disposition: form-data; name="file"; filename="file.txt"
Content-Type: text/plain
Hello, world!
This is the content of the file.
--xyz
Or to put it another way, like the CURL equivalent option: -F "file=file.txt;type=text/plain".
This worked for me:
http -f POST :8080/submit-file metadata=#metadata.json files#file1.jpeg files#file1.jpeg
This documentation helped me: https://httpie.org/doc#file-upload-forms
The =# makes httpie send it with content type text/plain.
This isn't possible with httpie. There is an issue for it, with some pull requests to fix it, but none have been merged yet.
I have just noticed that the documentation for multipart/form-data now reads like the following (at the end):
When uploading files, their content type is inferred from the file
name. You can manually override the inferred content type:
$ http -f POST httpbin.org/post name='John Smith' cv#'~/files/data.bin;type=application/pdf'
I do not know when that happened, but it seems that now:
the "part Content-Type" is inferred automatically
it is possible to override it from the command line option
So all I was looking for! :-)
Ref: https://httpie.org/docs#file-upload-forms.
I have the following code:
http = Net::HTTP.new("www.something.com", 80)
http.set_debug_output($stdout)
http.post("/blah", "something", {'random-parameter' => 'value1="something",value2="somethingelse"'})
Then when I read the output from stdout, it looks like this:
<- "POST /blah HTTP/1.1\r\nAccept: */*\r\nContent-Type: application/x-www-form-urlencoded\r\nConnection: close\r\nrandom-parameter: value1=\"something\",value2=\"somethingelse\"\r\nContent-Length: 9\r\nHost: www.something.com\r\n\r\n"
<- "something"
with the quotes being escaped. The problem is that the slashes seem to be sent to the server, which doesn't like that. I'm getting an error that says
Unknown value for random-parameter in header: {value1=\\\"something\\\"value2=\\\"somethingelse\\\"}
So my question is, is there a way to tell Net::HTTP to not insert those slashes, or strip them out before sending the header?
Clarifications:
I'm using Ruby 1.8.7 with Rails 2.0.2.
I think it may be Rails that is escaping the characters, but I'm not sure how to make it stop.
Are you sure you're building up the header correctly? Net::HTTP does not quote quotes when sending the request. You can easily verify it by using for example netcat (nc):
Terminal 1:
> nc -v -l -p 2323
Terminal 2 (in irb):
> http = Net::HTTP.new("localhost", 2323)
> http.post("/blah", "something", {'random-parameter' => ... )
Result (in terminal 1):
listening on [any] 2323 ...
connect to [127.0.0.1] from localhost [127.0.0.1] 37598
POST /blah HTTP/1.1
Connection: close
Accept: */*
Random-Parameter: value1="something",value2="somethingelse"
Content-Type: application/x-www-form-urlencoded
Content-Length: 9
Host: localhost:2323
something
I think what you actually may want to do (not sure, but I'm guessing) is something more along the lines of the HTTP spec:
> http.post("/blah", "something", {
'random-parameter' => 'value1="something"; value2="somethingelse"' })
Rails is probably interpreting your first value1=... as the whole value..when you probably need to separate the values with ';', not ','.
Also note that you don't usually pass parameters through request headers. But maybe that's what you want to do in this case(?) Otherwise you should either pass the parameters in the URL itself like param1=foo¶m2=bar, or use x-www-form-urlencoded to pass the parameters.
See here for a cheat sheet:
http://www.rubyinside.com/nethttp-cheat-sheet-2940.html
Isn't the problem that your name/value pairs are the value for "random-parameter", hence they need to be escaped.
I'd expect that when you inspect the random-parameter value in the controller method that it would not have the slashes - can you debug the code or print the parameter to see whats being received?
I am doing curl POST to a service with body containing some json data. I am getting the response as "request body was not mime-encoded as application/octet-stream". What does the response mean?
You probably need to supply a Content-Type header. Depending on what your web server is expecting, you might want to supply it either the mimetype "text/plain" or perhaps "application/x-www-form-urlencoded". In Curl, just include the argument:
-H "Content-Type: text/plain"
So your request will be something like:
curl -i -X POST --data-binary "#your.json" -H "Content-Type: text/plain"
Or substitute "text/plain" for the appropriate mime-type.
So, probably what's happening at the moment is that your web server is being given the content type application/octet-stream, and not understanding what to do with your plain text json content. If you run curl with "-v" it will give you a verbose description of the sent and received headers so you can see what Content-Type it's giving your server by default.
The input have to be a "application".
What you asking for is a mimetype.
Read more here: http://en.wikipedia.org/wiki/Internet_media_type
Please add more information for more informations.
Here's an excerpt from Live HTTP headers, I've replaced several values for anonymity.
POST blah/admin.php?module_id=1&action=update&id=129&pageNum=17&&eid=362 HTTP/1.1
Host: blah
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.12) Gecko/20101027 Firefox/3.6.12
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
Referer: blah
Cookie: blah
Content-Type: multipart/form-data; boundary=---------------------------21278813472729408841849703914
Content-Length: 5110
-----------------------------21278813472729408841849703914
Content-Disposition: form-data; name="MAX_FILE_SIZE"
300000000
This request has both GET and POST values. The script on the other end of this is PHP and expects certain values to be in the GET and others to be in the POST.
I know how to issue a GET
curl -G -d "key=val" "http://yadayadayada"
And I understand how to do a POST
curl -d "key=val" "http://yadayadayada"
curl -F "key=val" "http://yadayadayada"
But how do I mix the two in a single request? Every attempt I've made so far has ended in an error.
GET variables can be included in the URL. You just include the GET variables in the query string. For example, if you wanted to send a GET request with "username=fred" to www.example.com/index.php, you would send a simple GET request to "http://www.example.com/index.php?username=fred". So to answer your question, just use the POST method, but have the URL contain your GET data.
To clarify, GET and POST are HTTP request methods, not value types.
GET variables are called query string parameters. They are part of the URL, and can be included in any request.
POST variables are the contents of a urlencoded message body. These might also be sent with a PUT request.
Therefore, if you want to send both types of values, send the POST data as normal while explicitly writing your query string.
curl -d "key=val" "http://example.com?query_var=1"