I am doing curl POST to a service with body containing some json data. I am getting the response as "request body was not mime-encoded as application/octet-stream". What does the response mean?
You probably need to supply a Content-Type header. Depending on what your web server is expecting, you might want to supply it either the mimetype "text/plain" or perhaps "application/x-www-form-urlencoded". In Curl, just include the argument:
-H "Content-Type: text/plain"
So your request will be something like:
curl -i -X POST --data-binary "#your.json" -H "Content-Type: text/plain"
Or substitute "text/plain" for the appropriate mime-type.
So, probably what's happening at the moment is that your web server is being given the content type application/octet-stream, and not understanding what to do with your plain text json content. If you run curl with "-v" it will give you a verbose description of the sent and received headers so you can see what Content-Type it's giving your server by default.
The input have to be a "application".
What you asking for is a mimetype.
Read more here: http://en.wikipedia.org/wiki/Internet_media_type
Please add more information for more informations.
Related
Both GET and POST methods supported by the endpoint. The POST method is recommended to call endpoint with a huge number of user ids to follow, because the GET method will lead to an oversized URL that the server can't handle. How the "follow" parameter can be passed in the body of the request?
UPD: here is what I've already tried using Insomnia (the URL is always 'https://stream.twitter.com/1.1/statuses/filter.json' and the method is always 'POST' and the server response is always "No filter parameters found. Expect at least one parameter: follow track locations"):
A plain text body with Content-Type: text/html
follow=2731236345
A json body with Content-Type: application/json
{
"follow": "2731236345"
}
Another json body
{
"follow": [
2731236345
]
}
However, when I use form-url-encoded with field "follow" and the value "2731236345" I receive the response "Unauthorized".
First of all, consider looking at the Twitter Developer Labs new endpoint, because this existing API will be retired, likely (but not yet confirmed) in 2020.
When you say "without any success", what libraries are you using, and at what levels of query parameters - you're not being very clear about what is not working here. 5000 user IDs is very large. Can you please be more specific about the errors you're seeing, and the code you're trying to run?
I've managed to connect using curl:
curl --request POST \
--url 'https://stream.twitter.com/1.1/statuses/filter.json' \
--header 'authorization: <censored>' \
--data 'follow=2731236345'
The same request doesn't work in Insomnia for some reason, but it doesn't matter for the goal of this post.
I use HTTPie to POST a
multipart/form-data request (passing the -f option). It includes a
file field (using a # option). The corresponding part of the
multipart request has a pseudo-header Content-Disposition, but does
not have a pseudo-header Content-Type.
How is it possible to add such a Content-Type for a specific file?
For completeness, here is what I send now (as shown by -p B):
--xyz
Content-Disposition: form-data; name="file"; filename="file.txt"
Hello, world!
This is the content of the file.
--xyz
but here is what I need to send:
--xyz
Content-Disposition: form-data; name="file"; filename="file.txt"
Content-Type: text/plain
Hello, world!
This is the content of the file.
--xyz
Or to put it another way, like the CURL equivalent option: -F "file=file.txt;type=text/plain".
This worked for me:
http -f POST :8080/submit-file metadata=#metadata.json files#file1.jpeg files#file1.jpeg
This documentation helped me: https://httpie.org/doc#file-upload-forms
The =# makes httpie send it with content type text/plain.
This isn't possible with httpie. There is an issue for it, with some pull requests to fix it, but none have been merged yet.
I have just noticed that the documentation for multipart/form-data now reads like the following (at the end):
When uploading files, their content type is inferred from the file
name. You can manually override the inferred content type:
$ http -f POST httpbin.org/post name='John Smith' cv#'~/files/data.bin;type=application/pdf'
I do not know when that happened, but it seems that now:
the "part Content-Type" is inferred automatically
it is possible to override it from the command line option
So all I was looking for! :-)
Ref: https://httpie.org/docs#file-upload-forms.
I am using Ruby on Rails 4.1 and I am trying to implement an API with a custom mime type. That is, in config/initializers/mime_types.rb I register an alias as like the following:
Mime::Type.register_alias 'application/json', :my_json
From another system I am trying to access the API with curl by running a HTTP PUT request, this way:
curl http://www.my_api.org/articles.my_json --request PUT --header "Content-Type: application/json" --data-binary '{\"key\": {\"a\": \"1\", \"b\": \"2\"}}'
However, by inspecting the ArticlesController parameters in my Rails application, I get the following output (note: article parameters are "unwanted" and those duplicate the "wanted" key parameters):
Parameters: {"key": { "a"=>"1", "b"=>"2" }, "article": { "a"=>"1", "b"=>"2" }}
What is the problem? Is it a bug? How can I solve that?
Note: I have implemented and access other similar API by executing HTTP GET requests and all works as expected. The problem seems to happens only when I execute HTTP PUT requests.
#rafaelfranca - No it is not a bug. It is how wrap_parameters works. You can disable at this file in your application config/initializers/wrap_parameters.rb.
See github.
I have a simple question regarding to the usage of cURL. Didn't find much during my Google search or Man page to get a clear answer.
In here talks about using either --data vs --form on sending file/attachment. I'm curious to know what are the main difference and under what scenarios you would pick --data-binary VS --form ?
The POST "body" can be sent via either --data (for application/x-www-form-urlencoded) or --form (for multipart/form-data):
-F "foo=bar" # 'foo' value is 'bar'
-F "foo=<foovalue.txt" # the specified file is sent as plain text input
-F "foo=#foovalue.txt" # the specified file is sent as an attachment
-d "foo=bar"
-d "foo=<foovalue.txt"
-d "foo=#foovalue.txt"
-d "#entirebody.txt" # the specified file is used as the POST body
--data-binary "#binarybody.jpg"
The difference is explained in the HTML 4.01 Specification section on Forms:
application/x-www-form-urlencoded is the default content type.
The content type "application/x-www-form-urlencoded" is inefficient for sending large quantities of binary data or text containing non-ASCII characters. The content type "multipart/form-data" should be used for submitting forms that contain files, non-ASCII data, and binary data.
That's exactly the main difference, type of data that's being sent to the server (application/x-www-form-urlencoded vs multipart/form-data)
I'm been playing with calling my rails controller using an HTTP POST. I can get it to work with a curl command such as this, given a model named item and an attribute in that item called name:
curl -X POST -d "<item><name>myname</name></item>" -H "Content-Type: text/xml" http://localhost:3000/items.xml
What I'm curious about is how to make the same call using text instead of xml as my content type.... I tried:
curl -X POST -d "name=myname" http://localhost:3000/items.xml
but that seems to pass the 'name' parameter as the top level scope; so it doesn't end up in my params in the controller....
I'm a noob at this; just want to understand how to do it both ways....
Thanks!
Short answer, you need to
(a) send it as -H "Content-type: application/x-www-form-urlencoded".
(b) specify the parameters as Object[field]=value - for example User[name] would refer to the name field in some user object.
(c) [Not Required because you use curl] encode the parameters and POST/PUT them.
Encoding the parameters
The encoding is simple enough and although curl will do it for you, it's useful to understand it. I've quoted an extract from the W3C spec...
"
application/x-www-form-urlencoded
This is the default content type. Forms submitted with this content type must be encoded as follows:
Control names and values are escaped. Space characters are replaced by +', and then reserved characters are escaped as described in [RFC1738], section 2.2: Non-alphanumeric characters are replaced by%HH', a percent sign and two hexadecimal digits representing the ASCII code of the character. Line breaks are represented as "CR LF" pairs (i.e., `%0D%0A').
The control names/values are listed in the order they appear in the document. The name is separated from the value by =' and name/value pairs are separated from each other by&'.
"
Simple Example (New User Form)
The example below is how to send a simple "New User" form.
If I have fields in the user object for name, password, email etc, I specify them like this ...
user[firstname]=chris&user[login]=cmccauley&user[company_id]=8&user[email]=&user[surname]=mccauley
then curl will escape them to give ...
user%5Bfirstname%5D=chris&user%5Blogin%5D=cmccauley&user%5Bcompany_id%5D=8&user%5Bemail%5D=&user%5Bsurname%5D=mccauley
... before posting them like this extract from a wire dump ...
Wireshark dump
POST /users/19 HTTP/1.1
Host: localhost:3000
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.10) Gecko/2009042523 Ubuntu/9.04 (jaunty) Firefox/3.0.10
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: http://localhost:3000/users/19/edit
Cookie: _my_session=048d330143de668e027c8cd52654e8c5
Content-Type: application/x-www-form-urlencoded
Content-Length: 259
user%5Bfirstname%5D=chris&user%5Blogin%5D=cmccauley&user%5Bcompany_id%5D=8&user%5Bemail%5D=&user%5Bsurname%5D=mccauley&user_password=********&user%5Bjob_id%5D=14&user%5Bpassword%5D=dd793a64b74e108fcdc5d809040e24afcc21ad2c&authenticity_token=&id=19&_method=PUT
"Content-type: application/x-www-form-urlencoded" is already default with curl -d, no need for anything extra
-X POST is superfluous, as -d implies POST
-d does not URL encode the data. To get curl to do that for you, you need to use --data-urlencode instead