I have an API I want to invoke that wants something like this to be POSTed in the body: ["foo","bar"]
I have read the official docs and can't see how to do this, is it possible?
I have tried: http --verbose PUT foo.org :='["foo","bar"]' but httpie still wants to turn that into a field of a json object, with no name, as in:
{
"": [
"foo",
"bar"
]
}
The shorthand syntax httpie offers doesn't support empty strings as keys. The easiest option here will be to pass the data via redirected stdin:
$ echo '{"": ["foo","bar"]}' | http --verbose PUT foo.org
See also: Sending nested JSON object using HTTPie
Related
Both GET and POST methods supported by the endpoint. The POST method is recommended to call endpoint with a huge number of user ids to follow, because the GET method will lead to an oversized URL that the server can't handle. How the "follow" parameter can be passed in the body of the request?
UPD: here is what I've already tried using Insomnia (the URL is always 'https://stream.twitter.com/1.1/statuses/filter.json' and the method is always 'POST' and the server response is always "No filter parameters found. Expect at least one parameter: follow track locations"):
A plain text body with Content-Type: text/html
follow=2731236345
A json body with Content-Type: application/json
{
"follow": "2731236345"
}
Another json body
{
"follow": [
2731236345
]
}
However, when I use form-url-encoded with field "follow" and the value "2731236345" I receive the response "Unauthorized".
First of all, consider looking at the Twitter Developer Labs new endpoint, because this existing API will be retired, likely (but not yet confirmed) in 2020.
When you say "without any success", what libraries are you using, and at what levels of query parameters - you're not being very clear about what is not working here. 5000 user IDs is very large. Can you please be more specific about the errors you're seeing, and the code you're trying to run?
I've managed to connect using curl:
curl --request POST \
--url 'https://stream.twitter.com/1.1/statuses/filter.json' \
--header 'authorization: <censored>' \
--data 'follow=2731236345'
The same request doesn't work in Insomnia for some reason, but it doesn't matter for the goal of this post.
I am trying to fetch product related data from Amazon Product API using this URL mentioned here:
http://docs.aws.amazon.com/AWSECommerceService/latest/DG/ItemLookup.html
Item Lookup URL:
http://webservices.amazon.com/onca/xml?Service=AWSECommerceService&AWSAccessKeyId=[AWS_Access_Key_ID]&Operation=ItemLookup&ItemId=B00008OE6I&Timestamp=[YYYY-MM-DDThh:mm:ssZ]&Signature=[Request_Signature]
I have entered required details/parameters in URL but I am unable to understand where to get this Signature from or how to generate this signature? Any help would be appreciated.
The easiest and fastest way would be using one of AWS SDKs. You can find the full list here: https://aws.amazon.com/tools/
When you use the SDK you just supply your access key and secret key. The libraries handle preparing the signature.
Well, there's no point in doing this by yourself in Ruby, once it has a SDK, but an explanation about signature process may be useful for some people.
Signature is the way AWS authenticates API requests. It is passed as a query string parameter on your HTTP request and is validated against your own request content and you credentials on server-side.
There are 2 versions of it: 2 and 4, explained here and here. Version 2 is not supported on newer regions (China and Frankfurt).
In version 2, which I'm more familiar with, the signature is generated based on a string with this format:
HTTPVerb + "\n" + ValueOfHostHeaderInLowercase + "\n" + HTTPRequestURI
+ "\n" + CanonicalizedQueryString
The CanonicalizedQueryString is your query string, with parameters sorted (UTF-8 byte order) by paramater names and with parameter values url-encoded.
Then you calculate an HMAC of that string, using your AWS secret key, and SHA256 or SHA1 as the hash algorithm (depending on which one you have specified on query string parameter SignatureMethod), and encode it to base64. The result of all this will be the value of the Signature parameter.
I will try to walk you through a CloudWatch request, generated manually with bash tools.
Let's say you want to list you CloudWatch metrics. The request (without the signature) would be like the following.
http://monitoring.amazonaws.com/?AWSAccessKeyId=AVUKYAMNIIGWALNMA6IG&Action=ListMetrics&SignatureMethod=HmacSHA1&SignatureVersion=2&Timestamp=2015-08-20T22%3A40%3A00.000Z&Version=2010-08-01
So, your string to sign would be:
GET
monitoring.amazonaws.com
/
AWSAccessKeyId=AVUKYAMNIIGWALNMA6IG&Action=ListMetrics&SignatureMethod=HmacSHA1&SignatureVersion=2&Timestamp=2015-08-20T22%3A40%3A00.000Z&Version=2010-08-01
Well, let's put it in a variable and do all the magic (hmac + base64)
#!/bin/bash
# your secret key
key='ry+SyjvXLvi4lYie/QfZYnxYOBQDggf01DTUxFSK'
# using 'read' to assign the signing string
# to a variable for readability reasons
read -r -d '' payload <<EOF
GET
monitoring.amazonaws.com
/
AWSAccessKeyId=AVUKYAMNIIGWALNMA6IG\
&Action=ListMetrics\
&SignatureMethod=HmacSHA1\
&SignatureVersion=2&\
Timestamp=2015-08-20T22%3A40%3A00.000Z\
&Version=2010-08-01
EOF
echo -n "$payload" | openssl dgst -sha256 -hmac $key -binary | openssl enc -base64
Then, after all this, you will end up with something like this:
xrIYF8wO0C8YOQgcdpq6MoJpBWzXEWxuTBAAoqYoGWM=
This is your signature, and finally, you just need to url-encoded it and append it to your request as one more query-string parameter.
&Signature=xrIYF8wO0C8YOQgcdpq6MoJpBWzXEWxuTBAAoqYoGWM%3D
Hopefully, if you did everything correctly (putting the sign string together, generating the hash and encoding it), your request will be accepted and, if you have enough permission, executed.
So, as you can see, it's something not worth to do by yourself. Use the SDK. ;)
Thank You guys for your response. But I have finally figured out my problem and my solution was in this gem:
https://github.com/jugend/amazon-ecs
This is what I was actually looking for and it finally helped me achieve what I was trying to. Posting it over here so people can find a better solution.
I am using an API provided by my web host with whom I can register new clients and domains. Unfortunately they don't provide much documentation (basically none) and I'm not very used to cURL
They give ONE and very superficial example of how to create a new client, and it is as follows
curl -d "clienteTipo=I&clienteCPFCNPJ=00112135045&clienteEmpresa=NomeEmpresa&clienteNome=meunome&clienteEmail=email#dominio.com.br&clienteEmailCobranca=emailcobranca#dominio.com.br&clienteSenhaPainel=654321&clienteFone=555100000000&clienteFax=555100000001&clienteCEP=44587421&clienteEndereco=ruanome&clienteBairro=meubairro&clienteCidade=porto alegre&clienteEstado=rs&clienteLimiteMapeamento=1&clienteLimiteSubdominio=2&clienteLimiteMysql=3&clienteLimiteMssql=1&clienteLimitePgsql=1&clienteLimiteFirebird=1&clienteLimiteFTPADD=1&clienteUniBox=on&clienteAcessoFTP=on&clienteAcessoDownloadBackup=on" -k --digest -u usuario:senha -X POST https://api.kinghost.net/cliente
How can I transform this on a cURL request on Ruby keeping the parameters as informed above?
This must be a POST, but I wonder how I have to inform these parameters, they look like a query string. And what is this Digest thing?
Here is an (untested) example using rest-client that might get you started https://github.com/rest-client/rest-client).
According to what I was able to find online, rest-client automatically supports the digest authentication that that curl request is doing.
require 'rest-client'
response = RestClient::Request.execute(
method: :post,
url: 'https://api.kinghost.net/cliente',
verify_ssl: false,
user: "usuario",
password: "senha",
headers: {
params: {
clienteTipo: "I",
clienteCPFCNPJ: "00112135045",
clienteEmpresa: "NomeEmpresa",
clienteNome: "meunome",
clienteEmail: "email#dominio.com.br",
clienteEmailCobranca: "emailcobranca#dominio.com.br",
clienteSenhaPainel: "654321",
clienteFone: "555100000000",
clienteFax: "555100000001",
clienteCEP: "44587421",
clienteEndereco: "ruanome",
clienteBairro: "meubairro",
clienteCidade: "porto alegre",
clienteEstado: "rs",
clienteLimiteMapeamento: "1",
clienteLimiteSubdominio: "2",
clienteLimiteMysql: "3",
clienteLimiteMssql: "1",
clienteLimitePgsql: "1",
clienteLimiteFirebird: "1",
clienteLimiteFTPADD: "1",
clienteUniBox: "on",
clienteAcessoFTP: "on",
clienteAcessoDownloadBackup: "on"
}
}
)
the wonders of curl :)
-k means unsecure (i.e don't check SSL)
--digest is the authentication type
-u name:password is the actual username and password
-X POST it's a post
-d "blah blah" blah blah is the payload
now onto ruby:
I would look at https://github.com/rest-client/rest-client or at https://github.com/jnunemaker/httparty
It's just a matter of figuring out how to perform the post. The digest might be tricky but IIRC it can be some via Authorization header.
after struggling for a while and testing different libraries, I used Ruby to run a system command, since cURL is not dependant on Ruby
output = `curl -d "clienteTipo=#{client_type}&clienteCPFCNPJ=#{client_id}&clienteEmpresa=#{company_name}&clienteNome=#{client_name}&clienteEmail=#{client_email}&clienteEmailCobranca=#{client_billing_email}&clienteSenhaPainel=#{password}&clienteFone=#{client_phone}&clienteFax=#{client_fax}&clienteCEP=#{client_zip_code}&clienteEndereco=#{client_address}&clienteBairro=#{client_neighborhood}&clienteCidade=#{client_city}&clienteEstado=#{client_state}&clienteLimiteMapeamento=1&clienteLimiteSubdominio=1&clienteLimiteMysql=1&clienteLimiteMssql=1&clienteLimitePgsql=1&clienteLimiteFirebird=1&clienteLimiteFTPADD=1&clienteUniBox=on&clienteAcessoFTP=on&clienteAcessoDownloadBackup=on" -k --digest -u MY_USER:MY_PASSWORD -X POST API_URL`
p output
xml = Hash.from_xml(output) #in my case it returns a XML string
Now output contains the response
I am using Ruby on Rails 4.1 and I am trying to implement an API with a custom mime type. That is, in config/initializers/mime_types.rb I register an alias as like the following:
Mime::Type.register_alias 'application/json', :my_json
From another system I am trying to access the API with curl by running a HTTP PUT request, this way:
curl http://www.my_api.org/articles.my_json --request PUT --header "Content-Type: application/json" --data-binary '{\"key\": {\"a\": \"1\", \"b\": \"2\"}}'
However, by inspecting the ArticlesController parameters in my Rails application, I get the following output (note: article parameters are "unwanted" and those duplicate the "wanted" key parameters):
Parameters: {"key": { "a"=>"1", "b"=>"2" }, "article": { "a"=>"1", "b"=>"2" }}
What is the problem? Is it a bug? How can I solve that?
Note: I have implemented and access other similar API by executing HTTP GET requests and all works as expected. The problem seems to happens only when I execute HTTP PUT requests.
#rafaelfranca - No it is not a bug. It is how wrap_parameters works. You can disable at this file in your application config/initializers/wrap_parameters.rb.
See github.
I have the following code:
http = Net::HTTP.new("www.something.com", 80)
http.set_debug_output($stdout)
http.post("/blah", "something", {'random-parameter' => 'value1="something",value2="somethingelse"'})
Then when I read the output from stdout, it looks like this:
<- "POST /blah HTTP/1.1\r\nAccept: */*\r\nContent-Type: application/x-www-form-urlencoded\r\nConnection: close\r\nrandom-parameter: value1=\"something\",value2=\"somethingelse\"\r\nContent-Length: 9\r\nHost: www.something.com\r\n\r\n"
<- "something"
with the quotes being escaped. The problem is that the slashes seem to be sent to the server, which doesn't like that. I'm getting an error that says
Unknown value for random-parameter in header: {value1=\\\"something\\\"value2=\\\"somethingelse\\\"}
So my question is, is there a way to tell Net::HTTP to not insert those slashes, or strip them out before sending the header?
Clarifications:
I'm using Ruby 1.8.7 with Rails 2.0.2.
I think it may be Rails that is escaping the characters, but I'm not sure how to make it stop.
Are you sure you're building up the header correctly? Net::HTTP does not quote quotes when sending the request. You can easily verify it by using for example netcat (nc):
Terminal 1:
> nc -v -l -p 2323
Terminal 2 (in irb):
> http = Net::HTTP.new("localhost", 2323)
> http.post("/blah", "something", {'random-parameter' => ... )
Result (in terminal 1):
listening on [any] 2323 ...
connect to [127.0.0.1] from localhost [127.0.0.1] 37598
POST /blah HTTP/1.1
Connection: close
Accept: */*
Random-Parameter: value1="something",value2="somethingelse"
Content-Type: application/x-www-form-urlencoded
Content-Length: 9
Host: localhost:2323
something
I think what you actually may want to do (not sure, but I'm guessing) is something more along the lines of the HTTP spec:
> http.post("/blah", "something", {
'random-parameter' => 'value1="something"; value2="somethingelse"' })
Rails is probably interpreting your first value1=... as the whole value..when you probably need to separate the values with ';', not ','.
Also note that you don't usually pass parameters through request headers. But maybe that's what you want to do in this case(?) Otherwise you should either pass the parameters in the URL itself like param1=foo¶m2=bar, or use x-www-form-urlencoded to pass the parameters.
See here for a cheat sheet:
http://www.rubyinside.com/nethttp-cheat-sheet-2940.html
Isn't the problem that your name/value pairs are the value for "random-parameter", hence they need to be escaped.
I'd expect that when you inspect the random-parameter value in the controller method that it would not have the slashes - can you debug the code or print the parameter to see whats being received?