Send files in request using JSONAPI - ruby-on-rails

I am using Rails 5 as the backend of a mobile app. The problem I am trying to solve is receiving a request from the app containing information about the customer plus 2 photos. After a short approach two options appeared:
Send the files first in a multipart/form-data POST, and return an ID to the client. After that a "real" request is sent again and the server should associate the ID(metadata) and the file.
Send the files is Base64.encoded format without changing the JSON header. Something like:
curl -X POST \
-H "Content-Type: application/vnd.api+json" \
-H "Cache-Control: no-cache" \
-d '{
"data": {
"type": "identities",
"attributes": {
"param1": "first param",
"param2": "second param",
"image1": "data:image/png;base64,iVBORw0KGgoAAAANSU.....",
"image2": "data:image/png;base64,iVBORw0KGgoAAAANSU....."
}
}' "http://API_URL/identity"
My concerns about these 2 approaches respectively are:
Since we are expecting 2 files should we make a request for each one associating it with ID? What is expected to happen if the second call doesn't reach the server or is not valid?
What amount of bytes should we accept? I was thinking of 10MB but I am not sure if this is a good idea and how the server will react? Is it a good idea firstly to validate the type and size of the file on a UI level(the mobile app)?
If someone can suggest something else I would really appreciate it. Also if you have any experience with this problem please share useful references that you have used, I would appreciate them as well.

1) If you need 2 files at one request - pass 2 files as multipart/form-data, that is totally fine. If you use b64, you encode everything first and then decode everything. Not the best idea.
2) You should validate those files both at front and back end. Max amount of bytes should be like max_file1_size + max_file_2_size + max_other_fields_size + headers_size, it's less about guessing then trying.
3) It will be a good option to use carrierwave - nice gem, and you'll got a lot less space to mess :)

Related

Http status code for PUT when path parameter ID is different than request body ID

I am using put to modify a resource. I was wondering what is the appropriate status code to return when the id given in the path is different from the id given to the body of the resource.
e.g.
REST api action method is: /resources/{id}
Body:
{
"id": "test",
"name": "my_resource"
}
What HTTP Status Code should be returned to the following curl request?
curl -X PUT 'localhost:8080/resources/test2' \
--header 'Content-Type: application/json' \
--data-raw '{
"id": "test",
"name": "my_resource"
}'
You can probably make an argument for any of
403 Forbidden (I understand your request, and I decline to fulfill it - see response body for details)
409 Conflict (Your proposed change conflicts with the current state of the resource - see response body for details)
422 Unprocessable Content (The body of your request is internally consistent, but doesn't make sense here - see response body for details).
For the general purpose elements of the HTTP application itself, it doesn't very much matter which of these you choose - they are all non-cacheable errors, the component isn't going to have any sort of automatic recovery, etc.
One tie breaker you might consider is how these entries are going to appear in your access logs / how your automated monitoring is going to handle them: if you want these messages to stand out in your logs, then you'll want to choose status code that's not overloaded with other meanings.
For more details, see section 15.5 of HTTP Semantics
Do you think that a 500 Internal Server Error with a proper explanation would be fine as well or is it confusing?
5xx Server Error is inappropriate when the information in the request is the source of the problem, because the form of the request is the client's responsibility, not the server's.
Loosely: 4xx covers all of the cases where the server is announcing "you should not have asked for that". 5xx is for variations of "what you asked for is fine, but I couldn't do it."
See the HTTP Semantics spec, in particular sections 15.5 and 15.6 (or, if you prefer the earlier reference, RFC 7231 6.5 and 6.6)

How to call Twitter's POST /statuses/filter with 5000 user ids?

Both GET and POST methods supported by the endpoint. The POST method is recommended to call endpoint with a huge number of user ids to follow, because the GET method will lead to an oversized URL that the server can't handle. How the "follow" parameter can be passed in the body of the request?
UPD: here is what I've already tried using Insomnia (the URL is always 'https://stream.twitter.com/1.1/statuses/filter.json' and the method is always 'POST' and the server response is always "No filter parameters found. Expect at least one parameter: follow track locations"):
A plain text body with Content-Type: text/html
follow=2731236345
A json body with Content-Type: application/json
{
"follow": "2731236345"
}
Another json body
{
"follow": [
2731236345
]
}
However, when I use form-url-encoded with field "follow" and the value "2731236345" I receive the response "Unauthorized".
First of all, consider looking at the Twitter Developer Labs new endpoint, because this existing API will be retired, likely (but not yet confirmed) in 2020.
When you say "without any success", what libraries are you using, and at what levels of query parameters - you're not being very clear about what is not working here. 5000 user IDs is very large. Can you please be more specific about the errors you're seeing, and the code you're trying to run?
I've managed to connect using curl:
curl --request POST \
--url 'https://stream.twitter.com/1.1/statuses/filter.json' \
--header 'authorization: <censored>' \
--data 'follow=2731236345'
The same request doesn't work in Insomnia for some reason, but it doesn't matter for the goal of this post.

Is this Calendly.com URL Valid?

I am using webhooks for the first time with calendly.com. I would like to create a subscription as described here. When I get to doing this
curl --header "X-TOKEN: <your_token>" --data "url=https://blah.foo/bar&events[]=invitee.created" https://calendly.com/api/v1/hooks
I am concerned that the url is not in a valid format. I tried replacing the & with a ? as in here
curl --header "X-TOKEN: <your_token>" --data "url=https://blah.foo/bar?events[]=invitee.created" https://calendly.com/api/v1/hooks
but I receive this error
{"events":{"events":["can't be blank]}}
Likewise, I try to leave the & and everything after it blank and it give the same error as above.
But using a url with the & and everything in it gives a 404 not found error. Can a url even have the format they are saying is required?
But using a url with the & and everything in it gives a 404 not found error.
The URL part in the following is just the https://blah.foo/bar part before the & character.
curl --header "X-TOKEN: <your_token>" --data "url=https://blah.foo/bar&events[]=invitee.created" https://calendly.com/api/v1/hooks
The events[]=invitee.created part is a separate parameter and value. It’s not part of the url.
See the Create A Webhook Subscription page in the calendy docs:
So the URL is the site that runs your webhook; you don’t send the events parameter/value back to that site—instead you’re sending both the url param and events param to the calendly API endpoint at https://calendly.com/api/v1/hooks. The calendly backend presumably then calls to https://mywebsite.com/webhooks/invitee_created or https://blah.foo/bar whatever url you’ve given it—but without the events parameter, which is for calendly’s internal use.

Amazon Web Service Signature issue

I am trying to fetch product related data from Amazon Product API using this URL mentioned here:
http://docs.aws.amazon.com/AWSECommerceService/latest/DG/ItemLookup.html
Item Lookup URL:
http://webservices.amazon.com/onca/xml?Service=AWSECommerceService&AWSAccessKeyId=[AWS_Access_Key_ID]&Operation=ItemLookup&ItemId=B00008OE6I&Timestamp=[YYYY-MM-DDThh:mm:ssZ]&Signature=[Request_Signature]
I have entered required details/parameters in URL but I am unable to understand where to get this Signature from or how to generate this signature? Any help would be appreciated.
The easiest and fastest way would be using one of AWS SDKs. You can find the full list here: https://aws.amazon.com/tools/
When you use the SDK you just supply your access key and secret key. The libraries handle preparing the signature.
Well, there's no point in doing this by yourself in Ruby, once it has a SDK, but an explanation about signature process may be useful for some people.
Signature is the way AWS authenticates API requests. It is passed as a query string parameter on your HTTP request and is validated against your own request content and you credentials on server-side.
There are 2 versions of it: 2 and 4, explained here and here. Version 2 is not supported on newer regions (China and Frankfurt).
In version 2, which I'm more familiar with, the signature is generated based on a string with this format:
HTTPVerb + "\n" + ValueOfHostHeaderInLowercase + "\n" + HTTPRequestURI
+ "\n" + CanonicalizedQueryString
The CanonicalizedQueryString is your query string, with parameters sorted (UTF-8 byte order) by paramater names and with parameter values url-encoded.
Then you calculate an HMAC of that string, using your AWS secret key, and SHA256 or SHA1 as the hash algorithm (depending on which one you have specified on query string parameter SignatureMethod), and encode it to base64. The result of all this will be the value of the Signature parameter.
I will try to walk you through a CloudWatch request, generated manually with bash tools.
Let's say you want to list you CloudWatch metrics. The request (without the signature) would be like the following.
http://monitoring.amazonaws.com/?AWSAccessKeyId=AVUKYAMNIIGWALNMA6IG&Action=ListMetrics&SignatureMethod=HmacSHA1&SignatureVersion=2&Timestamp=2015-08-20T22%3A40%3A00.000Z&Version=2010-08-01
So, your string to sign would be:
GET
monitoring.amazonaws.com
/
AWSAccessKeyId=AVUKYAMNIIGWALNMA6IG&Action=ListMetrics&SignatureMethod=HmacSHA1&SignatureVersion=2&Timestamp=2015-08-20T22%3A40%3A00.000Z&Version=2010-08-01
Well, let's put it in a variable and do all the magic (hmac + base64)
#!/bin/bash
# your secret key
key='ry+SyjvXLvi4lYie/QfZYnxYOBQDggf01DTUxFSK'
# using 'read' to assign the signing string
# to a variable for readability reasons
read -r -d '' payload <<EOF
GET
monitoring.amazonaws.com
/
AWSAccessKeyId=AVUKYAMNIIGWALNMA6IG\
&Action=ListMetrics\
&SignatureMethod=HmacSHA1\
&SignatureVersion=2&\
Timestamp=2015-08-20T22%3A40%3A00.000Z\
&Version=2010-08-01
EOF
echo -n "$payload" | openssl dgst -sha256 -hmac $key -binary | openssl enc -base64
Then, after all this, you will end up with something like this:
xrIYF8wO0C8YOQgcdpq6MoJpBWzXEWxuTBAAoqYoGWM=
This is your signature, and finally, you just need to url-encoded it and append it to your request as one more query-string parameter.
&Signature=xrIYF8wO0C8YOQgcdpq6MoJpBWzXEWxuTBAAoqYoGWM%3D
Hopefully, if you did everything correctly (putting the sign string together, generating the hash and encoding it), your request will be accepted and, if you have enough permission, executed.
So, as you can see, it's something not worth to do by yourself. Use the SDK. ;)
Thank You guys for your response. But I have finally figured out my problem and my solution was in this gem:
https://github.com/jugend/amazon-ecs
This is what I was actually looking for and it finally helped me achieve what I was trying to. Posting it over here so people can find a better solution.

Gzip decompress JSON POST body in Rails/Passenger/Nginx

We have a function in our Rails code that accepts a JSON POST body:
contacts = ActiveSupport::JSON.decode(request.raw_post.gsub("+", ""))
(I'm aware that I can get this from params["_json"] as well, but we have extremely large (MBs) POST bodies that do not get put into params["_json"] for some reason (and + throws errors too).
Since the JSON is usually sent from a mobile client, it's important to us to optimize the upload size. We want to switch to having the POST body gzipped.
However, no matter what we do, we get the same error with no line number:
MultiJson::DecodeError (743: unexpected token at ''):
We have tried:
gzipped_contacts = Zlib::GzipReader.new(StringIO.new(request.raw_post)).read
contacts = ActiveSupport::JSON.decode(gzipped_contacts.gsub("+", ""))
This:
gzipped_contacts = ActiveSupport::Gzip.decompress(request.raw_post)
contacts = ActiveSupport::JSON.decode(gzipped_contacts.gsub("+", ""))
And the solution found here: Rails: how to unzip a compressed xml request body?
I'm pretty sure this is not occurring at the controller level because I can't log anything there, so it needs to be done in the middleware or at the server (but I can't find anything for Nginx that lets us deflate). Please assist!
Ok, turns out the iPhone client was sending the wrong headers. So the solution for anyone encountering this is to see the advice here:
Rails: how to unzip a compressed xml request body?
And verify that you are sending Content-Type: gzip/json.

Resources