I'm starting an iOS app that consume a Restful API.
I have control over that API and I'm confusing with the caching policies.
To begin, I only need caching a concrete resoruce, but the problem is that resource can change when I insert new record in the database.
Then, how can I tell to the application "Hey! Make the request only if there have been changes and if not, you get the data from the cache!"
I'm using AFNetworking to make requests..
You'll have to make a decision on either server or client side and build your own protocol.
Example:
You could send the server JSON post request which contains the 'version' of the data you have in the app. On the server-side you will increment the version number each time the data gets refreshed. If the version number does not match at server-side, the server will respond with all new data, else the server responds JSON with 'up to date'
EDIT:
If you are looking for an HTTP response saying that the data is not modified. This is done on server side. You'll have to implement this in the server.
Related
I have a Rails API which can handle requests from the clients. Clients use that API to perform analysis of their data. Client POSTs the data to API, API checks if that data have been analysed before. If so API just respond with analysis result. If the data haven't been analyzed before API:
Tells client that analysis started.
Establishes the connection with analyzing microservice.
Performs asynchronous (or deferred or i don't know) request to the analyzing microservice and waiting for response. The analysis takes much time so neither the API nor the microservice should be blocked while doing it.
When the response from analyzing microservice is returned API hands it to the client.
The main issue for me is to set up things such way that client could receive somehow the message "Your data had been sent to analysis" right after he performed the request. And then when analysis will be done client could receive its result.
The question is what approach I have to use in that case? Async responses, deferred responses, something else? And what known solutions could help me with that? Any gems?
I'm new to that stuff so I'm really sorry if I ask dumb questions.
If using HTTP you can only have one response to every request. To send multiple responses, i.e. "work in progress", then later the "results", you would need to use a different protocol, e.g. web sockets.
Since HTTP is so very common I'd stick with that in combination with background jobs. There are a couple of options which spring to mind.
Polling: The API kicks off a background jobs (to call the microservice) and responds to the client with a URL which the client can ping periodically for the result. The URL would respond with some kind of "work in progress" status until the result is actually ready). The URL would need to include some kind of id so the API can lookup the background job.
The API would potentially have two URLS; /api/jobs/new and /api/jobs/<ID>. They would, in Rails, map to a controller new and show action.
Webhooks: Have the client include a URL of its own in the request. Once the result is available have the background job hit the given URL with the result.
Either way, if using HTTP, you will not be able to handle the whole thing within a request/response, you will have to use some kind of background processing (so request to the microservice happens in a different process). You could look at Sidekiq, for example.
Here is an example for polling:
URL: example.com/api/jobs/new
web app receives client request
generates a unique id for the request, SecureRandom.uuid.
starts a background job (Sidekiq) passing in the uuid and any other parameters needed
respond with URL such as example.com/api/jobs/
--
background job
sends request to microservice API and waits for response
saves result to database with uuid
--
URL: example.com/api/jobs/UUID
look in database for UUID, if not found respond that job is "in progress". If found return result found in database.
Depending on what kind of API you use. I assume your clients interact via HTTP.
If you want to build an asynchronous API over HTTP the first thing that you should do: accept the request, create a job, handle it in the background and immediately return.
For the client to get the response you have to 2 options:
Implement a status endpoint where clients can periodically poll the status of the job
Implement a callback via webhooks. So the client has to provide a URL which you then call after you're done.
A good start for background processing is the sidekiq gem or more general ActiveJob that ships with Rails.
I know it is a basic Question. Does a POST Json Request for update in User interface actually Post data in to database or just simulates the load for Post data without actually posting in database. But can someone please clarify
An HTTP Post request with the mandatory body (can be parameters, JSON, XML etc.), is intended to upload the data into server (upload images, Sign Up etc.), or to post data which server wants (to validate Sign in etc., not necessarily insert into the DB). So, it is basically how Server treats the data and the purpose.
If server puts the received data into the database, when performed the action using the browser, then the same operation expected when performed through JMeter also irrespective of the type of body data (JSON, XML etc). So, If you post the JSON data, it must be inserted into DB.
In DB, One thing to note is that server, either can directly dump the JSON data as JSON type itself or parse the JSON data and take the required values and store them in the Database. It depends on how the server is implemented.
So, how the server is implemented, it should behave the same way, whether you send the request from the browser or JMeter.
It depends on implementation. From JMeter's perspective API endpoint is yet another URL, JMeter sends a request to it, measures time between request and response and marks result as passed or failed depending whether HTTP Status Code is below 400 or not.
So it is up to you to check:
What does API endpoint actually do
What is correct request syntax (mandatory arguments, headers, cookies, URL parameters, whatever)
What is the expected result.
Optionally, what happens if "bad" request is being sent.
When designing a JMeter test always run it with 1-2 users and View Results Tree listener enabled to ensure that it does what it is supposed to be doing.
Coming back to your question: if HTTP response code is below 400, JMeter will mark sampler as successful, it won't check response body or database so I would recommend using the following test elements for confirmation:
JDBC PostProcessor - to check whether database was updated as a result of the request or not.
Response Assertion - to check that API response doesn't have errors, status code, variables, returned from the database, etc.
I've created and application and a paginated api which is hooked up to each other. However i'm a bit confused on what is best practice in terms of only showing updated data. For instance if i retrieve data one day and save it into my mobile database. How will the app the next day know that it should make a request and only show that particular data that just has been fetched from the database. Do i need to make somekind of flag or look at createdAt?
When making the request, include either the If-None-Match header with the local resource's ETag or the If-Modified-Since header with the date the local resource was requested.
Configure your server to look for the header and return a 304 Not Modified if the data hasn't changed. That will at least save you some traffic on the responses.
In addition, if the resource data is relatively static, or if the client can tolerate having stale client data, then you can add caching headers to your response. As long as the cached request is valid, the request will never leave your client.
Ideally, you want do design your API to support this where possible. For example, have the request "give me all things in 50 meters" return a list of URIs. Then the API will only have to hit the server for those URIs which are stale.
Assume following scenario:
Client is sending HTTP POST to server
Request is valid and
have been processed by server. Data has been inserted into database.
Web application is responding to client
Client meets timeout
and does not see HTTP response.
In this case we meet situation where:
- client does not know if his data was valid and been inserted properly
- web server (rails 3.2 application) does not show any exception, no matter if it is behind apache proxy or not
I can't find how to handle such scenario in HTTP documentation. My question are:
a) should client expect that his data MAY be processed already? (so then try for example GET request to check if data has been submitted)
b) if not (a) - should server detect it? is there possibility to do it in rails? In such case changes can be reversed. In such case i would expect some kind of expection from rails application but there is not...
HTTP is a stateless protocol: Which means by definition you cannot know on the client side that the http-verb POST has succeeded or not.
There are some techniques that web applications use to overcome this HTTP 'feature'. They include.
server side sessions
cookies
hidden variables within the form
However, none of these are really going to help with your issue. When I have run into these types of issues in the past they are almost always the result of the server taking too long to process the web request.
There is a really great quote to that I whisper to myself on sleepless nights:
“The web request is a scary place, you want to get in and out as quick
as you can” - Rick Branson
You want to be getting into and out of your web request in 100 - 500 ms. You meet those numbers and you will have a web application that will behave well/play well with web servers.
To that end I would suggest that you investigate how long your post's are taking and figure out how to shorten those requests. If you are doing some serious processing on the server side before doing dbms inserts you should consider handing those off to some sort of tasking/queuing system.
An example of 'serious processing' could be some sort of image upload, possibly with some image processing after the upload.
An example of a tasking and queuing solution would be: RabbitMQ and Celery
An example solution to your problem could be:
insert a portion of your data into the dbms ( or even faster some NoSQL solution )
hand off the expensive processing to a background task.
return to the user/web-client. ( even tho in the background the task is still running )
listen for the final response with ( polling, streaming or websockets) This step is not a trivial undertaking but the end result is well worth the effort.
Tighten up those web request and it will be a rare day that your client does not receive a response.
On that rare day that the client does not receive the data: How do you prevent multiple posts... I don't know anything about your data. However, there are some schema related things that you can do to uniquely identify your post. i.e. figure out on the server side if the data is an update or a create.
This answer covers some of the polling / streaming / websockets techniques you can use.
You can handle this with ajax and jQuery as the documentation of complete callback explains below:
Complete
Type: Function( jqXHR jqXHR, String textStatus )
A function to be called when the request finishes (after success and error callbacks are executed). The function gets passed two arguments: The jqXHR (in jQuery 1.4.x, XMLHTTPRequest) object and a string categorizing the status of the request ("success", "notmodified", "error", "timeout", "abort", or "parsererror").
Jquery ajax API
As for your second question, is their away to handle this through rails the answer is no as the timeout is from the client side and not the server side however to revert the changes i suggest using one of the following to detect is the user still online or not
http://socket.io/
websocket-rails
I have an app running where a socket connection is constantly maintained (using socket.io). Data that needs to be sent is similar to that which you might see in a chat application. Would it be better to have it sent through POST (essentially, post that data, prevent page redirect, and then return the new page state with websockets), or just send it through websockets? What are the advantages to each?
(You might want to explain what you're trying to accomplish in more detail. Do you want to implement chat-like functionality).
A WebSocket gives you a TCP-like connection protocol over an HTTP connection. It's full duplex and lets you push and pull content in both directions. The connection is initiated from HTTP which "upgrades" the connection type. It gives you flexibility with some added complexity. I don't think it works across old HTTP 1.0 proxies.
A simple HTTP POST is more brute force. Unless you use ajax-ish techniques it pushes data to a web service and responds with a new web page to replace whatever's in your browser.