One of my application is using Parse.com as its backend service. There is one table called Product which is queried through cachePolicy as kPFCachePolicyCacheElseNetwork. The problem is the client side always got the cached data even after I modified some of the fields. The reason I don't always get data through network is that I'm trying to save data traffic as much as possible.
My question is if there is a way to expire the cache in server side so that I'll get new data in client side as soon as I modify the data? Thanks (My only solution by far is to delete the client app and reinstall it.This obviously is not an ideal solution.)
You need to decide some time limit on the cache validity, generally on the client, and either call clearCachedResult on the query instance or clearAllCachedResults on PFQuery when the limit is exceeded.
You could create a cloud function which returns a minimal amount of data and informs the app about changes so it can decide how / when / which caches to remove. For example, you pass a list of class names and last requested dates and the cloud function returns the names of classes which have new data since those dates.
Related
Currently we are using Breeze.js and Angular to develop our applications. Due to some persistent legacy issues, we have two databases ('Kenya' and 'Rwanda') that cannot be merged at this time, but have the same schema and metadata. Most of the time, the client knows which database to hit and passes the request through the .withParameters() function or the .saveOptions() function. Sometimes we want to request the same query from both databases (for example, if we are requesting a list of all available countries), and we use a EntityManager wrapper on the client to manage this and request the same query from each database. This is implemented through a custom EFContextProvider which uses the data returned to determine the appropriate database and creates the appropriate context in CreateContext().
To further complicate things, in some instances one or the other database won't exist (these are local deployments created through filtered replication), but the client won't know this. Therefore, when querying for a list of all countries, it issues two requests and one will cause failures because the context cannot be instantiated properly.
This is easy enough to detect on the Server. What I would like to do is to detect whether the requested context is available and, if not, return a 200 response and an empty set.
I can detect this in the Breeze DBContextProvider CreateContext() method, but cannot figure out how to cause the request to fallback gracefully to a empty-set response.
Thanks
Not exactly what I was looking for, but it probably makes more sense since most of the work is being done on the client-side:
Instead of trying to change the controller, I added a getAvailableDatabases to the C# controller actions and use that to determine which of the databases I will query from the client.
I am looking for solution of logging data changes for public API.
There is a need to tell client app which tables form db has changed and need to be synchronised since the app synchronised last time and also need to be for specific brand and country.
Current Solution:
Version table with class_names of models which is touched from every model on create, delete, touch and save action.
When we are touching version for specific model we also look at the reflected associations and touch them too.
Version model is scoped to brand and country
REST API is responding to a request that includes last_sync_at:timestamp, brand and country
Rails look at Version with given attributes and return class_names of models which were changed since lans_sync_at timestamp.
This solution works but the problem is performance and is also hard to maintenance.
UPDATE 1:
Maybe the simple question is.
What is the best practice how to find out and tell frontend apps when and what needs to be synchronized. In terms of whole concept.
Conditions:
Front end apps needs to download only their own content changes not whole dataset.
Does not invoked synchronization when application from different country or brand needs to be synchronized.
Thank you.
I think that the best solution would be to use redis (or some other key-value store) and save your information there. Writing to redis is much faster than any sql db. You can write some service class that would save the data like:
RegisterTableUpdate.set(table_name, country_id, brand_id, timestamp)
Such call would save given timestamp under key that could look like i.e. table-update-1-1-users, where first number is country id, second number is brand id, followed by table name (or you could use country and brand names if needed). If you would like to find out which tables have changed you would just need to find redis keys with query "table-update-1-1-*", iterate through them and check which are newer than timestamp sent through api.
It is worth to rmember that redis is not as reliable as sql databases. Its reliability depends on configuration so you might want to read redis guidelines and decide if you would like to go for it.
You can take advantage of the fact that ActiveModel automatically logs every time it updates a table row (the 'Updated at' column)
When checking what needs to be updated, select the objects you are interested in and compare their 'Updated at' with the timestamp from the client app
The advantage of this approach is that you don't need to keep an additional table that lists all the updates on models, which should speed things up for the API users and be easier to maintain.
The disadvantage is that you cannot see the changes in data over time, you only know that a change occurred and you can access the latest version. If you need to track changes in data over time efficiently, than I'm afraid you'll have to rework things from the top.
(read last part - this is what you are interested in)
I would recommend that you use the decorator design pattern for changing the client queries. So the client sends a query of what he wants and the server decides what to give him based on the client's last update.
so:
the client sends a query that includes the time it last synched
the server sees the query and takes into account the client's nature (device-country)
the server decorates (changes accordingly) the query to request from the DB only the relevant data, and if that is not possible:
after the data are returned from the database manager they are trimmed to be relevant to where they are going
returns to the client all the new stuff that the client cares about.
I assume that you have a time entered field on your DB entries.
In that case the "decoration" of the query (abstractly) would be just to add something like a "WHERE" clause in your query and state you want data entered after the last update.
Finally, if you want that to be done for many devices/locales/whatever implement a decorator for the query and the result of the query and serve them to your clients as they should be served. (Keep in mind that in contrast with a subclassing approach you will only have to implement one decorator for each device/locale/whatever - not for all combinations!
Hope this helped!
I have a application need a list of data, but these data may be very large. If I'm going to show this list of data in client (mobile app), I can't get all of the data from server because the limit space of mobile.
For example, like Facebook app, there are tons of newsfeed in server, and user can only see some of them. If user want to see more, they need to scroll down and fresh. So how to implement something like this in both client and server? (Currently my server is written in ruby on rails, and client is iOS)
And once the client get those data, does it store in memory or in local database? I'm worried about memory limit in mobile phones.
Thanks
On the server-side, you could probably write an API supporting pagination and custom results count, i.e.: myapp.com/api/get?start=0&count=20 to get the first 20 results, and when the user scrolls all the way down your view on the iPhone, fetch the next items, like that: myapp.com/api/get?start=20&count=20.
If you plan your design well, you'll get something very flexible that you'll be able to change later if you realize that 20 results is too much/not enough, etc.
Depending on your app's architecture and the amount of data your app will handle, you might also need to provide API methods based on the last-updated time, to ensure you're not missing data (e.g., if you call your second get?start=20 a few minutes after the first one, the start index might not have the same meaning).
As for storing data locally, it all depends on what you want to achieve. Are you sure you need to save everything the user has downloaded? You could store only the most recently fetched items in a local SQLite database and query them the next time your app starts up, before refreshing the view (I don't know how it is implemented in Facebook's iPhone app but at least it looks like it's done that way).
In an app I'm building, I'm using Core Data to cache remote content from an API for offline viewing. This all works pretty well except for one big issue: if a record on the server is deleted there's no way for me to detect that and delete its cached counterpart.
The only thing I can think of is somehow marking all the current data as 'invalid' when I pull data from the API and only mark the records returned by the API as valid again, but it seems like this is a clunky solution to the problem. Additionally, as data from the API I'm using is paginated it doesn't scale well for lots of records.
So what I want to know is: is there a better way to invalidate local cache data in response to it being deleted server-side?
I would suggest, although not the easiest route, is to have the server side cache items that are deleted and expose an endpoint you can call to get the deleted items. In a perfect world right.
What you can do is in a background thread, download all the data from the server and compare it to what you have locally. So instead of just invalidating all of it and re-parsing it back in (which can take time for large data sets), just run through and compare id's of objects on the server to your objects in CoreData. If it's there great, if not delete it from you local db. Hope this helps.
I'm going to attempt to create an open project which compares the most common MP3 download providers.
This will require a user to enter a track/album/artist name i.e. Deadmau5 this will then pull the relevant prices from the API's.
I have a few questions that some of you may have encountered before:
Should I have one server side page that requests all the data and it is all loaded simultaneously. If so, how would you deal with timeouts or any other problems that may arise. Or should the page load, then each price get pulled in one by one (ajax). What are your experiences when running a comparison check?
The main feature will to compare prices, but how can I be sure that the products are the same. I was thinking running time, track numbers but I would still have to set one source as my primary.
I'm making this a wiki, please add and edit any issues that you can think of.
Thanks for your help. Look out for a future blog!
I would check amazon first. they will give you a SKU (the barcode on the back of the album, I think amazon calls it an EAN) If the other providers use this, you can make sure they are looking at the right item.
I would cache all results into a database, and expire them after a reasonable time. This way when you get 100 requests for Britney Spears, you don't have to hammer the other sites and slow down your application.
You should also make sure you are multithreading whatever requests you are doing server side. Curl for instance allows you to pull multiple urls, and assigns a user defined callback. I'd have the callback send a some data so you can update your page with as the results come back. GETTUNES => curl callback returns some data for each url while connection is open that you parse it on the client side.